xdt99: New TI 99 cross-development tools available

[ralphb]

3 hours ago, Lee Stewart said:

The XORG range is bracketed with labels LLVSPT above it and LLVEND below it.

Sorry, I had missed that (also ignored the comment saying so).

 

I would suggest the following: Send me the entire modified code (here or PM), and I'll try to make it work with XORG.  You did everything right, but maybe a combination of things breaks the program.

[+Lee Stewart]

15 minutes ago, ralphb said:

Sorry, I had missed that (also ignored the comment saying so).

 

I would suggest the following: Send me the entire modified code (here or PM), and I'll try to make it work with XORG.  You did everything right, but maybe a combination of things breaks the program.

 

I was wrong about the value of one of the labels. LLVSPT was OK, but, after checking the proper symbol list (I had too many balls in the air—still do!), LLVEND was, indeed, a low RAM value. I corrected the misinformation in my previous post. Putting the label right on AORG fixed that problem. 

 

There was one remaining problem, viz., a small, 8-byte piece of code that needed to be executed from ROM was near the end of, but inside(!), the XORGed code. I had accounted for that in my old code, but replacing all of my calculated labels by using XORG, caught that one in its web. I am not sure why it was there in the first place, except that it had to be in Bank 1—probably got lost in previous widespread changes. It was being unnecessarily copied to low RAM, but never executed from there, wasting 8 bytes of low RAM—not serious, but certainly sloppy of me! I moved it to the end of its file, well after the

LLVEND AORG
       BANK 1

 

Still needing to patch the next bank change (Bank 2 in this instance) notwithstanding, all seems to be well in ROM-land. It loads and detects SAMS. I have certainly not yet performed exhaustive testing, but all is well so far. Thanks for putting up with me.

 

...lee

[+Lee Stewart]

One operation I may try to get xas99 to do for me as part of the TEXT directive is to add a value to the last byte. This is not as bizarre as it may seem at first blush. The last byte is already modifiable by negating the string, which negates only the last byte. My wanting to modify the last TEXT byte has more to do with code readability than program function, but certainly would be useful to me. When I was converting TI Forth code to fbForth, it was very frustrating trying to read the name field of a Forth word’s header when all you could see was a DATA directive. Modified first and last bytes made it even less readable:

L102F DATA >8654,>4F47,>474C,>45A0     ; 6 "TOGGLE " ..padded with a space to make even

The first byte (name length) and last byte have a termination bit (>80) added, which is what makes them difficult to read, even if one had the ASCII codes memorized.

 

The most readable form would, of course, be TEXT, but the only modification allowed is negation. I even though of modifying fbForth to recognize a negated last byte as the termination signal, but that would take finding everywhere in the code the terminator bit is used (not terribly difficult) as well as fundamentally changing the language (not too crazy about that). The most readable solution I could envision was stringing bytes together in DATA directives and adding EQUated values for the terminator and precedence (length byte only) bits:

TERMBT EQU  >0080           ; Terminator bit
PRECBT EQU  >0040           ; Precedence bit
LSHFT8 EQU  >0100           ; Multiplier to get low byte shifted into high byte

DSCD_N DATA 10+TERMBT+PRECBT*LSHFT8+'D','OE','S>','CO','DE',': '+TERMBT   ; 10 "DOES>CODE: "
DSAS_N DATA 9+TERMBT+PRECBT*LSHFT8+'D','OE','S>','AS','M:'+TERMBT         ;  9 "DOES>ASM:"
SCD__N BYTE 5+TERMBT+PRECBT                                               ;  5 ";CODE"
       TEXT ';COD'
       BYTE 'E'+TERMBT

As you can see with “;CODE”, not even that would work all the time.

 

Here is what I would like to modify xas99 to allow:

terminator_bit equ  >80           ; Terminator bit
precedence bit equ  >40           ; Precedence bit

dscd_n byte 10 + terminator_bit + precedence bit
       text 'DOES>CODE: ' + terminator_bit          ; '+' operates on only the last byte, same as unary '-'
dsas_n byte 9 + terminator_bit + precedence bit
       text 'DOES>ASM:' + terminator_bit
scd__n byte 5 + terminator_bit + precedence bit
       text ';CODE' + terminator_bit

...just thinking out loud. Don’t mind me. And now—back to your regularly scheduled program....

 

...lee

[+TheBF]

In some modern Forth sources a lot people use a HEADER macro to create stuff like this.

Does xas99 have macros?

 

[+Lee Stewart]

14 minutes ago, TheBF said:

In some modern Forth sources a lot people use a HEADER macro to create stuff like this.

Does xas99 have macros?

 

It does.

 

...lee

[ralphb]

2 minutes ago, Lee Stewart said:

It does.

Yes, I also immediately thought of macros.

 

The problem, though, is that macros are "semantic", not "syntactic", and there is also no loop construct available.  But I wanted to improve the macro system anyway (not sure when, though).

[+Lee Stewart]

1 hour ago, TheBF said:

In some modern Forth sources a lot people use a HEADER macro to create stuff like this.

Does xas99 have macros?

 

40 minutes ago, ralphb said:

Yes, I also immediately thought of macros. The problem, though, is that macros are "semantic", not "syntactic", and there is also no loop construct available.  But I wanted to improve the macro system anyway (not sure when, though).

 

Perhaps my imagination is not active enough, but I cannot think of anything for a macro that would not increase the code real estate—and that I cannot afford! There are only 158 bytes free in the dictionary-header bank and there are 500+ words in there. Even one more byte/word would blow it up.

 

...lee

[+TheBF]

You are correct. The macros won't make anything smaller.

They simply might reduce the amount text required to create the data structures in the source code.

The space difference should be net zero in my little mind.

[+Lee Stewart]

With macros, the following is the best I can contrive:

terminator_bit equ >80
precedence_bit equ >40

   .defm name_field
       byte #1 + terminator_bit
       text #2
       byte #3 + terminator_bit
   .endm

   .defm name_field_immediate
       byte #1 + terminator_bit + precedence_bit
       text #2
       byte #3 + terminator_bit
   .endm

;*** DOES>ASM: ***
dsas_n:
       .name_field_immediate 9, 'DOES>ASM',':'

;[*** +/*OPERATOR ***
ssop_n:
       .name_field 11, '+/*OPERATO','R'

with listing:

Spoiler

0001      0080     terminator_bit equ >80
0002      0040     precedence_bit equ >40
0003               
0009               
0015               
0016               ;*** DOES>ASM: ***
0017               dsas_n:
0001 0000 C9              byte 9 + terminator_bit + precedence_bit
0002 0001   44            text 'DOES>ASM'
     0002 4F45     
     0004 533E     
     0006 4153     
     0008 4D       
0003 0009   BA            byte ':' + terminator_bit
0019               
0020               ;[*** +/*OPERATOR ***
0021               ssop_n:
0001 000A 8B              byte 11 + terminator_bit
0002 000B   2B            text '+/*OPERATO'
     000C 2F2A     
     000E 4F50     
     0010 4552     
     0012 4154     
     0014 4F       
0003 0015   D2            byte 'R' + terminator_bit

 

 

The source is not terrible, but having the last character of the name always separated by ',' is irritating to my sensibilities, dontcha know. The listing, of course, is a bit worse, with several lines separating the last character from the rest of the name.

 

...lee

[+TheBF]

It's not bad for sure.  But if it impedes your enjoyment of the source code that's always a problem. I guess only you can decide.

Pretty hi-tech for TI-99.  

 

[HOME AUTOMATION]

55 minutes ago, Lee Stewart said:

The source is not terrible, but having the last character of the name always separated by ',' is irritating to my sensibilities, dontcha know.

A feeling I know, only too-well!

[+Lee Stewart]

On 8/29/2020 at 3:48 PM, Lee Stewart said:

The source is not terrible, but having the last character of the name always separated by ',' is irritating to my sensibilities, dontcha know. The listing, of course, is a bit worse, with several lines separating the last character from the rest of the name.

 

On 8/29/2020 at 3:53 PM, TheBF said:

It's not bad for sure.  But if it impedes your enjoyment of the source code that's always a problem. I guess only you can decide.

Pretty hi-tech for TI-99.

 

Well—I finally managed it with a patch to xas99! I added unary ‘~’ as an allowable operation on a TEXT string and had the implementation code OR the terminator bit (>80) to the last character instead of inverting it. Aesthetically, it is a little shabby to co-opt one operation to perform another, but I figured there was no harm in doing it for a previously disallowed operation. It would be better to implement a proper addition operation, but I am new to Python and it was easier to do what I did. ? Of course, now I need to keep a copy of the modified xas99 with my code.

 

Now I can do this:

; In the two macros below, '~' is used to signal Ralph Benzinger's xas99.py
; assembler, with my modification to do it, to OR the terminator_bit (>80)
; with the last character in a TEXT string.

terminator_bit equ >80
precedence_bit equ >40

   .defm name_field
       byte #1 + terminator_bit
       text ~#2      ; OR terminator_bit (>80) with last char
   .endm

   .defm name_field_immediate
       byte #1 + terminator_bit + precedence_bit
       text ~#2      ; OR terminator_bit (>80) with last char
   .endm

;*** DOES>ASM: ***
dsas_n:
       .name_field_immediate 9, 'DOES>ASM:'

;[*** +/*OPERATOR ***
ssop_n:
       .name_field 11, '+/*OPERATOR'
       
       end

 

which results in what is in this spoiler:

Spoiler

XAS99 CROSS-ASSEMBLER   VERSION 3.0.0
     **** ****     > testNameField.a99
0001               ; In the two macros below, '~' is used to signal Ralph Benzinger's xas99.py
0002               ; assembler, with my modification to do it, to OR the terminator_bit (>80)
0003               ; with the last character in a TEXT string.
0004               
0005      0080     terminator_bit equ >80
0006      0040     precedence_bit equ >40
0007               
0012               
0017               
0018               ;*** DOES>ASM: ***
0019               dsas_n:
0001 0000 C9              byte 9 + terminator_bit + precedence_bit
0002 0001   44            text ~'DOES>ASM:'      ; OR terminator_bit (>80) with last char
     0002 4F45     
     0004 533E     
     0006 4153     
     0008 4DBA     
0021               
0022               ;[*** +/*OPERATOR ***
0023               ssop_n:
0001 000A 8B              byte 11 + terminator_bit
0002 000B   2B            text ~'+/*OPERATOR'      ; OR terminator_bit (>80) with last char
     000C 2F2A     
     000E 4F50     
     0010 4552     
     0012 4154     
     0014 4FD2     
0025               
0026                      end

    DSAS_N.............. >0000 : REL 
    PRECEDENCE_BIT...... >0040 :     
    SSOP_N.............. >000A : REL 
    TERMINATOR_BIT...... >0080 :     

 

 

Now to see if I can contrive a replacement macro.

 

...lee

[+TheBF]

Very clever.  It makes the code look pretty too.  

[+TheBF]

I am curious.

Why is there a terminator bit and count byte?

Could not the count byte define the string length?

[+Lee Stewart]

12 hours ago, TheBF said:

I am curious.

Why is there a terminator bit and count byte?

Could not the count byte define the string length?

 

There are actually two occurrences of the terminator bit, at the beginning (length byte) and end of the name field. It comes from figForth, which, as you know, is the major source for TI Forth and fbForth. So, I am kind of stuck with it—or, at least, I thought I was. I might be able to change that without breaking much old code. I kind of doubt anyone actually used TRAVERSE on purpose, which is the only word other than CREATE that actually makes use of the terminator bits—and, CREATE only uses them to set them up for TRAVERSE . The only words I can think of that use TRAVERSE are NFA and PFA . Any word that searches the name field must deal with the terminator bits. If I dispense with either one of them, I will need to look carefully for the words that will be affected—probably only a handful. Except for finding the impacted words, removing the trailing terminator bit would be nearly inconsequential. The leading terminator bit, though?—enter the dissertation in the spoiler below at your own risk:

Spoiler

In figForth, the four fields that comprise a Forth word are Link, Name, Code and Parameter and their addresses are referenced as lfa, nfa, cfa and pfa, respectively. (The “fa” in the preceding abbreviations stands for Field Address. The first would be Link Field Address, etc.) The link field points to the previous word’s name field. The word LATEST points to the last-defined word’s name field and is the starting point for dictionary searches. The word tick ( ' ) uses -FIND to search for a word in the dictionary by starting with LATEST to search the last word’s name field and walking back through the dictionary using the lfa of the current word by simply subtracting 2 from its nfa. If the word is found, tick leaves that word’s pfa on the stack.

 

If you want to find a word’s lfa, nfa or cfa with words, LFA , NFA or CFA , respectively, you need its pfa. The cfa is easy—simply subtract 2 from its pfa. The other two lie on the other side of the name field and the terminator bit in the length byte is the signal TRAVERSE uses to find it. Its address is the nfa and two before that is its lfa. The only other way to perform this search is to search the entire dictionary starting at LATEST and use the nfa it contains (points at the length byte of last-defined word) because name fields have variable lengths. Then, you would increment the length byte and force it to an even number by masking off the low bit, add that to the nfa to get to the word’s cfa, add 2 to get the pfa and check whether it matches the pfa on the stack. If it does not match, the current nfa is decremented by 2 to the lfa to get the nfa of the previous word in the dictionary, continuing with the above process until failure or the pfa is found. If the pfa is found, you now have whichever of lfa or nfa you needed. It is a fairly quick process because only one comparison per word is actually made, even though you are potentially searching the whole dictionary (500+ words in the fbForth 2.0 resident dictionary), whereas TRAVERSE makes a maximum of 32 comparisons (maximum name length is 31 characters) because the pfa on the stack does, after all, already point to the word of interest. On average, Forth words are much shorter than 31 characters, so TRAVERSE does not work very hard to discover the lfa or nfa of interest.

 

On the other hand, to get a word’s pfa with the word, PFA , you need its nfa on the stack. Obviously, you can use the length byte to immediately calculate the address of the parameter field! The terminator bit at the end of the name field seems an unnecessary complication—perhaps symmetry was important to the designer(s), who knows.

 

I went through all of this discussion, particularly the second paragraph back, to give you a feel for the trouble involved in removing the terminator bit from the beginning of the name field so that I could now reveal that it is not really that much trouble, after all! You see, almost all of the 500+ words in the resident dictionary are in ROM with their headers (lfa, nfa and pfa-pointer) in a different ROM bank from their cfas and pfas, which complicates the above searches a little. For all words in the RAM part of the dictionary (a handful of resident + all user words), the above discussion still obtains. BUT—for all the words in ROM, it goes like this: CFA works exactly the same because the cfas and pfas are in the same bank. PFA works almost the same, except that the address you find from the nfa is not the pfa, but rather, a pointer to the pfa. You simply need the extra step of de-referencing the pointer. If the pointer to the pfa is in R0:

       MOV  *R0,R0

will get the job done.

 

However, for LFA and NFA , we need to first find in ROM the pointer to the pfa supplied on the stack, which involves starting at the top link in ROM that points to the name field of the last-defined word in ROM. Here is how the linked list is arranged in ROM, showing the last two Forth word headers:

;*** >MAP ***       ( bank addr --- )
       DATA S0TS_N
TOMP_N DATA 4+TERMBT*LSHFT8+'>','MA','P '+TERMBT
       DATA TOMAP+2        ; pfa ptr
;*** VLIST ***      ( --- )
       DATA TOMP_N
VLST_N DATA 5+TERMBT*LSHFT8+'V','LI','ST'+TERMBT
       DATA VLIST+2        ; pfa ptr  
DCTPTR DATA VLST_N         ; last nfa ptr in this bank (a pseudo lfa, if you will)

This search starts by decrementing by 2 the register, say R0, holding DCTPTR (the last address above—consider it an lfa) to get to the last pfa pointer. If that contains the pfa we are seeking, we get back the previous lfa and de-reference it to get the nfa. If, instead, we are seeking the lfa, we simply decrement it by 2 and we are done. Not finding the pfa, we still get back to the previous lfa, but this time we decrement it by 4 to get the previous word’s pfa pointer, rinse and repeat. What you should notice here is that this procedure is a little faster than the one described a few paragraphs back. The point I wish to make is that removing the terminator bit from the length byte at the beginning of the name field would not be quite as catastrophic for dictionary searching as I at first implied—still a good bit longer for RAM dictionary searches than with a leading terminator bit—but not as long as I led you to believe because ROM searching is the same for either scenario! 

 

The take-homes here are 

• Ending terminator bit removal should be pretty simple—a lot of work, but ultimately not onerous. This would obviate the necessity for the ‘~’ modification I made to xas99.py.
• Beginning terminator bit removal, though not difficult to implement, would cost more in dictionary search time for words, LFA and NFA in RAM parts of the dictionary.

...lee

[+TheBF]

That dissertation is worth another degree.

 

So we had this discussion around finding fields in the Camel Forth header.  

I don't fully understand why the terminator bit at the end is needed but it sounds like the one in the length byte has a purpose.

 

Anton Ertl who shepherds gforth has moved to a name field based header. Searching is done for the name field and everything is accessible from the nfa.

It's pretty consistent and it means that all the words that use stack strings (addr,len) can be brought to bear. FIND is deprecated and replaced with SEARCH-WORDLIST as I understand things.

 

It's the eternal argument in Forth building I suppose.

Don't break your code on my account. It just was an innocent question.  

[ralphb]

19 hours ago, Lee Stewart said:

Well—I finally managed it with a patch to xas99! I added unary ‘~’ as an allowable operation on a TEXT string and had the implementation code OR the terminator bit (>80) to the last character instead of inverting it.

 

Now to see if I can contrive a replacement macro.

That's clever, but also a bit dubious.  You'll have to reapply your change to every release xas99, or stay at this release forever (which would mean no bug fixes and no enhancements).

 

The biggest disadvantage, however, is that only you will be able to assemble your source files.  Maybe this is fine for you.  If not, you could supply a "patch" (not sure about the availability on Windows) to introduce your ~ into current versions of xas99.  Then other users could apply this patch to their xas99 versions in order to assemble fbForth.

 

EDIT: BTW, I really liked your earlier definition, which you deemed ugly.  For me, beauty is achieving our complicated goals with the limited means we have at our disposals, so your earlier macro qualifies as elegant. ?

Edited September 2, 2020 by ralphb

[+Lee Stewart]

7 hours ago, TheBF said:

That dissertation is worth another degree.

Hah!

 

7 hours ago, TheBF said:

So we had this discussion around finding fields in the Camel Forth header.  

I don't fully understand why the terminator bit at the end is needed but it sounds like the one in the length byte has a purpose.

 

I have pretty much decided to do away with the terminal  terminator bit. I just need to do some careful vetting of the code to avoid breaking something. I could even change TRAVERSE to use the length byte to get to the end of the name field. The problem with that would be any user code (or mine, for that matter) that uses carnal knowledge of that terminator bit.

 

I still have not decided whether to eliminate the leading terminator bit. The fact that it is not at all useful in the 500+ words in the resident dictionary in ROM pulls me toward eliminating it, but the resulting longer searches (LFA and NFA) in the RAM section of the dictionary pulls me in the other direction. I guess I am leaning toward keeping it.

 

...lee

[+Lee Stewart]

5 hours ago, ralphb said:

EDIT: BTW, I really liked your earlier definition, which you deemed ugly.  For me, beauty is achieving our complicated goals with the limited means we have at our disposals, so your earlier macro qualifies as elegant. ?

Thanks.

 

5 hours ago, ralphb said:

That's clever, but also a bit dubious.  You'll have to reapply your change to every release xas99, or stay at this release forever (which would mean no bug fixes and no enhancements).

 

The biggest disadvantage, however, is that only you will be able to assemble your source files.  Maybe this is fine for you.  If not, you could supply a "patch" (not sure about the availability on Windows) to introduce your ~ into current versions of xas99.  Then other users could apply this patch to their xas99 versions in order to assemble fbForth.

 

Yeah, that is definitely not the best way to manage it, even were I to package my patched version of xas99 with the source code. If I eliminate the trailing terminator bit from dictionary headers, I will then only need the use of ‘$’ anywhere in labels longer than one character and the option to allow cross-bank references. If these never become part of xas99, it will be easy enough to include a note on how to patch xas99 with just those changes. I should include a note that the new code can only be assembled by xas99, anyway.

 

...lee

[+Lee Stewart]

I hit a minor, unexpected snag with macros in xas99. A label on the same line as a macro does not appear in the listing. The label reference is correct, but I like seeing a label in context. Furthermore, A missing label wreaks havoc with listing searches. I said it was a minor snag because I should be able to fix it with a regex search and replace, but I do not think I should have to do that. Besides, it adds another 500+ lines to the listing (1000+, if I do it for commented versions for consistency). I guess I should quit my bellyaching, though, since my listing is already more than 31,000 lines. At least, the required regular expressions will not be nearly as involved as the ones required for the name-field replacements I managed.

 

...lee

[ralphb]

On 9/4/2020 at 6:39 AM, Lee Stewart said:

I hit a minor, unexpected snag with macros in xas99. A label on the same line as a macro does not appear in the listing. The label reference is correct, but I like seeing a label in context. Furthermore, A missing label wreaks havoc with listing searches. I said it was a minor snag because I should be able to fix it with a regex search and replace, but I do not think I should have to do that. Besides, it adds another 500+ lines to the listing (1000+, if I do it for commented versions for consistency). I guess I should quit my bellyaching, though, since my listing is already more than 31,000 lines. At least, the required regular expressions will not be nearly as involved as the ones required for the name-field replacements I managed.

 

...lee

The problem is that the preprocessor handles everything with a '.' before the assembly passes start.  So the labels of all the the macro instantiation would appear before source in the list file.

 

 Also, when writing something like this,

   .defm m
!a data 1
   .endm
   
x  .m

you'd have a label conflict.

 

But I'm in the process of rewriting this.  You'll have one comment line with the "outer" label, and then the instantiation with the "inner" labels.  That'll also tell you which macro you invoked.

[+Lee Stewart]

Like I said, it should be simple enough to write a regex to find/replace

DATC_N .name_field_immediate 5, ']DATA'

to

DATC_N:
       .name_field_immediate 5, ']DATA'

 

...lee

[ralphb]

54 minutes ago, Lee Stewart said:

Like I said, it should be simple enough to write a regex to find/replace

DATC_N .name_field_immediate 5, ']DATA'

to

DATC_N:
       .name_field_immediate 5, ']DATA'

 

...lee

That probably wouldn't help, since both lines are equivalent.  I'm hedging my statement since the preprocesser happens before label conversion, so I don't actually know.  But I wanted to rewrite the macro handling anyway.

[+Lee Stewart]

4 hours ago, ralphb said:

That probably wouldn't help, since both lines are equivalent.  I'm hedging my statement since the preprocesser happens before label conversion, so I don't actually know.  But I wanted to rewrite the macro handling anyway.

 

Well—at least, it appears inline in the listing the second way (the first does not) and has the correct value (>7302) in the symbol table ?:

2455               DATO_N:
0001 7302 C5              byte 5 + terminator_bit + precedence_bit
0002 7303   44            text 'DATA['
     7304 4154     
     7306 415B     

...lee

[ralphb]

I've released version 3.1.0 of xdt99, in which I only updated xas99.  The new features, which are partially incompatible with 3.0.0, are:

 

BANKS.  To define a bank, use BANK <b>, [<addr>].  If an address is given, it sets the base address for other BANK directives without address.  Again, BANK ALL starts at the lowest address after all other banks, and raises the base address.

 

SAVES.  I've reverted the "minimal" address range when using SAVE and now generate values for the entire range.  For example, a SAVE >2000,>4000 yields a file of 8K.  To get the previous behavior back, use the minimize option -M.

 

AUTO.  A new directive AUTO determines the program location where all auto-generated constants should be places.  Omitting AUTO is an error.  Auto-constants are now bank-aware, so AUTO should be placed in each bank where auto-constants are used.  After AUTO, no further auto-constants must be used.

 

BANK CROSS-CHECKS.  I now disabled cross-checks by default.  To enable them, use -X.

 

MACROS.  Labels should now be retained.  Also, the listing now contains information about source units/macros entered and resumed, e.g.

     **** **** > source1.asm
               COPY "source2.asm"
     **** **** > source2.asm
               ...
               < source1.asm
               ...
               .mac r0, r1
     **** **** > MAC
               ...
               < source1.asm

COLORS.  The output of warnings and errors now uses color.  On Linux and macOS, color is on by default.  Since older version of Windows don't support so-called ANSI esc sequences, it's off by default.  To enable/disable color manually, use --color on/off.

 

MISC.  Some additional warnings about incorrect operand usage, and more.