fbForth fbForth—TI Forth with File-based Block I/O [Post #1 UPDATED: 06/09/2023]

[Willsy]

And we call this "a hobby"!

 

Hmmmm....!

[+Lee Stewart]

Floppy boot tracking is generally do-able through some DSR gymnastics. Honestly, I tend to use a simple configuration file to avoid pitfalls of the various routines and devices.

 

One trick I've used is a dedicated image file in the load chain specifically and only containing the path(s) and configuration data. Because it is in the chain it loads automatically; and since it is a simple file, I can use LOAD/SAVE image (opcodes 05/06) to save/load the data to/from disk. Then if the program cannot find the files, I display the currently stored path and request the user point to the proper files. Once a path is provided I allow the user to continue or to save the path back to the file.

 

Might not 's suggestion in his Current Device thread work just fine, viz., check the full pathname at VRAM=100Ah before something in my program changes it?

 

...lee

[+InsaneMultitasker]

Might not 's suggestion in his Current Device thread work just fine, viz., check the full pathname at VRAM=100Ah before something in my program changes it?

 

...lee

For that specific application, it would probably work. However, the pathname and pabs can exist "anywhere" in vram. The dinosaur disk used the PAB pointer and a few related scratchpad pointers to find the last PAB. This method may be your best option though from Tursi's and my digging, there seem to be flaws to overcome.

[+Lee Stewart]

For that specific application, it would probably work. However, the pathname and pabs can exist "anywhere" in vram. The dinosaur disk used the PAB pointer and a few related scratchpad pointers to find the last PAB. This method may be your best option though from Tursi's and my digging, there seem to be flaws to overcome.

 

When I set a breakpoint in Classic99 at fbForth's startup label (BOOT), the PAB for the load is, indeed, at 1000h, but the filename-length pointer (8356h) is now pointing at the '.' of "DSK1.FBFORTH". I guess that means the DSR (or DSRLNK) meddles with 8356h while doing its thing?

 

Also, 831Ch (per Rich's comment in Current device) is pointing at the beginning of the same PAB (1000h in this case) at this breakpoint. Do you think it's safe for me to use this pointer or just assume that 1000h is the PAB location for this load, seeing as how E/A is loading only one program?

 

...lee

[Tursi]

I don't think it's safe to use the pointer, and Classic99 will eventually enforce my belief. The only time that pointer is guaranteed is on ENTRY to the DSR, the DSR is allowed to modify it. Which means on DSR return, technically, anything goes.

 

Today Classic99 doesn't change the pointers, so they are left just as they are on entry to the DSR. It's DSRLNK that moves the pointer to the '.', not the disk DSR. (The TI disk controller does NOT leave this pointer there!)

 

I think for moving forward, when we have many choices in file systems on the TI and by all indications, more to come, we should avoid relying on side-effects specific to the floppy controller.

Edited July 16, 2013 by Tursi

[+Lee Stewart]

I don't think it's safe to use the pointer, and Classic99 will eventually enforce my belief. The only time that pointer is guaranteed is on ENTRY to the DSR, the DSR is allowed to modify it. Which means on DSR return, technically, anything goes.

 

Today Classic99 doesn't change the pointers, so they are left just as they are on entry to the DSR. It's DSRLNK that moves the pointer to the '.', not the disk DSR. (The TI disk controller does NOT leave this pointer there!)

 

I think for moving forward, when we have many choices in file systems on the TI and by all indications, more to come, we should avoid relying on side-effects specific to the floppy controller.

 

But—in my case with fbForth as, currently, the one and only file I load with E/A, the first thing done after fbForth gets control and with no reliance on pointers, should I not be safe reading the PAB E/A uses at VRAM 1000h for the full pathname of fbForth entered by the user?

 

...lee

[Tursi]

But—in my case with fbForth as, currently, the one and only file I load with E/A, the first thing done after fbForth gets control and with no reliance on pointers, should I not be safe reading the PAB E/A uses at VRAM 1000h for the full pathname of fbForth entered by the user?

...lee

 

Yeah.. that's relying on a side-effect from Editor/Assembler, I suppose, and since the DSR isn't supposed to trash the PAB, it should be safe. The only caveat I'd say there is that it may prevent loading fbForth from anything other than Editor/Assembler, since another loader may place the PAB somewhere else.

 

The areas that Classic99 will trash as "unsafe to use" will be scratchpad locations known to be used by DSRs, and the VDP file buffers (ie: the reserved area at the top of VDP set by CALL FILES). The goal is to try and prevent new software created on Classic99 from relying on disk system side effects which may cause problems on hardware with different disk systems. Hopefully it will reduce the number of problems future code has moving from emulation to hardware.

[Willsy]

The areas that Classic99 will trash as "unsafe to use" will be scratchpad locations known to be used by DSRs, and the VDP file buffers (ie: the reserved area at the top of VDP set by CALL FILES). The goal is to try and prevent new software created on Classic99 from relying on disk system side effects which may cause problems on hardware with different disk systems. Hopefully it will reduce the number of problems future code has moving from emulation to hardware.

 

That would be cooler than a shed full of cucumbers! I got bit many times by Classic99's disk simulation - what worked fine on Classic99 didn't work on the real iron, and every time it was because of scratch-pad RAM usage. I had code in PAD that TF needs, and each invocation of a disk access would overwrite an instruction here or an instruction there and boom!

 

I'm ashamed to say I got bitten more than once by this during development. I had to rely on MESS and Bob Carmany to test on real iron. Luckily, it seems that MESS disk emulation is totally spot on. From what I've seen, if it works in MESS, it WILL work on the real iron.

 

WRT to loading fbForth (I think we should call it Forthly - as in ForthLee HA HA!!! ) I wonder where the PAB is if fbForth is loaded by the FunnelWeb loader in an EA and an XB environment? The classic99 debugger should be able to help there.

 

Unfortunately, I don't have Funnelweb, otherwise i'd try it.

[Asmusr]

Sorry to go slightly off topic. I found an 'all purpose' DSRLNK implementation here:

http://atariage.com/forums/topic/163692-ea-file-access/#entry2071618

Anyone knows what 'all purpose' means in this context? I'm a bit worried by the comment: SINCE WE KNOW WERE USING "DSKn."

 

I would like to use something like this because I want the entire area from >2000 to >3FFF for my own stuff, so I don't want to load any of the standard utilities.

It works well in Classic99 but not in MESS. Is the real floppy DSR using other memory in the >2000 area or what could be the problem?

[Willsy]

The floppy DSR doesn't use any memory in the low memory expansion. It uses some PAD memory, and also some VDP memory (the more file handles reserved (power-up default is 3) the more VDP ram is reserved at the top of VDP.

 

From power-up, VDP from >37D8 to >3FFF is in use by the TI disk DSR.

 

You are probably suffering from something in PAD being changed by the disk DSR operations. Where is your workspace? In TurboForth, I restore all of scratch pad to a known state after every disk operation.

 

I had exactly the same problems as you: TF was working in Classic99 (because the disk DSR in Classic99 isn't TI's DSR and doesn't use pad RAM) but wouldn't work in MESS. In my case, I had a trampoline function (a function that lets me change cartridge banks and jump to an address in the new bank) that was getting (IIRC) 2 bytes changed because that location was used by the DSR. That had the effect of writing a different instruction into the code-path of the trampoline function, and boom... Into the weeds. Took me ages to find it, and IIRC I needed the help of Michael Zapf to analyse what was going wrong (I wrongly assumed a bug in MESS since "hey, it works in Classic99!").

 

I've been using a DSRLNK routine written by Paolo Bagnaresi for years. Always worked without any problems.

 

I've attached the code, plus supporting equates/buffers as a text file. Shout if you have a problem integrating it into your application.

 

DSRLNK.txt

[Asmusr]

Thanks a lot, I will try your routine.

[Asmusr]

Thanks again Willsy, I finally got it to work. It turns out the problem wasn't the DSRLNK routines (both of them work) but the pointer in >8370 to the "Highest available address of VDP RAM". I have been using the scratch pad at my leisure without any concern about what the OS is expecting. After setting the pointer back to >37D7 it began to work. I also switch from >8300 to a workspace in upper memory and save the contents of scratch pad in a buffer before calling DSRLNK, and restore it afterwards as you suggested.

[Tursi]

The TI (indeed most) Disk DSR needs the >8370 pointer to be valid to know where to store its file buffer, so that would explain the difference. Classic99 doesn't use any TI memory to run disk so it doesn't care what the pointers are. I'm sorry that you also got bit by this, it's good incentive to get a version out that makes this problem more obvious for the programmer.

[Asmusr]

Thanks again Willsy, I finally got it to work. It turns out the problem wasn't the DSRLNK routines (both of them work) but the pointer in >8370 to the "Highest available address of VDP RAM". I have been using the scratch pad at my leisure without any concern about what the OS is expecting. After setting the pointer back to >37D7 it began to work. I also switch from >8300 to a workspace in upper memory and save the contents of scratch pad in a buffer before calling DSRLNK, and restore it afterwards as you suggested.

 

Well, my code now works on Classic99 and MESS but not on my console with a nanoPEB. All I'm trying to do is to read a 4k file as 32 records of 128 bytes from DSK1. I tried with two different DSRLNK routines that both work on the emulators. Any suggestions for what I might be missing? What should the >8370 pointer be set to for the nanoPEB?

[+Lee Stewart]

Well, my code now works on Classic99 and MESS but not on my console with a nanoPEB. All I'm trying to do is to read a 4k file as 32 records of 128 bytes from DSK1. I tried with two different DSRLNK routines that both work on the emulators. Any suggestions for what I might be missing? What should the >8370 pointer be set to for the nanoPEB?

 

The nanoPEB uses 8 additional bytes more than the TI floppy DSR. This would make the default value for 3 simultaneously open files 37CFh instead of 37D7h. This is not something you can set explicitly, however.

 

...lee

Edited July 19, 2013 by Lee Stewart

[+Lee Stewart]

It happens that the 8 additional bytes used by the nanoPEB/CF7+ are at the top of VRAM. On my nanoPEB at the moment, those 8 bytes are AA03h 0007h 0005h 0006h. The high 3 words (2 bytes each) indicate the mounted volumes—DSK1=VOL7, DSK2=VOL5, DSK3=VOL6. I am guessing the AA03h word holds validation information, but I don't really know how to decipher it.

 

...lee

[+Lee Stewart]

I am beginning to anticipate how I will be coding fbForth for cartridge space. Considering that a cartridge-based fbForth will not be loading the kernel from disk, I think I will forego my attempts at boot tracking and just deal with the user via holding down a key to indicate the location of the system files (FBLOCKS, FBCHARS and FBMSGS) when not on DSK1 and a warning message if they cannot be found on the default or indicated disk a la @Willsy's TurboForth.

 

I suppose I could also provide a word to effect the change after bootup, say, SYSPATH or somesuch.

 

Otherwise, I think I will limit changes to converting block I/O to level 3 file record I/O from the level 2 file sector I/O I am currently using.

 

...lee

[+OLD CS1]

In the case of the character set being loaded from the blocks disk (sigh... why didn't I think of that??!) you might want to consider putting a header at the beginning of the data on that block. That way, fbForth could determine if the data it is looking at is valid character set data, and load it if it is, but more importantly, not load it if it isn't (leaving the default TI character set in place, presumably).

 

Of course, in a cartridge, you could place the character set and error messages in ROM. Early in TFs development, I was considering reserving 4K of the EPROM space as 'read only blocks'. They would be accessed with negative block numbers, -1, -2, -3, -4 and would house various utilites, or system messages etc. It just needs a trap in BLOCK to check the block number and take a different course of action in the event of a negative block number. I ran out of ROM space though! I nievely thought that 12K would be enough code space! Ha! I need more like 32K of ROM, possibly 64K for what I'd *really* like to do!

 

This might require a bit more hacking than desired. You could also give the user the option of including character sets and error messages in the expected blocks, and if that data is either invalid or not present fall back to your ROM blocks.

[Asmusr]

The nanoPEB uses 8 additional bytes more than the TI floppy DSR. This would make the default value for 3 simultaneously open files 37CFh instead of 37D7h. This is not something you can set explicitly, however.

 

Thanks. It's working now. I had to save to contents of the >8370 pointer and restore it again when it's needed instead of just setting it to >37D7.

[+Lee Stewart]

... I naively thought that 12K would be enough code space! ...

 

But—TurboForth uses 16KB, does it not?

 

...lee

[+Lee Stewart]

But—TurboForth uses 16KB, does it not?

 

...lee

 

Oops! This is embarrassing! I figured out a month ago that it is, indeed, 16KB (4KB for proposed negative blocks + 12KB remaining)! You'd think I could remember that. It's hell gettin' old!

 

...lee

[+Lee Stewart]

I'm thinking about using the 64KB Guidry board for fbForth. I have little to no idea about cost and availability. @acadiel? I can probably do it in 16KB; but, I think I want to put into the cartridge all of the major code from the system blocks file (FBLOCKS)—editors, graphics, file I/O, TMS9900 Assembler, BSAVE , floating point (FP) math, ... .

 

For FP math, I would use the package I modified for TurboForth.

 

I would need to work out a system for searching the dictionary over multiple banks. I suppose the trampoline code for 8 banks would be longer than for 2. @Willsy?

 

I might also make the additional dictionary code optional by patching link field addresses in and out at the user's whim. The reason I would do this is that having everything in the dictionary available, when the user may not be interested, will increase the time required to search the dictionary for a given word. I haven't run any tests to see what the increase in search time might be, however.

 

I'll be getting back to the current project ere long—you know, the blocks file I/O conversion. I just needed to ramble a bit to try to get some ideas.

 

...lee

[Willsy]

I would need to work out a system for searching the dictionary over multiple banks. I suppose the trampoline code for 8 banks would be longer than for 2. @Willsy?

 

Well, you probably don't need any trampoline code (at least, in PAD RAM) at all. What I *should* have done in TurboForth, early on, is have some code in the exact same address in EVERY bank, which simply pages in the next bank and exits. Probably the easiest way to do this (you'll appreciate that I worked this all out *after* I had written TurboForth and it was just too much work to re-engineer everything) is to have an *exact* copy of FIND in every bank, at the *exact* same address. By doing this, page changes have no effect whatsoever on the code that the 9900 is executing. Then, all you need to do is slightly expand the dictionary search/traversal technique. As you know, traditionally, FIND just walks the dictionary, using the link field to get the address of the next word. When it sees a link field of 0 it knows it has reached the end of the dictionary and gives up the search. That just needs to be expanded very slightly: We need a flag to invoke a bank switch. My intention was to use 0 as the "end of dictionary flag" as before, and >FFFF as the "hey, this is the end of this bank, do a bank-switch now" flag. In the case of a bank-switch, an extra field will be required to point to the *actual* address of the word in the next bank, but obviously that’s simple to code. FIND will do its own bank switch, no need to use external routines, and resume the search.

 

I suppose at the beginning of FIND you could push the current bank number to the return stack and restore it at the end so that FIND leaves the system in the state that it was before FIND was invoked. However, you need to consider the datum that FIND returns. I can only speak for Forth 83, but FIND is:

 

FIND ( addr len -- xt type )

 

Where addr and len point to a string in memory. Upon exit, xt represents the execution token (code-field address) of the word, and type tells you if it's immediate (-1) or standard (1) or not found (0).

 

However, you *also* now need to return the bank number, otherwise you won't be able to compile it into memory with the compiler. Consider this:

 

: SQUARE ( n - n*n) DUP * ;

 

Lets assume that DUP is in bank 1, and * is in bank 2. Let us also assume that DOCOL pages in bank 0 for us upon invocation, and ; pages in bank 0 for us when the word exits. Your compiled code (assuming ITC) needs to be this:

 

DOCOL BANK1 DUP BANK2 MPY EXIT

 

That's what the compiler needs to compile. Obviously, the compiler will be extended to know which bank the last word it compiled was in, so that in the event of two or more words being compiled from the same bank, it will omit the compilation of the bank switching instructions.

 

Therefore, I think FIND will have to be non-standard. No way around it. You'll need something like:

 

FIND ( addr len -- xt type bank )

 

Clearly there is a cost (performance cost) with each bank switch, so you'll want to arrange the dictionary as cleverly as possible, such that the most frequently accessed words are all in the same bank.

 

You'll also want copies of all the bank switching routines in every bank, at the same address. One could conceivably use a parameter for the bank switching, but I think that that would cause a quite significant performance penalty:

 

DOCOL LIT 1 BANK DUP LIT 2 BANK MPY EXIT

 

Not nice, since LIT is invoked first to push the bank number to the stack. And now there's a cost of 6 bytes per bank switch, rather than 2.

 

The solution to that particular problem, of course, is to not use the stack, but put the bank number in-line, ala BRANCH et al:

 

DOCOL BANK 1 DUP BANK 2 MPY EXIT

 

Now there's a cost of 4 bytes per bank switch, but BANK is signifcantly faster (because LIT isn't used, so one less round-trip via the inner interpreter). Now you only need one bank switch routine in each bank (at the same address) but it's slightly slower than individual ones.

 

As you can see, multi-bank switching introduces a number of (very interesting) challenges which need to be solved as elegantly and efficiently as possible. For TurboForth I was just too far down the turn-pike (to use an Americanism!) to turn back. Shame. But that's what happens when you don't do a *design* and just jump in with both feet. There again, this is hobby programming, so who cares

 

Mark

Edited July 26, 2013 by Willsy

[RXB]

With the SAMS I solved the problem of my AI bot being in the next page but looking ahead or behind.

As the SAMS has 4K pages upper and lower I just keep moving the code to the middle like this:

 

1500 IF X>15351 THEN B=B+1 :: X=X-4096 :: GOTO 1600

1510 IF X<8224 THEN B=B-1 :: X=X+4096 :: GOTO 1600

 

1600 CALL AMSBANK(B,B+1) :: GOT0 270

 

 

B is the lower 4K banks of SAMS and B+1 is the upper 4k bank.

X is the address in lower 8K.

IN THE DARK used these lines. How it works is if it is within 32 bytes of the end of lower 8K then swap the two 4K pages up and subtract the address to put it in the middle of lower 8K and going down works just the opposite way.

 

That way a single variable keeps track of the SAMS banks and the other the address.

[+Lee Stewart]

Thanks, Mark—definitely a lot to ponder!

 

... However, you need to consider the datum that FIND returns. I can only speak for Forth 83, but FIND is:

FIND ( addr len -- xt type )

 

It's (FIND) (invoked by -FIND ) in fbForth:

 

(FIND) ( str-addr nfa-start --- false | pfa name-len true )

 

where str-addr is the address of the search string, nfa-start is the name field address (points to the length byte) and name-len is the length byte of the found word.

 

...

DOCOL LIT 1 BANK DUP LIT 2 BANK MPY EXIT

 

Not nice, since LIT is invoked first to push the bank number to the stack. And now there's a cost of 6 bytes per bank switch, rather than 2.

 

Don't need LIT for 1, 2, 3 or 4 in fbForth. Just use ONE , TWO , THREE or FOUR —each defined with LIT . Of course, this only saves storage for this definition— LIT still gets called!

 

The solution to that particular problem, of course, is to not use the stack, but put the bank number in-line, ala BRANCH et al:

 

DOCOL BANK 1 DUP BANK 2 MPY EXIT

 

Does this mean a different definition for BANK that does not expect a number on the stack? I'm not sure I understand how this one would work.

 

Maybe, I should commit one bank to the dictionary, with every word switching to the bank that holds the remainder of its definition—probably too much bouncing around, huh?

 

As you say, a lot to consider!

 

...lee