fbForth fbForth—TI Forth with File-based Block I/O [Post #1 UPDATED: 06/05/2024]

[+Lee Stewart]

Thanks for the links. Just took a look. The only thing to be careful of when using sector access is the possibility of file fragmentation.

 

This could potentially happen. I seem to remember that it's possible to 'expand' a blocks file in TF just by writing something to a record number that doesn't exist. So, for example, in TF:

 

MKBLK DSK2.TESTBLOCKS 20
S" DSK2.TESTBLOCKS" USE
1 EDIT
< EDIT BLOCK 1 AND TYPE HELLO THEN EXIT EDITOR>
FLUSH
1 BLOCK DROP \ get block one
UPDATE \ flag it for update
1 BUF? DROP \ get the buffer for block 1
75 SETBLK \ associate it with block 75
FLUSH \ write block one to block 75
75 LIST

 

You'll now find that the file has grown magically to 75 blocks, even though it was created as a 20 block file. This is nothing special done in TF, it was the TI DOS that did it.

 

So... Imagine that the file was created, then a DV80 laid down on disk, then the blocks file was expanded - you'd have a fragmented file (shown in catalog listings with an asterisk next to the file name IIRC).

 

It's rare, but it can happen...

 

Just something to bear in mind...!

 

Yeah, I understand that, I think. If you mean that the clusters of sectors are not contiguous, I understand. If you mean that the file has been broken, I do not understand, because writing to a nonexistent record is a perfectly legitimate way to extend a file as long as there's room on the disk—the material in the intervening records just happens to be whatever was on the disk before they were grabbed by your now larger file. That's exactly how I expanded the SYS-SCRNS file to occupy the whole remainder of the system disk in §L.1 of my edition of the TI Forth manual. And, even though subprograms 014h and 015h are accessing a file by sectors, it does appear they are limited to file access, not the type of sector access managed by subprogram 010h, which is sector access with total disregard to what the VIB, FDIR and FDRs say about disk organization—the reason TI Forth blocks access is so dangerous and a principal reason I want to change it to file access. I am just thinking that subprograms 014h and 015h (if they work as I think) will get the job done a block (4 sectors) at a time. After all, the current system already has a 4-sector VDP RAM buffer.

 

...lee

[+Lee Stewart]

..

You'll now find that the file has grown magically to 75 blocks, even though it was created as a 20 block file. This is nothing special done in TF, it was the TI DOS that did it.

...

 

It is for this reason I am actually considering for blocks files that my new TI Forth system not allow expansion this way, but rather do it through a copy mechanism to a larger blocks file.

 

...lee

[+Lee Stewart]

It is for this reason I am actually considering for blocks files that my new TI Forth system not allow expansion this way, but rather do it through a copy mechanism to a larger blocks file.

 

...lee

 

And, I can check the size of the file by setting the access code for subprogram 014h to read 0 sectors of the file, which will retrieve file allocation information. I must try it to see what happens!

 

...lee

[Willsy]

And, even though subprograms 014h and 015h are accessing a file by sectors, it does appear they are limited to file access, not the type of sector access managed by subprogram 010h, which is sector access with total disregard to what the VIB, FDIR and FDRs say about disk organization

 

Ah! Okay, well if that is the case that's really cool, and I could potentially be persuaded to change TF to use the same mechanism. If you do get a chance to try it I'd love to hear the results. I had a look on Thierry's site, but I'll now go check the document that you linked to.

 

Mark

[+Lee Stewart]

Ah! Okay, well if that is the case that's really cool, and I could potentially be persuaded to change TF to use the same mechanism. If you do get a chance to try it I'd love to hear the results. I had a look on Thierry's site, but I'll now go check the document that you linked to.

 

Mark

 

GOOD NEWS! DSR Subprograms 014h and 015h work quite well on disks opened in Classic99's FIAD mode for reading and writing any number of sectors from anywhere in the file!! :thumbsup:

 

Using Subprogram 014h with an access code of 0 reports the file information properly.

 

Using Subprogram 015h with an access code of 0 creates a file with the proper parameters , but, unfortunately does not properly populate the file with the correct number of sectors . Instead, a file with 0 sectors is created, which can be managed by executing Subprogram 015h a second time with the last sector desired—but it shouldn't need to be done that way.

 

Also, Subprogram 014h does not work properly in disk-image mode. Access code=0 properly reports all file information except total sectors and reading file sectors in does not work at all. Maybe @Tursi can shed some light on this.

 

I've looked at Win994a for some of this and what I've checked so far, works. I need to check the file creation feature.

 

...lee

[+Lee Stewart]

I've looked at Win994a for some of this and what I've checked so far, works. I need to check the file creation feature.

 

H-m-m-m...Win994a does the same thing about file creation with Subprogram 015h. The only thing left is to check it on real iron, which I don't think I'm doing very soon—the TI docs are probably wrong about this. The info I'm referencing (GPL Interface Specifications for the 99/4 Disk Peripheral***) says, "The # of first AU entry indicates the AU number at which the read should begin. If the access code = 0 (parameter passing), the total number of AUs to be allocated for the file has to be indicated." The first sentence is obviously wrong because 'read' should be 'write'—this is an output function, after all! So....

 

...lee

 

________________

***Actually this is a transcription of the 2^nd document in TI DSDD FDC Manual and DSDD FDC GPL Interface Manual.pdf, which is where the quote actually originates.

[+Lee Stewart]

Ah! Okay, well if that is the case that's really cool, and I could potentially be persuaded to change TF to use the same mechanism. If you do get a chance to try it I'd love to hear the results. I had a look on Thierry's site, but I'll now go check the document that you linked to.

 

Mark

 

And another thing: it doesn't look to me as though any file space at the top of VDP memory is co-opted for these two subprograms. The files appear to be opened and closed implicitly. For TurboForth, you wouldn't need an intermediate buffer—just reference the block buffer and read or write 4 sectors of the file in one pass. I, on the other hand, will just use the intermediate block buffer TI had used in VDP RAM because I need the space for TI Forth graphics modes and I won't need to change much, if any, memory real estate. I was sort of looking forward to an extra 896 bytes of VDP RAM I would have gained from using a 128-byte record buffer instead of the 1KB buffer TI used; but, I think the speed and ease of use outweigh the space advantage. The only real problem is bitmap mode, where there's practically no extra space for more than one or two simultaneous files—I think I'm still going with the speed.

 

...lee

[+InsaneMultitasker]

Lee, it's been a while since I've reviewed the level 2 IO code where VDP is concerned, but I am 51% certain it is used where the bitmap and FDR operations are concerned. So while you may not have true file IO buffers in use, the TI DSR is still dependent upon it to buffer and manipulate the files. Whether or not the routines are encumbered by the CALL FILES subprogram is something else I don't recall off the top of my head. Since each operation is technically "point in time" the DSR may not require more than the last VIB and FDR buffers. Sorry... been a long month already...

[+Lee Stewart]

Making progress! I now have the BLOCKS file whipped into shape, I think. I need to work on the ALC for some new block functionality and adding new and some old words to the resident dictionary.

 

...lee

[Tursi]

GOOD NEWS! DSR Subprograms 014h and 015h work quite well on disks opened in Classic99's FIAD mode for reading and writing any number of sectors from anywhere in the file!! :thumbsup:

 

Using Subprogram 014h with an access code of 0 reports the file information properly.

 

Using Subprogram 015h with an access code of 0 creates a file with the proper parameters , but, unfortunately does not properly populate the file with the correct number of sectors . Instead, a file with 0 sectors is created, which can be managed by executing Subprogram 015h a second time with the last sector desired—but it shouldn't need to be done that way.

 

Also, Subprogram 014h does not work properly in disk-image mode. Access code=0 properly reports all file information except total sectors and reading file sectors in does not work at all. Maybe @Tursi can shed some light on this.

 

The sector read/write opcodes were always supported in Classic99 for FIAD, but file creation was always a pain. That was the drive behind always having a header, actually. I would have to recheck the creation bug you mention there (not allocating all the needed sectors), that should be an easy fix.

 

DOAD mode doesn't support all the opcodes, no. I'm still not a fan.

 

Classic99 is in pieces on my hard drive at the moment with too many projects fighting for my time, so I can't release a quick fix. But we'll see what I can do over the next few months.

 

[+Lee Stewart]

The sector read/write opcodes were always supported in Classic99 for FIAD, but file creation was always a pain. That was the drive behind always having a header, actually. I would have to recheck the creation bug you mention there (not allocating all the needed sectors), that should be an easy fix.

 

DOAD mode doesn't support all the opcodes, no. I'm still not a fan.

 

Classic99 is in pieces on my hard drive at the moment with too many projects fighting for my time, so I can't release a quick fix. But we'll see what I can do over the next few months.

 

Like I said in the post following this one, the TI docs may be in error on this point because at least one other emulator/simulator (Cory Burr's Win994a) does the same thing. I guess creating a file on real iron is the best way to tell for sure. I may try to tease it out of the DSR itself—that should reveal the answer, as well.

 

...lee

[Tursi]

Like I said in the post following this one, the TI docs may be in error on this point because at least one other emulator/simulator (Cory Burr's Win994a) does the same thing. I guess creating a file on real iron is the best way to tell for sure. I may try to tease it out of the DSR itself—that should reveal the answer, as well.

 

...lee

 

Thierry has the TI DSRs fully commented on his site, so that's helpful if you plan to take that route.

 

[+Lee Stewart]

Thierry has the TI DSRs fully commented on his site, so that's helpful if you plan to take that route.

 

Thank you, Sir! I'll check it later this evening.

 

..lee

[+Lee Stewart]

Thierry has the TI DSRs fully commented on his site, so that's helpful if you plan to take that route.

 

Yup. It looks like Subprogram 015h is supposed to expand the file, upon creation, to the number of sectors indicated in "# of first AU" (sector) of the Additional Information Block. For my purposes, it may not be that important to have this right because I should write blank sectors to the file, anyway. That process will properly create the number of sectors I want the file to have.

 

...lee

[+Lee Stewart]

OK...Another quandary: BOOT , which is called by COLD just before COLD calls ABORT , seems to me to be wasting time checking that the starting boot block contains nothing but ASCII characters before loading it. I would like to dispense with that ASCII check. I am going to have COLD reset to DSK1.BLOCKS as the default blocks file and that reset will fail if that file is not found, so I think that should be good enough. What do you think? @Willsy? @Vorticon? @Opry99er? @idflyfish? @Rod Van Orden? Anybody?

...lee

[Willsy]

It does seem strange that it would perform such a check. What if one wanted to put binary code (like assembly code) in there? I say scrap it.

 

What do you mean by "the reset will fail if that file is not found"? Do you mean that you will check for the existence of DSK1.BLOCKS before doing a reset and not perform the reset if the file is absent? Seems long winded. In TF if you type COLD it actually just runs the entire startup sequence from the EPROM. So, it will reset the screen mode, reset the ASCII character set, reset the stacks, the memory pointers, and reset the file to DSK1.BLOCKS - the whole shebang, as if it had just been powered up. If DSK1.BLOCKS isn't found then an error message is issued (but thats a function of the startup routine, not a function of COLD. COLD just does a B @START in TF!!)

[+Lee Stewart]

On 5/21/2013 at 3:28 AM, Willsy said:

It does seem strange that it would perform such a check. What if one wanted to put binary code (like assembly code) in there? I say scrap it.

 

What do you mean by "the reset will fail if that file is not found"? Do you mean that you will check for the existence of DSK1.BLOCKS before doing a reset and not perform the reset if the file is absent? Seems long winded. In TF if you type COLD it actually just runs the entire startup sequence from the EPROM. So, it will reset the screen mode, reset the ASCII character set, reset the stacks, the memory pointers, and reset the file to DSK1.BLOCKS - the whole shebang, as if it had just been powered up. If DSK1.BLOCKS isn't found then an error message is issued (but thats a function of the startup routine, not a function of COLD. COLD just does a B @START in TF!!)

 

"That reset" simply refers back to "reset to DSK1.BLOCKS".

 

In TI Forth, COLD resets pointers and user variables to startup values and resets the dictionary to its startup condition (without reloading it) just before the boot block was loaded at powerup; but, it does nothing to the screen or character sets. I won't be able to restart the entire system as TurboForth does until I get TI Forth into ROM.

 

As to BOOT , it does appear that its sole function is to check that the boot block is all ASCII and then to LOAD it. This is what the TI Forth Glossary says:

 

Examines the Forth screen designated as the boot screen (screen #3). If it contains

only displayable characters (ASCII 32 – 127), it performs a LOAD on that screen.

 

It does, however, also clear the stack, reset the error count variable ( ECOUNT ), reset CURRENT and CONTEXT to the Forth vocabulary and BLK to 0 (terminal input) just before doing what the glossary says.

 

...lee

[+Lee Stewart]

As you can see by the title of this thread, I changed the name of the version of TI Forth I'm working on to "TI fbForth", the "fb" representing "file-based".

 

...lee

[+Lee Stewart]

I just ran into an interesting problem while mulling over the change from disk-sector access to file-sector access for TI Forth blocks. With disk-sector access, there was never any reason to track block I/O because the PAB was only 2 bytes and never needed to be stored—it was set up afresh for each and every block transfer. With fbForth, I need to have some information persist across accesses involving the same blocks file. This is only important if a programmer changes to/from bitmap mode. That action changes the location of the area in VRAM used for PABs. Like I said, not a problem for disk-sector accesses, but now it is! At the very least, it adds overhead to fbForth. This difficulty notwithstanding, it was always a monster waiting to pounce with TI Forth. Though you cannot do much in the way of file manipulations in bitmap mode (only one or two simultaneous files), switching back and forth (sorry ) will trash any PABs. Because of this fact, I am inclined to only worry about the integrity of the current blocks file PAB. Thoughts?

 

...lee

[Opry99er]

switching back and forth (sorry )

 

niiiiiiice...... =)

 

...only important if a programmer changes to/from bitmap mode.

 

Are you referring to the switch happening within the same program? Are there any examples of this occuring in current games/programs? It would seem a bit odd... but I guess a game in bitmap with a menu in text or GMode 1 might exist....

[+Lee Stewart]

On 5/22/2013 at 11:17 PM, Opry99er said:

niiiiiiice...... ?

he-he

 

Quote

Are you referring to the switch happening within the same program? Are there any examples of this occuring in current games/programs? It would seem a bit odd... but I guess a game in bitmap with a menu in text or GMode 1 might exist....

 

Nope...just covering possibilities. Actually, I may have worked it out: I will just pass the information to the block read/write routine and rewrite the block PAB wherever it is. It will just be closer to 16 bytes instead of 2—I was just tryin' to save cycles. 'Course that still does nothing for other (possibly) open files.

 

...lee

[Opry99er]

Hmmm.... How much control are you planning to exercise over file usage? By that I mean, are you planning to build in safeguards against the user inaccurately using the new file structure, or will you take more of a "hands-off" approach?

 

I don't know if that makes sense the way I worded it... but in assembly you have free range to screw up whatever you want to the point of frying hardware!!! In BASIC or a dialect like Wycove Forth, there are multiple safeguards in place to prevent the user from "screwing it all the way up"

[+Lee Stewart]

Hmmm.... How much control are you planning to exercise over file usage? By that I mean, are you planning to build in safeguards against the user inaccurately using the new file structure, or will you take more of a "hands-off" approach?

 

I don't know if that makes sense the way I worded it... but in assembly you have free range to screw up whatever you want to the point of frying hardware!!! In BASIC or a dialect like Wycove Forth, there are multiple safeguards in place to prevent the user from "screwing it all the way up"

 

I was planning only to prevent the user from screwing up the current blocks file. I may try to be more protective of non-blocks-file file access as I work on this; but, my principal objective is to maintain as much of TI Forth as possible, warts and all, with only the changes necessary to convert from disk-sector access to file-sector access.

 

...lee

[+Lee Stewart]

I now have most of the low-level file-sector access code worked out; but, it currently is eating into the return stack space by 104 bytes (from 806 bytes to 702 bytes). That is not likely to be a problem except for extreme recursion. This does, however, impact my plans for messaging, particularly error messaging.

 

I was planning to use 200-300 bytes of low-memory for error messages to obviate the necessity of using Forth blocks for them; but, unless I can dramatically shorten the file-sector access code, I should probably use blocks. Whereas TI Forth required that blocks #4 and #5 always have a copy of the system error messages, I think I will use a separate MSG file for fbForth. I am debating on whether to limit the use of MSG to 2 blocks of error messages or to allow its general use for any messages the user may want to employ via MESSAGE as with TI Forth. The latter will cause a little code bloat, I imagine.

 

...lee

[+Lee Stewart]

OK...For now, it will probably be easier to simply require that blocks #4 & #5 of every blocks file have a copy of the system messages—there are just too many system words I would have to modify, otherwise. Of course, that means I will need to provide a means of making that copy! Rather than include such a copy mechanism in the kernel, bloating it even more than I already have, I think I will provide it as an optional utility as part of the block copying facility, which is currently also optional—it is part of blocks #39 and #40 of TI Forth's system disk.

 

...lee