Some file questions...

[+adamantyr]

So I'm working on a custom loader for my assembly language CRPG... I'll need to in order for it to load all the various program and data segments into the AMS.

 

There are three potential techniques I think I can use:

1. Store each 4k page as a separate file. Each file is then loaded as an image into VDP, copied to a page in CPU. Iterative file names would help sort things out.
2. Store the data in a few (two or three) very large files as binary data, 128 byte length. Load it as records, 32 at a time, swapping pages as needed.
3. Store the data in a few (two or three) very large files but access them by sector instead of as records, so you can just grab a 256 byte chunk at a time. Otherwise identical to #2

What I'd LIKE is a fourth option, some way to store the data as a large image file but only load it as a partial image. Anyone know if that's possible?

 

Also, which method is the most efficient in terms of time?

[RXB]

In RXB I opted for loading 8K Program Image files, the reason is speed and buffer space needed.

 

1. 8K is large enough that this used 8K for Files to load into the Lower 8K each time then using RXB CALL MOVES to move the image from/to VDP.

2. With 12K of VDP space 12K-8K=4K for a XB loader to be used, then can load another XB Program or as demod in many of my demos of SAMS multiple XB and Assembly programs all running from CALL USER

3. Using CALL USER allows loading SAMS and XB multiple programs all run from a single DV80 Batch file server.

 

Varied size files loaded created all kinds of problems including catalogs and label problems. Thus even small unused portions means at least more consistency for looking at a catalog.

[senior_falcon]

If you intend to run these segments as a separate program then loading 4K or 8K at a time is fine. If you are running from the same XB program then you are likely to overwrite string information in the VDP RAM. You can avoid this by using the crunch buffer and edit/recall buffer from >0820 to >0957. Using IF128 or IV254, you can easily fit a PAB and buffer into this area without overwriting anything you care about. Also, don't overlook the possibility of loading character definitions, sprite definitions, and the screen image directly from disk to the proper VDP areas.

(edit) The speed of loading IV254 or IF128 is not significantly slower than loading using larger blocks.

Edited October 16, 2017 by senior_falcon

[RXB]

If you intend to run these segments as a separate program then loading 4K or 8K at a time is fine. If you are running from the same XB program then you are likely to overwrite string information in the VDP RAM. You can avoid this by using the crunch buffer and edit/recall buffer from >0820 to >0957. Using IF128 or IV254, you can easily fit a PAB and buffer into this area without overwriting anything you care about. Also, don't overlook the possibility of loading character definitions, sprite definitions, and the screen image directly from disk to the proper VDP areas.

RXB has this built into it since 2001, not like the problem has not been addressed. If this is to create a XB version never are you going to create something as easy to use as RXB doing this.

 

RXB can load 960K of the SAMS into memory along with an XB program or multiple using CALL USER batch files that really give you no limit of approaches.

 

RXB loaded 960K of SAMS from SCSI Hard Drive in 1 minute 58 seconds. From DSDD disk in 3 minutes 40 seconds.

Edited October 16, 2017 by RXB

[+adamantyr]

Hmm... Using 8k for two pages at a time. Not too bad! Fortunately I will only be doing a major load like this at the start... that's the point of the AMS card, keep it in memory.

 

I wonder if it's possible to free up enough VDP memory to load a 12k file? You'd only need to leave a bit of room at the top of the RAM for the PAB (1052 bytes, if you set it to a single file) and then room at the bottom for the screen, character, sprite and color tables. If you're willing to have a blank screen during loading you could even over-write the pattern table.

[+InsaneMultitasker]

 

What I'd LIKE is a fourth option, some way to store the data as a large image file but only load it as a partial image. Anyone know if that's possible?

 

Also, which method is the most efficient in terms of time?

 

Can you elaborate on your 'fourth' option? It sounds quite similar to your option #3.

 

If you go the large file route, you can use record IO which is probably the simplest and most portable across devices. Data transfers are limited by the file type and record size, for example, to read 4K of a DF128 file, you would need to read 32 (8records/1K *4) individual records.

 

A second file-based option is to use the level 2 subprograms to read/write one or more 256-blocks of the file. Like record IO, level 2 allows for random access without the file type restrictions, since you are essentially reading the file at the 'sector' level. It's one caveat is that unlike record IO, you must account for hard drive devices separately to manage their unique requirements (a true PITA). In any case, to read the same 4K using level 2 IO, you would set the starting block, total sectors, and call the DSRLNK routine once to transfer it all to VDP.

 

Record IO will always be slightly slower than level 2 IO, though that probably isn't a concern for a one-time load. Your option #2 seems like the best approach, IMO.

 

(You could also store and retrieve the data from cartridge ROM/image, if you didn't want to deal with the disk system)

[+adamantyr]

I just wrote up a loader that does a 12k memory image file by clearing most of VDP out to make room. That lets me have ten (120k) program files I can load.

 

Record IO was my second option but I think it's slower than memory images. As is I expect it's going to take 2-3 minutes on a regular TI with a standard disk controller to load the entire game.

 

I'm not using cartridge/ROM at all. I'm trying to avoid a SuperCart as well.

[+InsaneMultitasker]

I just wrote up a loader that does a 12k memory image file by clearing most of VDP out to make room. That lets me have ten (120k) program files I can load.

 

Record IO was my second option but I think it's slower than memory images. As is I expect it's going to take 2-3 minutes on a regular TI with a standard disk controller to load the entire game.

 

I'm not using cartridge/ROM at all. I'm trying to avoid a SuperCart as well.

 

Slightly slower, but not that big of a deal. Preparing the memory images can sometimes require more time to prepare, so who knows you may find yourself using one or both file types. Many ways to skin the proverbial loader 'cat'...

[Asmusr]

For the SAMS loader for the Megademo we were using 8K program files corresponding to 8K ROM banks. AFAIK it took a few minutes to load the 34 files on a standard TI FD controller.

[+adamantyr]

I am loading from disk so I can push them up to 12k. I figured I can downsize if it proves to be a hassle.

[Tursi]

For the SAMS loader for the Megademo we were using 8K program files corresponding to 8K ROM banks. AFAIK it took a few minutes to load the 34 files on a standard TI FD controller.

 

We did change it out to a single large file, though... we had so many files that the cost of the directory entries was cutting into our free disk space. I implemented both a sector-based read (using the level2 I/O mentioned above) and a record based read (as a fall back if the sector-based read failed). Testing suggested that the level2 I/O was about as fast as PROGRAM image loads, but records were indeed a little slower.

 

Edit: my mistake. I was reading the code to see if I was telling the truth, and I was not. The loader actually does raw disk sector accesses, assuming that the big file is un-fragmented (and so simple offsets work). I have a comment that says the level 2 I/O still required open/close calls and was not much faster than records.

Edited October 17, 2017 by Tursi

[+adamantyr]

It makes sense records are a bit slower... I haven't found any disassembled source for the E/A #5 but it can do away with the record abstraction and just grab the whole sector at a time.

 

It's unfortunate that the TI file system relies on the VDP at all, but given it's the only RAM in the case console it makes sense.

[Tursi]

Fixed length records are not terrible, at least, as the DSR calculates which sector the data will be in and just jumps. Variable length records are slower as it does indeed need to read each record to find the one you want. (I do think sequential reads work as expected though).

 

The Level 2 Sector I/O is slightly faster since there's no calculation, and you get 256 bytes for a call instead of 128 (assuming DF128).

 

Heh... and yeah on the VDP. If you need to read a 256 byte sector, there's really no choice but to use VDP, since you can't count on more than the 256 bytes of scratchpad. RAM in the controller card would have worked but cost a lot more.

[+adamantyr]

I assume then that if you do a "memory image load/save" the DSR simply uses the level 2 sector I/O method and reads/writes the data in sectors of 256 bytes?

[+jedimatt42]

I assume then that if you do a "memory image load/save" the DSR simply uses the level 2 sector I/O method and reads/writes the data in sectors of 256 bytes?

For legacy hardware, I believe that is true. TI describes the levels of IO for that purpose, one level is built upon the other. See section 4.3 of this:

http://atariage.oratronik.de/Software_Specification_for_the_99_4_Disk_Peripheral_V2_0_03_28_1983.pdf

 

I don't know what ramdisks are doing, or harddrive controllers. For TIPI, LOAD and SAVE operation do not break anything into sectors, and so end up having the least overhead.

 

-M@