TI-99 Infocom Interpreter Dev

[+InsaneMultitasker]

First post to contain information and source code.   Goal is to reconcile the 80 column Lost Treasures of Infocom executable with source code from earlier version.

 

Compatibility Notes:

Video:  V9938 and F18A, 80 column devices

Cartridges:  SuperCart 8k, Mini Memory, FG99 (pending)

Z-file support:  GAME1/GAME2 original format supported.  New release may migrate to native Z3 file support.

 

[+InsaneMultitasker]

The source code for v1.1 has some optimizations that do not exist in the LTOI release v1.2.   The interpreters are nearly identical, with a few subroutine fixes in v1.2 that were required for The Lurking Horror (and possibly others) to "understand" objects  typed in the parser. 

 

The interpreter code works fine running in the Supercart at 0x7000.  If someone has the time and desire to test with the FG99 cart, using EA and RAM at 0x7000, PM me and I'll send a copy sometime this weekend.  I think the mode bits are set for 80 column F18A but I'll need to verify.

 

Once the source is reconciled I'll consider making the changes needed for the large Z3DAT files like LGOP.  I have not decided if I'll do more than reconcile the source for this fun little project but if I do, adding larger Z3 file support would be high on the list and possibly changing the GAME1/GAME2 files to a native Z3 format that requires little to no manipulation.

[+InsaneMultitasker]

Last night I tested the current reconciled code and discovered the interpreter would spin for 30-45 seconds after I entered each command.   I called it a night and revisited today. Fresh eyes found that I had typed R0 instead of R8 when I reconciled a routine.  Fixing that error returned everything to operational status.  Last night I also wrote new video code that tests for V9938 vram in bank 1, and if not present, defaults to f18a mode. The new routines consolidate all of  the video setup and cache boundary settings into one place.  I also confirmed v9938 additions work now  (highlight status bar, pattern table moves) by loading The Lurking Horror  from TIPI  via my Geneve. 

[+InsaneMultitasker]

Making more headway.  There are still differences in the support routines but I got antsy wanting to test my changes to support the larger Z3 files. So.. here for the first time (for me) is LGOP in 80 columns on the TI...

 

[hlabrand]

Oooh, very nice! This is Supercart + 80 col, is that right?

Also, having an interpreter that supports Z3 files natively is very interesting :)

I'm following this thread very closely! :D

[+InsaneMultitasker]

1 hour ago, hlabrand said:

Oooh, very nice! This is Supercart + 80 col, is that right?

Also, having an interpreter that supports Z3 files natively is very interesting

I'm following this thread very closely!

Thx.  Yes, the screenshot is supercart+80 columns.  An AtariAge user is testing a version that works with the FG99 RAM at 0x7000, so you will be able to use Supercart or FG99 for the larger Z3 files if it all works as expected.  

 

Tonight I compared the implemented interpreter opcodes in the source code by using this site   http://www.inform-fiction.org/zmachine/standards/z1point1/sect14.html  

 

Edit: Your comments reminded me that I hadn't backed up my folder so I just did that tonight.  Losing the progress would make this fun into something not-so-fun.

[+InsaneMultitasker]

Posting for reference  Native Z3 file from @Torrax "The Masters Collection", prepended with a TIFILES header to represent a DF128 file, thanks to TI99Dir.  Note the debugger shows 0x80 (128d) byte records   - not DF255 as used by the GAME1/GAME2 files.

[Tursi]

Note that warning is an indication that some of our disk devices won't work right with that DSRLNK, since the CRU base isn't loaded where it's expected. Not sure if I ever wrote down /which/ devices, though.  But nearly all the warnings are a result of being bitten one way or another!

 

 

[+InsaneMultitasker]

12 hours ago, Tursi said:

Note that warning is an indication that some of our disk devices won't work right with that DSRLNK, since the CRU base isn't loaded where it's expected. Not sure if I ever wrote down /which/ devices, though.  But nearly all the warnings are a result of being bitten one way or another!

I've been ignoring that warning for a while now   Your note prompted me to look at the DSRLNK routine. It is mostly the standard code with modifications to skip device tests if the PAB pointer has not changed since the prior access.  Given that  the infocom games are set up to load from the same disk device for the entire session, this cuts out some processing time for what can be a disk-intensive program. I've noted this and am hoping I can just add code to save 83d0/83d2 during the initial scan. Thanks

 

Edit: a simple MOV R12,@>83d0 within the 'speedup' routine did the trick.

 

[+Torrax]

Looking great there!!

 

Do you think this will work with the Mini Memory with an appropriate OPT5 loader?

 

[+InsaneMultitasker]

13 minutes ago, Torrax said:

Looking great there!!

 

Do you think this will work with the Mini Memory with an appropriate OPT5 loader?

 

Good question, I forgot that Mini Memory could load object files!  I used "Load and Run" option 1 to load the object code.  Mini Memory places some support routines into memory at 0x7000, so it failed the first time through.  I had to bump the starting address of the interpreter from 0x7000 to 0x7200 and it fired right up after that. 

 

The only problem I encountered is that the subsequent attempt to load the program resulted in Mini Memory informing me there was a *duplicate definition*.  The DEF table at the end of the 0x7000 space already contained the START definition. I am not sure how to get around this - is it normal operation for minimem?  Anyway, the Interpreter does appear to run just fine from the mini memory RAM.

[HOME AUTOMATION]

7 hours ago, InsaneMultitasker said:

I am not sure how to get around this - is it normal operation for minimem?

Yes, normal... E\A seems to reset the REF/DEF tables when OPT3, "load and run" is selected... on MINIMEM you are expected to use OPT3, "RE-INITIALIZE" followed by FCTN 6 "PROCEED", in order to manually clear REF/DEF. I suppose this is intended to help avoid unintentional overwriting of previously loaded BASIC or ASSEMBLY programs/data.

 

This persistence seems handy when wanting to select OPT2, "RUN" ...but can become a pain ...when developing.

[+InsaneMultitasker]

12 hours ago, HOME AUTOMATION said:

Yes, normal... E\A seems to reset the REF/DEF tables when OPT3, "load and run" is selected... on MINIMEM you are expected to use OPT3, "RE-INITIALIZE" followed by FCTN 6 "PROCEED", in order to manually clear REF/DEF. I suppose this is intended to help avoid unintentional overwriting of previously loaded BASIC or ASSEMBLY programs/data.

 

This persistence seems handy when wanting to select OPT2, "RUN" ...but can become a pain ...when developing.

Thanks, yes, it all starts coming back now  I also read through Thierry's excellent write-up for the MiniMem cart http://www.unige.ch/medecine/nouspikel/ti99/mm_mod.htm  and confirmed a few observations.  

 

My  'solution' is as follows:  after MiniMem starts the interpreter, I will wipe out the REF/DEF table entry, thereby allowing it to load the next time without any initialization.  This should be possible to do for other development as well.

 

Edit: wiping out the table entry doesn't clear the REF/DEF pointer at 0x701e.  The table grows downward, so the updated solution is to read the pointer and clear from that address to the end of the miniemory (0x7fff), then reset the pointer to 0x8000 to show an initialized state. 

[+InsaneMultitasker]

I've completed the first full review of the source and am just about ready to introduce some of the new code.  There are a few things I'm mulling over.  Whether I implement some,all,or none remains to be seen.  Suggestions and ideas welcome.

 

1. I prefer the status line to be the same color as the rest of the screen (see above images); the v9938 inverted terminal status line draws attention away from the text. Remove the inverted line support?

2. How best to handle device paths and game file names.  Implement something similar to the Adventure cartridge, where the user enters the path.filename when the interpreter starts? Use an XB loader that pokes the information into a 'mailbox' for the interpreter at run-time?  Create a menu with the current known game files? 

3. How to implement the path and file support so that game saves/restores can be well-managed.  I don't necessarily want to lock someone into the same path for both.  For example, if the game file can be loaded via TIPI from somewhere over the internet, the game saves probably need to be local.

4. Interpreter enhancements such as keys to change foreground/background color or to recall the last text typed into the command line (retyping those complex commands due to a typo is irritating)

5. Roll the 40 column mode into this same interpreter (I do not yet know what differences there are between the 40 column v1.3 and 80 column 1.2) for one unified version? 

6. I am leaning very much towards implementing and solely supporting the native Z3 file format. The only up-front cost is converting the GAME1/GAME2 files from TI versions of the game, which I believe Torrax has already taken care of with his Master's collection, leaving only the need for a FIAD conversion to make the file usable.

 

[hlabrand]

10 hours ago, InsaneMultitasker said:

6. I am leaning very much towards implementing and solely supporting the native Z3 file format. The only up-front cost is converting the GAME1/GAME2 files from TI versions of the game, which I believe Torrax has already taken care of with his Master's collection, leaving only the need for a FIAD conversion to make the file usable.

 

This is probably for the best as there is much less friction in setting it up. Also, the game files have all been collected at https://eblong.com/infocom/ , when the Infocom archives and source code were uploaded to the Internet Archive and Github ; this means for instance that TI-99/4A users can play the last revision of Seastalker instead of the one in the master collection.

 

For the rest of the changes, they sound good, especially the "press the up arrow to display the previous command", which is a great usability functionality to have.

 

Edit: I forgot to ask: does the TI-99/4A support accented character display, like é or ù ? The Z-Machine has, in theory, two character tables, and most interpreters don't support the second one (it wasn't really used); but there are tricks that allow a redefinition of the first character table to include accented characters. This is mostly useful for other languages, like the German translation of Zork, or possibly the French translation of my own game; a very specific use case that probably will not include very many people, so don't worry too much about it

Edited September 10, 2020 by hlabrand

[+9640News]

13 minutes ago, hlabrand said:

Edit: I forgot to ask: does the TI-99/4A support accented character display, like é or ù ? The Z-Machine has, in theory, two character tables, and most interpreters don't support the second one (it wasn't really used); but there are tricks that allow a redefinition of the first character table to include accented characters. This is mostly useful for other languages, like the German translation of Zork, or possibly the French translation of my own game; a very specific use case that probably will not include very many people, so don't worry too much about it

 

This is for curiosity's sake.  Do those characters generate a single ASCII character?  Or is it a multiple character sequence?

 

Inquiring that if they are just redefine characters and lower ASCII values, then it may be just be a redefinition of a character.  If it is something outside the normal TI character sequence, then there would need to be a test for a particular character sequence to respond back to and display that character.  Like FCTN-Underline to generate the particular key which I am guessing would not be the specific key used, but an example.

 

Beery

 

[hlabrand]

Actually, never mind me... After going deep in documentation, it seems like the character table in v3 is fixed and cannot be changed. No need to worry about accent display. (The German version of Zork is z5, anyway.)

 

EDIT: sorry, I mean "the primary character table is fixed and cannot be changed". Maybe the secondary character table can be used, but most interpreters don't support this. I'll get back to you when I get a better understanding of what's going on and if it appears more reasonable to do than that.

Edited September 10, 2020 by hlabrand

[GDMike]

Yeah but you can change the char set and load and reload at Will.

[+InsaneMultitasker]

2 hours ago, hlabrand said:

This is probably for the best as there is much less friction in setting it up. Also, the game files have all been collected at https://eblong.com/infocom/ , when the Infocom archives and source code were uploaded to the Internet Archive and Github ; this means for instance that TI-99/4A users can play the last revision of Seastalker instead of the one in the master collection.

 

Is SeaStalker the game that has an optional second window for a simple sonar?  There isn't any windowing capability (that I can detect) in the TI interpreter.  If the newer versions support features the interpreter cannot handle, that would require some further research.  (For exmaple, the v1.2 interpreter has an additional bit test that the older interpreter ignores).  Fortunately, we have the TI collection at our disposal.

 

The added benefit of using the native Z3 files is that the interpreter can free up some space for other routines.

 

 

[wierd_w]

The finalgrom version you mentioned-- is this for the banked 4k feature?  Otherwise, the 4k minimem image for the FG should work straight up, if the minimem cart itself does.

 

If banking is supported (at some point), then up to 512k of data could be stored there.  (!)  That's a rather large text adventure indeed.

[+InsaneMultitasker]

I do not have any interest (at this time) in supporting the FG99 banking for the text adventures. I have considered creating a simple loader to utilize SAMS and V9938 Vram as cache for the story file but that is truly pie-in-the-sky for me right now.  I'll be happy with a documented, unified interpreter capable of running nearly all of the known Z3 games. 

 

Edit: Good call on the MiniMemory image. That sounds like a good option!  I'm hoping that a similar EA Supercart image is possible as that cart would have the option 5 loader. 

[hlabrand]

Re: accents, if @InsaneMultitasker is interested: what currently happens when I put Latin-1 characters in my game (which is possible under Z-Machine version 3) is that their value (determined using the ZSCII character table) are converted modulo 128. For instance, "ê" (character number 192 in that table) results in the display of "@" (character number 64), "è" (character number 182) results in the display of "6" (character 54), etc.

 

I don't know what you can do about that internally, or what the display possibilities are on a TI-99/4A However, there are quite a few other interpreters for other retro platforms (Apple II, Amiga, BBC Model B, Oric) that do the same thing. (Do you know if that's a conscious choice in the code (like, explicit "modulo 128"), or "casting" a variable, or an opcode that only takes 7-bit numbers, or something else? I'd be interested in learning more about it. But only if you have time.)

[+InsaneMultitasker]

Question: are you saying the TI interpreter displays character 64 for character 192?  Or is the modulo 128 conversion expected?  I'm not quite following.    On the TI, video memory is extremely limited the way the interpreter works today. To implement additional character definitions would require approximately 1024 bytes (128 characters x 8 bytes/character).  This would remove two story cache record slots from the interpreter's working memory.  If the idea is to display up to 256 characters, I can see this -possibly- happening with a 40 column interpreter, and maybe the V9938 flavor of the 80 column interpreter.  Share more details and I'll add a note to look into it at a later date.

[+InsaneMultitasker]

First pass at memory usage for the various interpreters.  I have not reviewed the 40 column interpreter code closely as my focus is the 80 column implementations. 

 

 

 

;10 Sept 2020

;===============
; CPU RAM USAGE
;===============
; LOWER 8K:
;------------
; >2000->21ff    <stack grows downwards; init to 0x2000>
;
; >2200-23ff   #another stack pointer! Initialized to 0x2400.  See ZBN and ZBR.
;        debugger shows it is consumed quickly. Is it sufficient for complicated/large games?
;    
; >2400-25FE     byte-record table for DF255 records [free this if we move to native Z3 format]
; >2600-265c    VDP record cache pointers; variable depending on interpreter
;      -2660    40 col interpreter extends cache pointers to this address
; >2662     *hardcoded in load process. Fix this.
;
; >2664-26ff    <possibly open, see cache pointers and hardcoded routines>
; >2700-3dff    Interpreter code for v1.2 80 col non-supercart
; >3e00-3f5f    <possibly open; review closely. check game load. Open for more code?>
;
; >3f60-3faf    *hardcoded, used to compose line output; fix hardcode
; >3FB0-3fff    *80 character scroll buffer; fix hardcode
;  ?   -?    *40 character scroll buffer; fix hardcode
;
; UPPER 24k:
;--------------
; >A000->F300    GAME1 information loaded from disk, 22K non-supercart
; >F300-FFFF    Game/system initialization routines and support/interpreter
; >F300-F7FF    (overwritten by Z3 file after initializaton)
;     or
; >A000-FFFF    24K game file; supercart/fg99/minimem interpreter. Init at 6000-7fff
;
;
; SUPERCART/MiniMem ENHANCEMENT:
;-------------------------------
; >a000->ffff     GAME1 informaton loaded from disk; 24K
; >6000->6D00    Interpreter init and support v1.2
; >7200->7fff    Interpreter init and support routines; use for MiniMem/FG99/Supercart compat
;        Pointers for GAME1, buffers, file IO checks must be adjusted accordingly
;        Interpreter does not use any other 0x6000-0x7fff space for dynamic memory
;
; SCRATCHPAD (0x8300-0x83ff):
;----------------------------
; >8300-831f    - primary Interpreter WS
;        r12=psuedo random rotating based on user kscan entry timing
;
; >8320-833f    - Vdp/kscan utility ws. DSRLNK WS as of v1.2 (which is why direct VDP is used vs. BLWP @Vxxx)    
; >8340-8349     - call/stack routine
; >8350-?    - unused scratchpad v1.1
; >83c0-83df    - interrupt WS
; >83e0-83ff    - gplws; kscan and dsrlnk. do not use.
;
;===================================================
; VDP VRAM BANK #0 MEMORY USAGE (v9938 and F18a)
;------------------------------------------------
; >0000->0780    Screen image table
; 780 - game1  (789 - len)
; 794 - game2  (79d - len /  790 is rec#)
; 7a8 - savex  
; 7bc - pio
; 7d0 - rs232
; 7e6 - buffer for pio/rs232
;
; >0806 - yyyy     Start of VDP cache via label H806 [v1.2 sets to >0836 to fix overlap w/80 char io buffer]
;        See ISTART routine for the setup loop
;        check debuffer for overlap between rs232 80 char buffer and the cache
;        possible overlap w/disk controller vram beyond 37d5
;     ->3806*    9918    >18 (24) slots = >3000
;     ->3606        V9938    >17 (23) slots = >2E00
;     ->3006        F18a    >13 (19) slots = >2600
;
; >3100-3400    Character set for F18A interpreter (v9938, see bank 1)
;        With an offset during display, we could gain 2 more slots.
;        Maybe even 3 slots if we do a CALL FILES(2) during startup
;
; >37d5-3fff    DSR space for TI/CC and certain other peripherals
;
;
; VRAM BANK 1 (R#14 >01)  V9938 only
; ----------------------------------
; 0100-0400     character set
; 0A00-0c00    blink attributes (v1.2 interpreter only)
;
;
; 40 column 16k vram allocation [define later]
; -------------------------------------------
; 0000-00ff    File IO PAB
;     0 - game1  (789 - len)
;     0 - game2  (79d - len /  790 is rec#)
;     0 - savex  (
;     0 - pio
;     0 - rs232
;     0 - buffer for pio/rs232
;
; 0100-03ff    character set (vreg set to 00)
; 0400-07ff    screen image table
; ? - ?        VRAM cache for 512 byte records
;
; 37d5-3fff    DSR space for TI/CC and certain other peripherals

 

[hlabrand]

2 hours ago, InsaneMultitasker said:

Question: are you saying the TI interpreter displays character 64 for character 192?  Or is the modulo 128 conversion expected?  I'm not quite following.    On the TI, video memory is extremely limited the way the interpreter works today. To implement additional character definitions would require approximately 1024 bytes (128 characters x 8 bytes/character).  This would remove two story cache record slots from the interpreter's working memory.  If the idea is to display up to 256 characters, I can see this -possibly- happening with a 40 column interpreter, and maybe the V9938 flavor of the 80 column interpreter.  Share more details and I'll add a note to look into it at a later date.

Yes, the former; character 192 isn't shown, and instead, it's character 64. But there are interpreters that do that (Apple II, Oric, etc.), and other interpreters that get it right: Frotz and Gargoyle on desktops, ZXZVM for the Spectrum +3, MaxZIP for Macintosh, the CPC interpreter (with a charset change), Ozmoo for C64 (with a charset change), etc. (Mostly computers that came out much later, I realize )

It would "only" be 69 extra characters, as other ones are used for input or are left blank. Some of them are 180° rotations of each other, if that helps with memory (¿, ¡, «); some of them can (and, really, should - nobody expects that level of glyph sophistication) be displayed with letters (æ -> ae, œ -> oe, and possibly the 4 icelandic glyphs; it is also common practice in French to remove the accents on the capital letters, because it's too hard to get on Windows, so people got used to it).

I understand that it's a memory tradeoff, and really, I'm asking a lot, being able to play games in French on an American interpreter. It might not be feasible, but just let me know either way

Edited September 11, 2020 by hlabrand