Jump to content
IGNORED

TI-99 Infocom Interpreter Dev


InsaneMultitasker

Recommended Posts

A few weeks ago, someone ( @Shift838 ? )  posted a comment that there were some Z3 games that were too small for the current interpreters.  I'm looking for an example or two for test purposes.  I'd like to test whether or not my native Z3 file interpreter code will now properly load and play smaller files.

Link to comment
Share on other sites

  • 3 weeks later...

Quick update. I held off on the testing so that I could disassemble and review the LTOI 40 column interpreter.  Most of the differences are related to VDP memory usage for the PABs, screen, and character patterns.  The scoring routine is different taking into account the limited real estate, and there are a few differences with the input routine.  Having completed the review, I am almost ready to commit to the native Z3 file format.  I need to map out a few things before I finalize my approach. 

  • Like 3
Link to comment
Share on other sites

I've been tracking down, documenting, and 'fixing' hard-coded routines and data.  One routine in particular has raised a few questions.   It extracts the keyboard input codes (word separators) from the game dictionary and places them into "unreserved" memory at 0x2662.  There is also a hard-coded list of codes (space, enter, period, comma, question mark, backspace, termination(NULL)) that is being appended to what is extracted.  My guess is an older iteration of the interpreter was not using the game file dictionary codes.

 

From what I've read the period comma and double-quote are the three standard separators for V3 and the separators should always be in the game file dictionary.  Is this accurate? 

 

This weekend I'll remove the hard-coded list to see what happens.  Current code is located in the spoiler.




;  n     list of keyboard input codes   entry-length  number-of-entries
; byte  ------n bytes-----------------      byte         2-byte word
;
;The keyboard input codes are "word-separators": typically (and under Inform mandatorily) these are the ZSCII ;codes for full stop, comma and double-quote. Note that a space character (32) should never be a word-separator. ;The "entry length" is the length of each word's entry in the dictionary table. (It must be at least 4 in ;Versions 1 to 3, and at least 6 in later Versions
;
;
       CLR  *R8            ok
       LI   R0,>2662    ; another pointer but hardcoded. $Fix to equate
       MOV  R0,@ZBB    ;save pointer to what we extract from dictionary (2662)

       MOV  @ZAV,R2    ;dictionary header 'n byte' [based on dictonary A008 value + >A000 offset]
       MOVB *R2+,R1    ;get 'n' total codes
       SRL  R1,8    
ZBC    MOVB *R2+,*R0+    and move the keyboard input codes into 2662+
       DEC  R1
       JNE  ZBC

;

; I -think- the following table represents the separators. Input routine doesn't seem to use them.
;      Codes:  space, enter, period, comma, question mark, backspace, termination(NULL)
;
; ZBE DATA >200D,>2E2C,>3F09,0     ;guessing this was moved earlier so it can be overwritten?
;
; It is possible this list (ZBE) is an artifact that is no longer needed,
; perhaps it was used before the interpreter processed the dictionary? Note that the 0x20 space character

; is in the list, yet the interpreter guide states that is not allowed as a word separator.

;
       LI   R1,ZBE    that are hardcoded at ZBE
ZBF    MOVB *R1+,*R0+
       JNE  ZBF        0?  no, get next

Link to comment
Share on other sites

I'm not 100% sure of what's going on here because I don't understand what the interpreter does. As far as I know, the parser is supposed to "process the input", then "recognize the words" (take a word, find it in the dictionary, match it with the corresponding word/token), then look to see if this corresponds to a line in the grammar. I believe the "process the input stage" is where stuff like "take key. open door with key" or "take key then open door with key" gets transformed in the right format. (But I can't remember if it's the parser or the interpreter that does this...)
I have a vague memory of a technical difficulty that can occur if you type "ask Ms. Belgrave about butler" in the Infocom detective games, because the "." is not a command separator then, but I don't know if it's the interpreter or the parser that is in charge of fixing that. (But maybe the period shouldn't be deleted in that case? I don't know) You might want to use "Deadline" or "Witness" to test if anything was broken.

Sorry, I'm not very helpful!

  • Like 1
Link to comment
Share on other sites

41 minutes ago, hlabrand said:

I'm not 100% sure of what's going on here because I don't understand what the interpreter does. As far as I know, the parser is supposed to "process the input", then "recognize the words" (take a word, find it in the dictionary, match it with the corresponding word/token), then look to see if this corresponds to a line in the grammar. I believe the "process the input stage" is where stuff like "take key. open door with key" or "take key then open door with key" gets transformed in the right format. (But I can't remember if it's the parser or the interpreter that does this...)
I have a vague memory of a technical difficulty that can occur if you type "ask Ms. Belgrave about butler" in the Infocom detective games, because the "." is not a command separator then, but I don't know if it's the interpreter or the parser that is in charge of fixing that. (But maybe the period shouldn't be deleted in that case? I don't know) You might want to use "Deadline" or "Witness" to test if anything was broken.

Sorry, I'm not very helpful!

no, this is helpful, as I hadn't even considered the cases in your examples!  My "paper trace" of the code strongly suggests that game file dictates the separation and the online documentation agrees with the code. As far as I can tell, the extra characters in the list are never compared against what has been parsed.  I'll need to run a quick debug session to confirm my suspicions.   Could I leave the extra characters in place?  Yes, but I'd like to clean house and have this well documented.   I like the idea of testing Deadline or Witness, and since the memory location is static in the current interpreters, I should be able to see what is in the older game dictionaries.  :)      

 

Here is what happens with Deadline if I use a period as a separator.  The action is parsed and acted upon but the game also 'complains' that a verb is needed.   When I check the debugger, the three separators (comma, period, quotes) were provided to the interpreter by the gamefile.

image.png.eb449a8f3e0fc8aba46a5cb58877f046.png

Link to comment
Share on other sites

I tested a few more game files and so far they all contain the same three separators in the dictionary: comma, period, and double quote.   The interpreter appends six more separators to the list:  space, enter, period, comma, question mark, backspace.  These separators are used by the parser to clean up the input for the word interpreter.  The good news is that I was able to document the routine and clear up a few of my misconceptions. The Infocom interpreter code can be almost as puzzling as one of their games!

  • Like 1
Link to comment
Share on other sites

I've identified all of the hard-coded screen output/input routines, so that I can now tell the interpreter to operate in either 40 or 80 column mode by adjusting a few DATA statements. Wordwrap was a bit tricky to find but as you can see, that is working as well.  I haven't yet finalized how to deal with the status line. 

 

image.thumb.png.bb54ec14b7d34b5be86d6865543376ed.png

(The 40 col display in an 80 col window is intentional - for debugging)

  • Like 5
Link to comment
Share on other sites

On 10/1/2020 at 7:25 PM, Torrax said:

Here's three small Z3 games

Made some progress with native Z3 files to cap the weekend.  Screenshot of DEJA VU running in the updated interpreter.  The only change to the file was to prepend a DIS/FIX128 TIFILES header via TI99Dir so that it could be recognized by the disk system.  

 

(Is unlocking the iron door possible with this demo?  One page I read says this isn't a game. If the iron key does unlock the door, the interpreter is not working properly with this z3 file).

 

image.thumb.png.90dde5db2f3f50d19ef74af4285476d0.png

  • Like 1
Link to comment
Share on other sites

  • 1 year later...

I was out of town for a bit over the holidays and had a few occasions when I could spend time reviewing the source code for the interpreter.  This time, I used paper and pen and highlighter to walk through the source and make my notes. There is just something about flipping through pages that I prefer :)   

 

I've replaced most of the hard-coded references (some easy to find, others not) within the disassembled code and made adjustments to a few more routines.  The result this weekend was a code base that can be assembled for either 40 or 80 column mode.

 

I committed the interpreter to the native z3 file format, and I've removed most of the code and prompts for the game1/game2 split files.  The interpreter still uses/must use VDP ram for its local buffer windows, and is at a premium for the F18A version and to a lesser degree the 9918 and V9938 versions. There are a few things I don't yet understand about the memory management routines that will require more time than I have right now to properly digest. 

 

With permission from @hlabrand, here is screenshot of the French version of Tristam Island, running in 40 columns, using the native z3 file and the previously implemented extended character set that @wierd_w created. (char table shown for testing only)

 

 

image.png.cbdd2b9de0c69fc17eb8de4024ad219b.png

 

 

  • Like 4
  • Thanks 2
Link to comment
Share on other sites

Last night I was testing some code to enable transcripting to a "disk" device.  I ran into a problem that I thought was of my own making until I eventually tested with the v1.2 and v1.3 interpreters, which also exhibit the same issue. The problem is that after encountering a file open error, the interpreter tries to flush the next entered command and then gets lost and/or restarts the story.   It would be most helpful if someone with a real system could test what happens if there is no RS232 in the system, to confirm whether a file error on a real system produces the same result.  To test, fire up any of the games (v1.2 or v1.3, including Lost Treasures of Infocom), type "script", select Rs232 or Pio, then type a command such as "version".   See picture below. 

 

image.png.dd9295a1ed236463abfd39beff029e5d.png

  • Like 1
Link to comment
Share on other sites

I ran into the same flushing problem with a 'disk' file IO operation.  It turns out that the transcription routine uses R9 to save a copy of R1 and unfortunately, one of the common display routines was not preserving R9.  So... R1 was being restored with a wrong value and the parser thought the user entered a few thousand words (into an 80 column stream, no less!) and justifiably attempted to purge the buffer beyond its limit. This cascaded into a bunch of other issues.   The interpreter is very stack oriented.  :)    Problem solved, for now!

 

 

 

  • Like 3
Link to comment
Share on other sites

SAMS would be a neat feature, but abuse of paged hyper-mini-mem, via FG99, would also be useful. Keep the interpreter and engine state in 32k, but have story data in 4k paged cartridge mem.  I would bet money more people have FG99s, but SAMS is more useful, program utility wise.

 

Maybe 2 versions?

 

 

  • Like 1
Link to comment
Share on other sites

10 hours ago, wierd_w said:

SAMS would be a neat feature, but abuse of paged hyper-mini-mem, via FG99, would also be useful. Keep the interpreter and engine state in 32k, but have story data in 4k paged cartridge mem.  I would bet money more people have FG99s, but SAMS is more useful, program utility wise.

 

Maybe 2 versions?

Possibly.  I don't own or use a FG99, as I am pretty well satisfied with my Ubergrom cartridge setup.  However... last year(2019?) I did verify that the interpreter could run from the MiniMem.  While programming the SAMS support, I decided to restrict memory access to one 4K window in the event that FG99 becomes something I pursue.  The nice thing about FG99 is the ram/rom mode, where 4k is used for ram and 4k for rom, meaning it could in theory support all available games if I understand the mode correctly.   A ROM cartridge with its 8k windows could also work for games <=22k (see below).*

 

The multiple version challenge stems from tight memory constraints:

 

Lower 8k RAM - stacks, buffers, and code

Upper 8k RAM - A000-F7FF (contiguous) is used for the read/write game data (up to 22k);  F800-FFFF is used for more interpreter code. 

For games that are larger (up to 24k), F800-FFFF is also used for game data and the 2k interpreter segment runs from the Supercart/mini mem. 

 

The above two scenarios call for two different memory models/versions, at present.

 

There is also a VDP memory requirement that in the past has required three distinct versions - 40 column 9918, 80 column F18A, and 80 column V9938.  I've reworked the code so that I can create these three VDP versions and with a little bit more rework, I should be able to consolidate into one.  All available, remaining VDP memory is used for internal buffers; the new extended character set reduces that space by 1-2 buffers. So far, no adverse results and I am looking at ways to restore at least one buffer, by using CALL FILES(2) and relocating the screen/pattern table, where possible.

 

The VDP memory rework is expected to reduce the interpreter count from six (3 video x 2 ram configs) to two (1 video x 2 ram configs) based on current progress. 

 

I had not intended to even consider a SAMS approach until I finished the basic system support but recent SAMS posts put me into a "I wonder if I understand the Interpreter well enough to implement something new" mindset. :) 

 

*Once I fully understand the remaining internal RAM/VDP secondary buffering routines, eliminating that code in favor of direcly accessing the rom/ram page (eg, 8k ROM cart or FG99) might free up just enough space to consolidate the 22k/24k.  Only time will tell -- standard/stock system is my first priority.

  • Like 1
Link to comment
Share on other sites

2 hours ago, wierd_w said:

One thing I have noted about paged minimem, is that it also pages ROM. Both ram and rom areas switch on page switch.

The mode I wanted to use is RAM mode, which independently banks ram and rom in 4k chunks.  This would be ideal.  See notes from the FG99 website.  Do you know if the FG99 RAM mode is emulated? I don't recall seeing it in js99er of Classic99.

 

In RAM Mode, each 8 KB bank is split into a ROM half-bank >6000->6fff and a RAM half-bank >7000-7fff. The RAM may be freely written to, but it is not battery-backed and will be lost on power down or cart reset. ROM half-banks are switched by writing to >60xx, and RAM half-banks are switched independently by writing to >68xx. Intermediate bits are again ignored. Note that writing to >7xxx will not switch banks but update the RAM at that location.

You can preset the RAM by putting the desired values in the upper half banks of your image. Only occupied banks contain RAM, so if you load a 4 bank image, you'll have 4 * 4 KB = 16 KB of RAM available.

Link to comment
Share on other sites

An initial comparative speed test using the internal $verify function, with a 126K z3 story file loaded entirely into SAMS:

 

$verify using the interpreter's retrieval routine:  ~48 seconds

$verify bypassing the same routines, using new direct SAMS access routine:  7 seconds

 

Directly accessing the SAMS memory is roughly 7 times faster.  I don't yet know if this could translate to a comparable performance increase for the Interpreter but it does give me a reason to pursue further analysis of the existing routines.

  • Like 4
Link to comment
Share on other sites

Further SAMS testing yielded unexpected results. I replaced the main interpreter's vdp-based cache/read routine with a SAMS read routine.  Paging the SAMS banks in/out for each byte reduced the interpreter speed by at least 30% during displays.  It seems that the 512-byte VDP caches are more efficient, even though the cached retrieval appears to execute more instructions.  I have retained the SAMS loader since it does enhance the overall gameplay, however, I have tabled further cache deep-dives to focus on a release.

Link to comment
Share on other sites

On 1/11/2022 at 2:18 PM, InsaneMultitasker said:

The mode I wanted to use is RAM mode, which independently banks ram and rom in 4k chunks.  This would be ideal.  See notes from the FG99 website.  Do you know if the FG99 RAM mode is emulated? I don't recall seeing it in js99er of Classic99.

@Asmusr will correct me if I'm wrong, but I do think he implemented FG99 RAM mode in JS99er.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...