TIPI - TI-99/4A to Raspberry PI interface development

[+jedimatt42]

TIPI Overview:

 

I am working on TIPI, with ElectricLab. It is an open sourced work in progress interface between a TI-99/4A and a Raspberry PI, so that we can provide a 'hosted' filesystem for the TI and leverage networking capabilities of the Raspberry Pi.
The initial designs are prototypes. ElectricLab has a pair wire-wrapped logic boards using 7400 series logic to address memory mapped IO with the Raspberry PI. He has built some really cool software off of that to get two 4A's interacting.

I have been working on making it play nice with the TI OS, giving it a proper DSR. My development borrows from ElectricLab, but I've implemented inside a Mojo Spartan 6 FPGA development board.

TIPI mission:

Provide memory mapped i/o to Raspberry PI for tinkering
Provide file on a disk Raspberry PI hosted filesystems to the TI-99/4A
Provide medium(TCP/UDP) and high(HTTP/HTTPS) level networking abstractions to the 4A, both in tinker mode and in TI-BASIC accessible Level 3 PAB DSRLNK mode.

Source and theoretical schematics are up on github:

https://github.com/jedimatt42/tipi/

 

TIPI Videos:

 

-----------------------------------------------------------------------------------------------------

The remainder of this thread should be for community visibility, and a constructive place to solicit advice on the compromises we will invariably have to make.

------ I opened this thread with the following questions / confusion --------

OK, I have a handle on PAB based DSR Rom entry points for the file usage model, referred to as Level 3:

OPEN, CLOSE, READ, WRITE, RESET, LOAD, SAVE, DELETE, SCRATCH, and STATUS...

I demonstrated at Fest West, the OPEN, CLOSE, READ, WRITE, LOAD all working (Although I have only performed a write to a TCP socket, not to a filesystem file yet)

But the Level 2 stuff is seriously an architectural problem... I have tracked down that in the DSR header, these use the facility for adding routines to TI BASIC from a DSR ROM, except the name of the 'CALL' is a single byte. What these bytes do, are the operations like rename, sector read/write, direct file input & output... etc... Leaving all of Level 2 subprograms out and only implementing Level 3 access through PAB would be too limiting.

It appears, that you cannot have two implementations of >13 ( TI FDC File Rename ) in a TI, for example, or at least only the one in the card with the lowest CRU address will be seen. These aren't like PAB routines, that look at the PAB, and say "nah, ask the next guy" ( PLEASE, SOMEONE SHOW ME THAT I'M WRONG ABOUT THIS ) They are each given minimal information such as the # of the DSK, but not the 'DSK' part. This leaves them assuming they are the unique implementor... Consequently, each device type has elected a different high nibble. ( Fred details the values in his 'Level 2 Subprograms' document on www.ti99-geek.nl )

From this I am assuming, you cannot use a Myarc harddisk controller and a Scuzzy controller in the same system, as they both use >20 as the base name for their subprograms.
You can't use 2 floppy controllers that respond to DSK either.

Client programs, such as Disk Manager 2, Myarc's Disk Manager V, and eventually Fred's DM2K, have to look at the device name (DSK, WDS, IDE, HDX) themselves, and then map that to the subroutine name to DSRLNK so they land on one in the correct DSR ROM. This means that each time a newly named device comes out, such as "TIPI.", you have to update any disk management software to know about what subprograms map to that devicename.

This isn't such a problem if I want to build TIPI with DSK1,2,3 and DSK.<diskname> support, and I implement the subprograms as the DSK device with a base of >10. However, existing disk managers won't know what I'm talking about if I use the device name "TIPI." Which I am want to do.

I have two goals for this project:

1st a sidecar variety, in which I could just pretend to be DSK. and DSKn. devices. But I really don't want to trick you into thinking it is simulating a floppy... I'm not trying to build a floppy replica. I will fall short of emulating a floppy. The project has no intention of handling disk image either, except to make it easy to drop them into the 'hard drive' as a directory of TIFILES and map DSKn to that directory.

2nd, to co-exist in the PEB with my TI FDC and HDX card.

Pretending to be a floppy drive with respect to these subprograms is fine, until I want to be able to copy data from a floppy to TIPI, or from TIPI to a floppy. If I want to do that, it seems I MUST pic a new base, and hack at, or beg Fred to update DM2K to understand TIPI ( is that "TIP" with a unit # of uppercase-i ? ) ,

or write a new file manager altogether. That wasn't expected to be in scope originally. I'd rather hard code aliases for SCS1. and IDE1. that map to TIPI. And put a jumper on the eprom so you can pick your damage.

---

There are questions in there somewhere.... Maybe in short, it is: What have I got right? What have I got wrong?

-M@

[Tursi]

The level 2 functions do rather expect that you either know which card to talk to already, or you only have a single disk driver in your system. I suspect most applications expect the latter. There's not really a lot of workaround for it.

 

<opinion mode on>

 

File copy is arguably the only function that needs to live out of that set with forward-looking devices, and to that end only the function that reads (edit: and writes, obviously) 'x' sectors of a file at a time. It's not trivial to copy files without that function, since you would need to identify the type and record size, and read/write appropriately. (Direct file access by raw sector makes too many assumptions about the disk layout to be a good practice moving forward.)

 

I'm not sure there's a good solution to having a program that doesn't expect multiple file drivers and uses that method of file copy. I think that if none of the existing programs support it (and I don't know whether they do), then addition of that support is the right answer. All file managers ask you what disk you're working from - so they COULD look up the device by name and thus also know which card they are dealing with for the low level functions. It's a little more complex, but it would be the "right" answer.

 

I've been toying in my head with methods for "overlaying" the disk DSR for my MPD adapter, so that it gets first stab at the disk access and then can relay it to the disk controller if it fails -- but that assumes you can get away with owning the system (and will probably require hardware mods). So it's probably not a good solution either.

 

I think your best bet for now is to go ahead and implement what you need - but I'm not convinced the problem of interworking between cards is the DSR's job. I think external software needs to do that.

Edited May 2, 2017 by Tursi

[+jedimatt42]

I am sure that interworking between cards is not the DSR's job. They just have to stay out of each other's way.

 

One should be able to use the level 3 status to gain the information required to copy files without using low level subprograms. Now I'm curious how hard that would be, and how much worse the record access will be wrt performance.

 

My intent is to offer a rich browser based file manager for anything that doesn't reach outside of TIPI.

 

Hmmm... There are definitely options here.

 

-M@

[Tursi]

For the record based files you could get away with it, but IMO it's annoying to track the details, but you're kind of screwed trying to copy PROGRAM images that way, you need a buffer the size of the file (which usually caps at 8k but doesn't have to).

[+InsaneMultitasker]

I'm just going to throw out a few things for conversation:

 

Level 2 is primarily used by terminal emulators (to transfer files), disk managers (to perform copy, rename, protect, subdirectory, and other related functions), archiver (to re-create the file based on the stored additional info block), and a handful of program types I've forgotten. Level 1 sector access shows up primarily in disk managers, sector editors, and sector-based disk catalog routines.

 

You -can- use more than one card with the same devices and/or subprograms. There is a working expectation that the DSRs for each card do not overlap. This is why Horizon RAMdisks generally start at DSK5. to avoid overlapping DSK1-4. Thus subprogram >13 would be found on the right card by disk (unit) number. If the cards (edit: and their subprograms) do overlap, the user is at the mercy of the CRU addresses and at which CRU the DSRLNK starts scanning for devices.

 

Hard drive devices follow a similar subprogram concept, though by doing so, TI and Myarc made a mess of future device access IMHO. To use level 2 routines you must provide the device number, the subprogram, and set the path. Unlike level 3, you can't just provide a device.path.filename to the DSR. So some programs test the device name to determine the proper card; others explicitly access the subprogram and roll the dice. The HFDC and SCSI cards use >2x subprograms. The IDE and HDX cards deal with device overlap by using unique subprogram prefixes: IDE is >8x; HDX is >9x. Compatibility with older programs is a mixed bag.

 

The Geneve went a different route by using the /full/ device.path.filename and a common PAB structure for -all- IO. There is no subprogram unit number foolishness. Perhaps this is an avenue you want to explore for your own means.

 

Now for a question... is there a reason you are wanting to alias the SCSI/IDE devices? If you want to meet goal 2, maybe the Classic99 floppy 'folder' approach would be sufficient for level 2 and level 3 access. Mapping a subdirectory as a "disk" seems like a good approach. (fyi, many programs also recognize disk "numbers" A-Z). For all other subdirectory IO you could limit the DSR to level 3 IO. Example: TIPI.SUB1.SUB2.file would be limited to open/save/load/close/write/status/delete. Most disk managers could still access the TIPI disk devices and copy to other system devices that are recognized. Use your browser or some other non-TI method for the heavy lifting?

 

Anyway, I don't think you need to mimic the hard drive device level 1/2 DSR routines to get most of the benefits of a folder-based file system. Tursi has proved much of that already. It really depends on where /you/ want to take this new device. (Maybe this is a good time for interested parties to review Tursi's earlier proposal, too)

Edited May 2, 2017 by InsaneMultitasker

[Tursi]

And a good reminder for me to put it up online, as I keep promising JediMatt I want his edits or at least suggestions.

 

https://docs.google.com/document/d/1zIMDd9gv5xTAa3ym_YcFc3FqLxDTFGIvHcY3xMVVEaw/edit?usp=sharing

 

(Ping me with your GoogleID if you want edit access )

[+jedimatt42]

And a good reminder for me to put it up online, as I keep promising JediMatt I want his edits or at least suggestions.

 

https://docs.google.com/document/d/1zIMDd9gv5xTAa3ym_YcFc3FqLxDTFGIvHcY3xMVVEaw/edit?usp=sharing

 

(Ping me with your GoogleID if you want edit access )

 

Edit request sent, thanks!

-M@

[+jedimatt42]

For the record based files you could get away with it, but IMO it's annoying to track the details, but you're kind of screwed trying to copy PROGRAM images that way, you need a buffer the size of the file (which usually caps at 8k but doesn't have to).

 

PROGRAM files seem to have a special bit set, but I thought they are also INT/FIX 0? (and 0 has dubious behavior) I've not tried to open one in that mode as of yet though... Per InsaneMultitasker's comments, this could be a highly misguided idea...

 

-M@

[+jedimatt42]

I'm just going to throw out a few things for conversation:

 

Level 2 is primarily used by terminal emulators (to transfer files), disk managers (to perform copy, rename, protect, subdirectory, and other related functions), archiver (to re-create the file based on the stored additional info block), and a handful of program types I've forgotten. Level 1 sector access shows up primarily in disk managers, sector editors, and sector-based disk catalog routines.

 

You -can- use more than one card with the same devices and/or subprograms. There is a working expectation that the DSRs for each card do not overlap. This is why Horizon RAMdisks generally start at DSK5. to avoid overlapping DSK1-4. Thus subprogram >13 would be found on the right card by disk (unit) number. If the cards (edit: and their subprograms) do overlap, the user is at the mercy of the CRU addresses and at which CRU the DSRLNK starts scanning for devices.

Ok, does this mean, if I get a call, and the unit # doesn't match the ones I want to handle, I can return without modifying the FAC space in scratchpad, and DSRLNK will keep searching?

 

Hard drive devices follow a similar subprogram concept, though by doing so, TI and Myarc made a mess of future device access IMHO. To use level 2 routines you must provide the device number, the subprogram, and set the path. Unlike level 3, you can't just provide a device.path.filename to the DSR. So some programs test the device name to determine the proper card; others explicitly access the subprogram and roll the dice. The HFDC and SCSI cards use >2x subprograms. The IDE and HDX cards deal with device overlap by using unique subprogram prefixes: IDE is >8x; HDX is >9x. Compatibility with older programs is a mixed bag.

I'm wondering if zero is a valid device number for level 2 routines? Is Classic99 the only occurrence of using DSK0?

 

The Geneve went a different route by using the /full/ device.path.filename and a common PAB structure for -all- IO. There is no subprogram unit number foolishness. Perhaps this is an avenue you want to explore for your own means.

 

Now for a question... is there a reason you are wanting to alias the SCSI/IDE devices? If you want to meet goal 2, maybe the Classic99 floppy 'folder' approach would be sufficient for level 2 and level 3 access. Mapping a subdirectory as a "disk" seems like a good approach. (fyi, many programs also recognize disk "numbers" A-Z). For all other subdirectory IO you could limit the DSR to level 3 IO. Example: TIPI.SUB1.SUB2.file would be limited to open/save/load/close/write/status/delete. Most disk managers could still access the TIPI disk devices and copy to other system devices that are recognized. Use your browser or some other non-TI method for the heavy lifting?

 

Anyway, I don't think you need to mimic the hard drive device level 1/2 DSR routines to get most of the benefits of a folder-based file system. Tursi has proved much of that already. It really depends on where /you/ want to take this new device. (Maybe this is a good time for interested parties to review Tursi's earlier proposal, too)

My ideas around pretending to be one of the pre-existing hard drives, was to not conflict with the TI FDC for subprogram names and be compatible with DM2K, but if calls to them can cascade, like calls to the level 3 operations can, then that shouldn't be necessary. In that case, I would prefer to just implement the routines for the DSK breed of disk, and then just allow 'TIPI.' and 'DSK0.' to be equivalent for greater legacy support. 'DSK1 - 3' can map to directories in the Raspberry Pi, and an archiver or DM2K might have a chance.

 

If I was the first one here, I'd probably read TI's spec and say 'OK, these level 2 routines are private extensions for the Disk Manager 2 cartridge' and look to extend the functionality in the PAB... I actually can't understand why this wasn't all done through PAB.. There is an entire byte of opcode, and a reference to whatever data you need.

 

I need to get one of the peb splitters So I can start testing level 2 behavior in the presence of an actual floppy controller.

 

-M@

[+jedimatt42]

(Maybe this is a good time for interested parties to review Tursi's earlier proposal, too)

And a good reminder for me to put it up online, as I keep promising JediMatt I want his edits or at least suggestions.

 

https://docs.google.com/document/d/1zIMDd9gv5xTAa3ym_YcFc3FqLxDTFGIvHcY3xMVVEaw/edit?usp=sharing

 

(Ping me with your GoogleID if you want edit access )

I have implemented portions of it. It is really well thought out. The hashing mechanism for long file names has been implemented, and works quite well. I'm only hosting TIFILES, ignoring anything else right now. I'd like to end the endeavor having this fully implemented ( after any revisions to the proposal are ironed out )

 

ElectricLab has begun a web browser based file manager for up-leveling the hosted filesystem based file management, and I imagine we'll put V9T9 and DSK image conversion in there, and continue to only support TIFILES, and eventually some native to TI file auto-conversions in the actual filesystem.

 

-M@

[+InsaneMultitasker]

Ok, does this mean, if I get a call, and the unit # doesn't match the ones I want to handle, I can return without modifying the FAC space in scratchpad, and DSRLNK will keep searching?

 

Yes, it should keep searching. This is the primary means for DSRLNK to find the right device (by device or subprogram) and should continue until it exhausts the CRU addresses it is programmed to scan. How you return control to the DSRLNK routine is important, ie, EQ bit, status byte, return address, and error return, within the context of devices or subprograms (the two use slightly different error reporting). The Interface Standard Design & Design Guide contains a lot of good DSR-related reference material.

 

I'm wondering if zero is a valid device number for level 2 routines?

Your DSR determines what is a valid device number, though programs may try to be "smarter than" the DSR. RAMdisks use drive numbers and letters A-Z, the Foundation card and Myarc RAmdisks use a number or letter, and there are other examples beyond the traditional DSK1-DSK3. To be honest, I've never tried to use device 0.

 

The usual caveats apply: programs might not accept device 0 or units >3 or letters. Programs that test the device number usually use '1' as the lower boundary, with the upper boundary loosening over the years as more hardware was released. Keep in mind some specification requirements were written to conform to the existing hardware, so new devices often struggled with earlier "requirements" that were too limiting.

[+jedimatt42]

Yes, that design guide has been valuable. Along with the disassembled code in there.

 

This subprogram area is where it is weak. Basic CALLs are treated as unknowns, but at least there is a disassembled CALL FILES.

 

This entire space is made extra fragile by every other application using its own implementation of DSRLNK.

 

It sounds like I should adopt this strategy:

 

* Level 3 access for "TIPI.", "DSK.", and "DSKn."

* DSK1-3 mapping (running at cru1000)

* Level 3 cascade support (return as unhandled when appropriate)

* Provide basic CALL for mapping "DSKn" to TIPI folders. Remember to provide unmap as well.

* Level 2 support for "DSKn" w/cascade if not mapped.

* Provide the browser/phone based disk manager for TIPI

* Provide a TI copy oriented file manager for legacy<->"TIPI." (I can use unit #0 for "TIPI.")

 

After hearing this, I think I should stick with >1x level 2 subprogram names.

 

I've tested that if using my mapped DSK1. devicename, DM2K will call >18 when I tell it to create a directory. (My implementation of >18 is BLWP @>0000.) I should be able to test cascade if I use one of the TI FDC supported level 2 subprograms implemented as RT. And I think I know how to prove that out in Classic99. If I step through that in the debugger I should be able to understand this better. Reading assembly on paper is not one of my strengths.

 

-M@

[+jedimatt42]

I've tested that if using my mapped DSK1. devicename, DM2K will call >18 when I tell it to create a directory. (My implementation of >18 is BLWP @>0000.) I should be able to test cascade if I use one of the TI FDC supported level 2 subprograms implemented as RT. And I think I know how to prove that out in Classic99. If I step through that in the debugger I should be able to understand this better. Reading assembly on paper is not one of my strengths.

 

-M@

Ok, that experiment worked well! With my DSR at CRU 1000, and the Classic99 DSR at 1100, I used the FBFORTH CAT package to trigger some sector I/O. I saw it call my implementation, which just RT'ed. Stepped back into DSRLNK, and watched it look up and use the next implementation.

 

I guess when I realized these share the same header linked list as the basic subprograms, my brain broke, and I made assumptions. It is restricted, in that you only get a unit # instead of the full device name in question.

 

I'm assuming again ( I'll get in trouble here soon ) that the reason we have setpath is to inform routines like file rename and createdir about where to operate, since unit # is no longer sufficient when you have directories. This is all beginning to seem more logical.

 

-M@

[+InsaneMultitasker]

I'm assuming again ( I'll get in trouble here soon ) that the reason we have setpath is to inform routines like file rename and createdir about where to operate, since unit # is no longer sufficient when you have directories. This is all beginning to seem more logical.

 

Correct, the setpath routine is used as you describe for hard drive operations.

 

Setpath is generally not used by floppy devices, as the floppy operates on the unit number. Subdirectories are generally not used on floppy devices in the TI environment.

[+mizapf]

Apart from the HFDC and Geneve users. I do have some floppy disks with subdirectories.

[+jedimatt42]

I won't be emulating floppies, as much as aliasing DSK. and DSKn. So that some assumptive software can be used.

 

DSK1 will still be a place on the hosted filesystem that can have directories.

 

-M@

[+jedimatt42]

I've renamed this thread, and added a pre-amble to the first post:

 

http://atariage.com/forums/topic/265136-tipi-ti-994a-to-raspberry-pi-interface-development/?do=findComment&comment=3753750

 

I am working on a series of short videos to showcase what is working so far. And hopefully I'll do one on how it works.

 

The first is here:

 

 

-M@

[sparkdrummer]

Freakin' Awesome!!!

[+jedimatt42]

Thanks! Here is another:

 

 

I think I have 2 or 3 more little videos to get everyone caught up

 

-M@

[+jedimatt42]

Here I describe the intention for a web based administration tool, and show what we have now, that just does listing information. I also show how to find the Raspberry Pi's ip-address from the TI.

 

 

-M@

[sparkdrummer]

This device will be coupled to your 32k sidecar, correct?

Can( or will) it be able to be coupled to a full blown peb with one Ksarul's splitters?

I for one am very glad you're a TI'er kid.

[+jedimatt42]

A bunch of my noise at the top of the thread, is trying to figure out exactly what I have to care about to co-exist properly with a real floppy disk controller connected at the same time.

I believe what I've implemented so far is taking the correct steps, but I haven't tested it in that environment yet.

 

I am targeting two form factors for pcboard design:

1. a daughter board for my 32k sidecar boards.

2. a PEB card (this is the version I want to use!)

 

The DSR will be the same for both.

 

ElectricLab and I met up for lunch today, and agreed to focus on the pcboard design for the 32k daughter board next. I'm hopeful that it can be the same size as the 32k sidecar. However, we're getting geared up for surface mount tech on this. Some of you saw the wire wrap versions in the game demo at Fest West. That was part of the logic we need using the 7400 series dip chips.

 

For the PEB card version, I may take a stab at using thru-hole dip chip based gal's so a version of this is accessible without going to surface mount.

 

We'll see.

 

Sockets for the DSR Eprom, on either board. So we can keep that simple, and updates accessible.

 

-M@

[chouimat]

cool

[Omega-TI]

Matt,

I ripped the image below out of one of your YouTube videos. I just had to tell you, it was really gratifying to see the "TIPI" addition on the startup screen. It's kind of like the wrapping on a Christmas present... knowing something good is inside. I can't wait to see the finished product.

 

 

[+jedimatt42]

Next installment: TIPI does T C P... In this video you can hear that the TE II speech algorithm can pronounce TIPY correctly if you spell it "TIPPY" Of course that isn't the point. The point of this video is TCP socket access, and HTTP protocol access possibilities... enjoy!

 

 

-M@