New HDX crossplatform server, and HDX hardware design discussion

[ckoba]

 

I've been having a good long think about this. A nice touch would be:

 

When the server receives a filename to open over the wire:

• If the file is a TIFILES or V9T9 file, then open the file according to the parameters in the file header
• If the file is a "PC" file (i.e. has an extension) then open the file according to a translation table (could be a simple tab delimited text file), for example:
  • .txt --> DV80
  • .obj --> DF80 (EA#3)
  • .pgm --> Binary (EA#5)

However, this would depend on how Fred's DSR is currently working. For example, if you open Editor Assembler, and attemp to edit a Source file called HDX1.SOURCE.TXT does it pass the ".TXT" over the wire, or does it barf? If it passes the extension over the wire then the server can see that and behave accordingly. The benefit to all this is that one could create files on a PC and not convert them at all, and, in fact, create files "TI-side" and store them as native PC files. One issue with V9T9 and TIFILES files is that, when viewing them on a PC one cannot identify the type of the file without using additional software such as TIDIR or TIImageTool.

 

This will be done. Fred's DSR doesn't mangle anything; it depends on the server to make decisions like this.

 

There will be a minimal (and optional) configuration file to do file mappings. Some mappings will be built in -- the ones you listed above, for example.

[ckoba]

Thank you for making this portable version, it's one of those things that has always kept me back from using an HDX.

 

No worries. The only reason I haven't ripped the HDX off of my serial board is to get this finished. Glad that my work won't be in vain.

[ckoba]

 

Fred's protocol doc references both level 2 and level 3 operations - you can see how they differ through a quick analysis.

 

Fred's protocol doc matches Fred's HDX server behavior only about 80%. An operation that you'd think would consist of all level-three operations -- like, say reading or writing a disk via DSK2PC would normally be an open, interative read/write, and close -- is, instead, a bunch of level 2 'R' or 'W' calls, no open and no close. I'd give him the benefit of the doubt and say that it's handled on level 2 because it's operating on the sector level, but then I see the directory routines and conclude that it's a mismash.

 

I'm guessing this is the complexity that you were referring to when you mentioned the HFDC? If I were designing it, the direct I/O (operations that manipulate data) would all live on level 2, and level 3 would sit on top of and use the level 2 ops to provide a higher-level abstraction (keeping track of file descriptors, directory information, and suchlike). Under normal circumstances a user program would not use level 2 unless it really, really needed to flip bit 192 on sector 14.

 

But that's just my frustration talking -- this is what it is, and the more I poke at the internals of the TI the more I'd like to go back in time and harangue the engineering team at their local watering hole, and perhaps ask to be hooked up with their supplier. Clearly, whatever the TI-99/4(A) teams were taking was some really high-quality stuff.

[ckoba]

Quick poll: how often do you need to reload your HDX DSR, and why? Sudden blue screen at boot? Corrupted transfers? Other?

 

I know, for a fact, that my serial board is good. I desoldered every non-proprietary IC and replaced them with a) sockets and b) either 74ALS or 74ACT, take your pick. I've hooked up an oscilloscope to pin two (or was it three?) of the second 9901, and I've watched the bits go across the wire.

 

I've built my board as it should be, using both 6264 and 62256 (with a ground wire for A14), and I know it's good to go. I *know* hardware.

 

Yet, after dozens of aborted transfers (due to misprogramming, mostly) the (expletive deleted) thing stops working because (I think) those aborted transfers aren't freeing buffers on the DSR side, and there seems to be no constraint enforced. Thus, I think that the RAM DSR is being killed when I reach a threshold of open files/buffers ...

 

... which leads me to ask: how (expletive deleted) reliable is this (expletive deleted) thing under non-destructive, normal use? Does it whack out every so often, and y'all just take it for granted, or is it rock-solid and the DSR never needs reloaded?

 

(*this* is why it should be an EPROM/RAM setup. Problem? Power-cycle, and you're back to a known good configuration. With the whole thing in battery-backed RAM, you're (expletive deleted) without cracking open the PEB and removing the battery for about a minute. Ask any engineer that works on gear intended for use by the great unholy unwashed {Ksarul, sir, I'm looking in your direction}, mention critical startup code in battery-provided NVRAM, and watch them gibber ...)

 

And as an unrelated followup: there is no (expletive deleted) reason why DSK2PC requires a HDX. Period, full stop, end of story. (Expletive deleted) thing can be accomplished using the stock DSR at 38400 using {X,Y,Z}MODEM on the serial side and straight byte read/writes on the TI side. Much more reliable than the crap that is DSK2PC, works with more hardware (including nanoPEB and Corcomp), and doesn't use an additive checksum for verification. Eliminating DSK2PC from the HDX inventory of features is a good thing.

Edited February 18, 2016 by ckoba

[Omega-TI]

Quick poll: how often do you need to reload your HDX DSR, and why? Sudden blue screen at boot? Corrupted transfers? Other?

 

Ah sooo, the Blue Screen of Death problem. Been there done that. I was pulling my hair out too, for a while... but $1.08 at my local Dollar Tree fixed my problem. Since << THIS POST >> I've not had to reload my DSR since. I' have installed a couple of switches, one on my interrupt line the other with the battery as insurance just in case, because pulling the card can be a pain. Like all insurance, not that I have it, nothing bad happen... funny how that works.

 

Looking back at that post I just realized I've not changed that battery or updated the DSR since NOVEMBER of 2014 and it's still working! DAMN!

 

On the DSK2PC program, it works and works great, so I've never asked questions. I don't have to fuss with stuff, it's fairly auto-magical. If I had anything negative to say about it, it would be that I can't do multiples at the same time and stack them all in a que sequentially. I have to come back and manually start the next, but since I don't use it every day, no biggie I guess.

[ckoba]

 

Ah sooo, the Blue Screen of Death problem. Been there done that. I was pulling my hair out too, for a while... but $1.08 at my local Dollar Tree fixed my problem. Since << THIS POST >>

 

(snip)

 

On the DSK2PC program, it works and works great, so I've never asked questions. I don't have to fuss with stuff, it's fairly auto-magical. If I had anything negative to say about it, it would be that I can't do multiples at the same time and stack them all in a que sequentially. I have to come back and manually start the next, but since I don't use it every day, no biggie I guess.

 

Battery is new, tests at +3VDC at both no load and a 10k resistor. The HDX transfers stop working while the PEB is powered on, which mostly eliminates a bad battery from the equation. Power-cycling the console and PEB after the glitches results in a blue screen, which means ...

 

... the problem isn't that the battery isn't doing the needful; problem is that the DSR is overwriting itself. This is why you don't use RAM for these things. Have I beat that horse to death yet? Because I can't stress the point enough -- don't (expletive deleted) use BBR for peripheral firmware. And your instructions on the link may well be wrong -- the critical action is to REMOVE POWER FROM THE DSR RAM. What happens afterwards depends on your use of the HDX.

 

Thanks for the data point, though: one user has no problems when the battery is new-ish.

 

(and DSK2PC is written *wrong*. Ever wonder why the sector transfer count is off by one? Or why it sometimes doesn't work until you use the HDX with something else, like DM2k?)

 

(edit: lots of stuff)

Edited February 18, 2016 by ckoba

[Omega-TI]

Ever wonder why the sector transfer count is off by one? Or why it sometimes doesn't work until you use the HDX with something else, like DM2k?)

 

(edit: lots of stuff)

 

On the sector count being off by one, I've always assumed that the count code was part of an operational loop that is not returned to when it gets to the end of the copy routine, so that is why the last increment is never displayed. Yeah, it's noticeable, but the program still performs it's function.

 

On the second, I've NEVER experienced that problem with the HDX, but for some reason I've gotten into the habit of loading the HDX server on the PC before I power up the TI. There is probably a reason for that, I just cannot remember why.

 

Now I really love the HDX, as with all things in life nothing is ever perfect, but there are a lot of us out here that use Fred's little gadget and it has served us all quite well for a number of years.

[+InsaneMultitasker]

 

Fred's protocol doc matches Fred's HDX server behavior only about 80%. An operation that you'd think would consist of all level-three operations -- like, say reading or writing a disk via DSK2PC would normally be an open, interative read/write, and close -- is, instead, a bunch of level 2 'R' or 'W' calls, no open and no close. I'd give him the benefit of the doubt and say that it's handled on level 2 because it's operating on the sector level, but then I see the directory routines and conclude that it's a mismash.

 

I'm guessing this is the complexity that you were referring to when you mentioned the HFDC? If I were designing it, the direct I/O (operations that manipulate data) would all live on level 2, and level 3 would sit on top of and use the level 2 ops to provide a higher-level abstraction (keeping track of file descriptors, directory information, and suchlike). Under normal circumstances a user program would not use level 2 unless it really, really needed to flip bit 192 on sector 14.

 

 

Sector IO is level 1. In this case, disk access is handled through sector reads and writes. Fred's DSR does not emulate the disk file system - only the sectors are accessible. Thus you are seeing proper operation, errors in sector count notwithstanding.

 

I recommend that you review the following document as it explains the three levels in a bit more detail. There is a method to the madness; however, if you are trying to understand the TI file system by inspecting Fred's DSR and looking at bits over the wire, you are missing pieces of the puzzle.

 

GPL Interface Specification for the 99_4 Disk Peripheral V2.0 03-28-1983.pdf

[+InsaneMultitasker]

Quick poll: how often do you need to reload your HDX DSR, and why? Sudden blue screen at boot? Corrupted transfers? Other?

 

I know, for a fact, that my serial board is good. I desoldered every non-proprietary IC and replaced them with a) sockets and b) either 74ALS or 74ACT, take your pick. I've hooked up an oscilloscope to pin two (or was it three?) of the second 9901, and I've watched the bits go across the wire.

 

I've built my board as it should be, using both 6264 and 62256 (with a ground wire for A14), and I know it's good to go. I *know* hardware.

 

Yet, after dozens of aborted transfers (due to misprogramming, mostly) the (expletive deleted) thing stops working because (I think) those aborted transfers aren't freeing buffers on the DSR side, and there seems to be no constraint enforced. Thus, I think that the RAM DSR is being killed when I reach a threshold of open files/buffers ...

Are you sure you don't have a finicky flex cable interface or GROM port?

 

In theory, you should be able to try opening multiple files (without closing them) to simulate the buffer problem. Fred's DSR does appear to test the maximum buffers allocated and should generate an error if this is exceeded.

 

 

 

And as an unrelated followup: there is no (expletive deleted) reason why DSK2PC requires a HDX. Period, full stop, end of story. (Expletive deleted) thing can be accomplished using the stock DSR at 38400 using {X,Y,Z}MODEM on the serial side and straight byte read/writes on the TI side. Much more reliable than the crap that is DSK2PC, works with more hardware (including nanoPEB and Corcomp), and doesn't use an additive checksum for verification. Eliminating DSK2PC from the HDX inventory of features is a good thing.

 

I disagree. The server option to transfer virtual disks by sector is a positive feature. One cannot (currently) use x/y/zmodem to transfer diskettes because the available terminal emulators do not support disk image transfers in the manner you describe.

 

I cannot speak to reliability beyond noticing you are having a lot of problems with the HDX that others either (1) are not experiencing or (2) are not reporting.

[+arcadeshopper]

I for one have only had to reload a DSR on my HDX after a major crash. I use it all the time to move files over from the PC without having to reload. I have had the same batteries in my 4/a since I built the hdx daughterboard and installed it.. which has been over 2 years. I'll turn on the geneve this weekend and test it's HDX DSR it's been off for months so a good test of the battery.

 

Greg

[ckoba]

Are you sure you don't have a finicky flex cable interface or GROM port?

 

In theory, you should be able to try opening multiple files (without closing them) to simulate the buffer problem. Fred's DSR does appear to test the maximum buffers allocated and should generate an error if this is exceeded.

 

 

 

I disagree. The server option to transfer virtual disks by sector is a positive feature. One cannot (currently) use x/y/zmodem to transfer diskettes because the available terminal emulators do not support disk image transfers in the manner you describe.

 

I cannot speak to reliability beyond noticing you are having a lot of problems with the HDX that others either (1) are not experiencing or (2) are not reporting.

 

I'm pretty sure the flex cable and its chips are okay -- I had the misfortune to blow a '244 on the card side by forgetting to power down the PEB before yanking the serial card for yet another battery disconnect, and I tested everything that had been socketed. The bad 244 tested bad; replacing it returned the PEB to functioning state (i.e, could see the rest of the cards).

 

I have replaced the '244s on both the card and the host interface with ACT. That *might* have something to do with it, but I doubt it, as their specs are superior to the LS they replaced. The '245s are untouched, because I didn't have any ACTs on hand the last time I was in a chip-replacing mood.

 

I'll perform a continuity check on the cable itself later today, but I doubt I'll find anything.

 

The nature of the error is that the checksum itself as received by hdx.py is bad. If you look at the code, you'll see places where I've either commented buffer hex dumps or uncommented them. That's so I can calculate the checksum by hand, and see if I'm dealing with Yet Another Fred Discrepancy between wire protocol and spec, or if I'm dropping packets.

 

In nearly all cases, the problem is a bad checksum, and it's caused by missing the read window for that byte (which shouldn't happen). There's a specific place in the 'R'/'W' routine where I set the number of bytes to be received to either 15 or 16 (working from memory here) depending on which command. That's because receiving 15, doing the check, and then receiving another for checksum resulted in a corrupted or missed checksum byte. Reading off the wire as much as possible, parsing, writing disk buffers AND FLUSHING THEM, then acking, seems to placate the checksum monster. This thing is horribly timing-sensitive, and that's part of my problem.

 

The other half is that I'm testing this thing to destruction (as Ksarul would say). My general test case is to use DM2K to read the directory, select a file, and then try to look at it using "B". hdx.py will bomb out somewhere along the line because it's a work-in-progress (natch), and the DSR will either time out and throw an error 6 or die immediately with an error 6. In either case, fifty-plus repetitions of this results in sudden corrupted checksums, which indicates that the DSR will go completely silent soon, and a reboot will bluescreen.

 

That doesn't sound like dodgy hardware to me, especially when *everything* else in the PEB works (including the p-code card). That sounds like DSR corruption caused by overrunning a buffer and wrapping the write around to scribble on the DSR. Others aren't seeing this because they're not beating the bejesus out of the thing; if the server behaves (as Fred's Windows implementation does), then the DSR is fine. It's only when you've got a broken server implementation (as hdx.py is at the moment) that you'll see problems.

 

Regarding the ZMODEM thing, I'm dead serious, but perhaps not in the way that you think. DSK2PC should use the ZMODEM protocol (or YMODEM) for wire transfer. I'm not talking about using telco to transfer files; it's obvious that the TI side needs direct sector access, so a standalone program like DSK2PC is needed. The problem that ZMODEM addresses is the wire transfer protocol (and, specifically, the hardware's complete lack of handshaking), and the need for a special DSR (read: HDX) just to perform the transfer. DSK2PC is IMO using the wrong tool set to perform its task. It should be talking to (whatever the UNIX incarnation of Telix is) and transferring from PC using a real transfer protocol, not something stapled into a DSR that has *no* real error recovery.

[ckoba]

I for one have only had to reload a DSR on my HDX after a major crash. I use it all the time to move files over from the PC without having to reload. I have had the same batteries in my 4/a since I built the hdx daughterboard and installed it.. which has been over 2 years. I'll turn on the geneve this weekend and test it's HDX DSR it's been off for months so a good test of the battery.

 

Greg

 

Thanks for that.

[ckoba]

 

On the sector count being off by one, I've always assumed that the count code was part of an operational loop that is not returned to when it gets to the end of the copy routine, so that is why the last increment is never displayed. Yeah, it's noticeable, but the program still performs it's function.

 

On the second, I've NEVER experienced that problem with the HDX, but for some reason I've gotten into the habit of loading the HDX server on the PC before I power up the TI. There is probably a reason for that, I just cannot remember why.

 

Now I really love the HDX, as with all things in life nothing is ever perfect, but there are a lot of us out here that use Fred's little gadget and it has served us all quite well for a number of years.

 

Well, you have to start the server before the TI comes on. Part of the DSR powerup routine is the port probe -- one transmission of the init string, then six or so hash marks if the server doesn't respond.

 

If the server isn't running when that probe happens, the DSR concludes that there is no HDX server on port 2 and therefore ignores it.

 

(regarding "perfection" -- yes, I'm a perfectionist. That goes with the territory. At this point, I'd settle for "stability". There's a lot of stuff I'm seeing in the behavior of the HDX when it talk to the server that I would personally do differently, but I let it go because I'm not going to rewrite whatever's causing it (DM2K? DSR?). Things like the sector counter, though ... that's just sloppy. And things like that make an engineer wonder what other sloppy shortcuts were taken in code by the author. What was the adage ... "the fundamental design flaws were completely obscured by the cosmetic design flaws"?

 

I'm getting to the point where I should take some time off from this project. I obviously don't think the HDX, as implemented, is a good thing. Nor would I use it once I completed hdx.py because *I do not trust it*. Perhaps someone who actually cares about the HDX and wants to use it without running a Windows emulator should take over. It'd be a good excuse for someone to learn python

[Willsy]

My experience with the HDX is that it's buggy. I was one of the first people (other than Fred) to get a HDX. In fact, he personally built mine - I sent him my RS232 board and he did the job himself (including opening the clan she'll up with a dremel to accommodate the HDX PCB!).

 

I used it for TurboForth. When I first got it it was quite unusable. There was quite a lot of emails between Fred and I and he made both DSR and server chances IIRC.

 

TurboForth only does level 3 disk access. When reading or writing blocks to/from disk it opens the file, does what it needs to do, and closes the file. Then it does the next block (open, read/write, close). This is slow but the disk access to blocks in TF is rock solid so I live with it.

 

This always caused problems with HDX and I had to give up using it. It would work for maybe 10 minutes then stop working. IIRC rather than crash it would just become increasingly erratic.

 

I get the feeling that if you're using it for just straight-up loading of programs then it's pretty good, but I was doing 100% file i/o (DF128) and it wasn't reliable long-term.

[ckoba]

My experience with the HDX is that it's buggy. I was one of the first people (other than Fred) to get a HDX. In fact, he personally built mine - I sent him my RS232 board and he did the job himself (including opening the clan she'll up with a dremel to accommodate the HDX PCB!).

 

I used it for TurboForth. When I first got it it was quite unusable. There was quite a lot of emails between Fred and I and he made both DSR and server chances IIRC.

 

TurboForth only does level 3 disk access. When reading or writing blocks to/from disk it opens the file, does what it needs to do, and closes the file. Then it does the next block (open, read/write, close). This is slow but the disk access to blocks in TF is rock solid so I live with it.

 

This always caused problems with HDX and I had to give up using it. It would work for maybe 10 minutes then stop working. IIRC rather than crash it would just become increasingly erratic.

 

I get the feeling that if you're using it for just straight-up loading of programs then it's pretty good, but I was doing 100% file i/o (DF128) and it wasn't reliable long-term.

 

Thank you VERY MUCH for chiming in with this. Really, I appreciate it. With everyone else saying "it works for me, there must be something wrong with your setup" and suchlike, I was starting to doubt myself.

 

That very closely mirrors my experience with it when I'm doing a protocol implementation/debug session. It starts out fine, then as I add more functionality, it starts flaking out until it just refuses to send commands at all. I haven't even gotten to the point where I'm loading files -- I'm still getting the directory parsing and translation done, which means a lot of mixed L2/L3 and unclosed buffers when I screw up hdx.py.

 

Going a bit further, I'm not convinced that the TI serial hardware is up to this sort of thing at all. There's no provision for hardware handshaking whatsoever -- or, rather, it's all one-way with the TI not even having the pins for the other side wired in at all. The DSR isn't doing XON/XOFF, so no software handshaking either. The closest thing for handshaking is sending "#" when the last message didn't parse, and all that does is have the other side retransmit its last packet.

 

Using a straight additive eight-bit checksum doesn't give me much confidence in the integrity of the data, either. There was a reason that XMODEM moved to CRC back in the days of yore.

 

Fred had the right idea at some point -- building a card, sticking RAM and a different UART on it (16x50, but a 6551 would work just as well), and using just that. Assuming the DSR were competently written (Harvard architecture -- code in read-only space, data in read/write, no mixing), that would have handled the data integrity issue. I wonder why he abandoned that. No, strike that, I might not like the answer. Going forward, though, the hypothetical HDX2 would be a separate PEB card so the user doesn't have to (expletive deleted) up his stock serial card for this.

 

All that having been said, the apparent common use case is to load files from server to TI. That does have rudimentary hardware handshaking, as the CTS output line is wired and might actually be functioning. The TI is never going to consume data faster than it's being sent from the server, so buffer underruns just aren't going to happen.

 

If it was unreliable for you doing 100% file I/O, that further points to badness in the DSR regarding file buffers and file descriptors.

 

In summary: thank you very much for piercing the echo chamber here. It means a lot to hear someone of your experience and stature say "yeah, it was unreliable, stopped using it", instead of questioning my competence/intelligence

 

(edit: s/DTR/CTS/ -- Fred unfortunately uses DTR to detect link rather than handshake, which is not a good idea)

Edited February 19, 2016 by ckoba

[Willsy]

Yes, I'm personally of the opinion that a dedicated board for the PEB would be a better idea with a brand new custom DSR. That is something I would be interested in pursuing. I've never written a DSR but I think I could do it and I've lots of experience with doing serial comms in days gone by. However I lack the hardware skills.

[ckoba]

Yes, I'm personally of the opinion that a dedicated board for the PEB would be a better idea with a brand new custom DSR. That is something I would be interested in pursuing. I've never written a DSR but I think I could do it and I've lots of experience with doing serial comms in days gone by. However I lack the hardware skills.

 

I can design the board -- hell, it's already designed and on Fred's website, albeit in KiCAD rather than Eagle.

 

I can put together an Eagle board schematic and .brd suitable for OSHPark in a couple of days max -- and I'd use discrete logic instead of the GAL he's using for address decode, so it can be built by someone with only rudimentary soldering skills.

 

Really want to do this? I have a few of Jim's PEB proto boards that I can use to get the design nailed down, and if you're okay with writing a DSR and burning it to an EPROM, I can build you one and send it to you. You choose the UART, based on what you're familiar with. (I'd personally go for the WDC65C51 if and only if they fixed that TX-buffer-empty-bit-means-nothing bug)

Edited February 19, 2016 by ckoba

[+Ksarul]

I haven't used HDX at all, so I can't feed too much useful data into that part of the conversation, but building a PEB HDX2 board using the Protoboards looks like it might be a good starting point for some real hardware goodness. I can't put that into my queue right now (too many other projects grinding forward with glacial slowness), But I may be able to contribute some tips to anyone else taking a stab at it. . .

 

And I like the methodical way you've been testing this to destruction, Chris--hardware projects need that kind of rigorous testing to flush out the bugs that would otherwise only serve to annoy the unlucky end user if they actually get into a production product. . .and I will definitely be bumping some of my near-term hardware projects up against you (if you are interested in doing such) to make sure they are ready for prime time.

[+InsaneMultitasker]

 

I'm pretty sure the flex cable and its chips are okay -- I had the misfortune to blow a '244 on the card side by forgetting to power down the PEB before yanking the serial card for yet another battery disconnect, and I tested everything that had been socketed. The bad 244 tested bad; replacing it returned the PEB to functioning state (i.e, could see the rest of the cards).

 

I have replaced the '244s on both the card and the host interface with ACT. That *might* have something to do with it, but I doubt it, as their specs are superior to the LS they replaced. The '245s are untouched, because I didn't have any ACTs on hand the last time I was in a chip-replacing mood.

 

I'll perform a continuity check on the cable itself later today, but I doubt I'll find anything.

 

The nature of the error is that the checksum itself as received by hdx.py is bad. If you look at the code, you'll see places where I've either commented buffer hex dumps or uncommented them. That's so I can calculate the checksum by hand, and see if I'm dealing with Yet Another Fred Discrepancy between wire protocol and spec, or if I'm dropping packets.

 

In nearly all cases, the problem is a bad checksum, and it's caused by missing the read window for that byte (which shouldn't happen). There's a specific place in the 'R'/'W' routine where I set the number of bytes to be received to either 15 or 16 (working from memory here) depending on which command. That's because receiving 15, doing the check, and then receiving another for checksum resulted in a corrupted or missed checksum byte. Reading off the wire as much as possible, parsing, writing disk buffers AND FLUSHING THEM, then acking, seems to placate the checksum monster. This thing is horribly timing-sensitive, and that's part of my problem.

 

The other half is that I'm testing this thing to destruction (as Ksarul would say). My general test case is to use DM2K to read the directory, select a file, and then try to look at it using "B". hdx.py will bomb out somewhere along the line because it's a work-in-progress (natch), and the DSR will either time out and throw an error 6 or die immediately with an error 6. In either case, fifty-plus repetitions of this results in sudden corrupted checksums, which indicates that the DSR will go completely silent soon, and a reboot will bluescreen.

 

That doesn't sound like dodgy hardware to me, especially when *everything* else in the PEB works (including the p-code card). That sounds like DSR corruption caused by overrunning a buffer and wrapping the write around to scribble on the DSR. Others aren't seeing this because they're not beating the bejesus out of the thing; if the server behaves (as Fred's Windows implementation does), then the DSR is fine. It's only when you've got a broken server implementation (as hdx.py is at the moment) that you'll see problems.

 

Regarding the ZMODEM thing, I'm dead serious, but perhaps not in the way that you think. DSK2PC should use the ZMODEM protocol (or YMODEM) for wire transfer. I'm not talking about using telco to transfer files; it's obvious that the TI side needs direct sector access, so a standalone program like DSK2PC is needed. The problem that ZMODEM addresses is the wire transfer protocol (and, specifically, the hardware's complete lack of handshaking), and the need for a special DSR (read: HDX) just to perform the transfer. DSK2PC is IMO using the wrong tool set to perform its task. It should be talking to (whatever the UNIX incarnation of Telix is) and transferring from PC using a real transfer protocol, not something stapled into a DSR that has *no* real error recovery.

Agreed re: the hardware based on your further comments. Sometimes a gut-check to rule out the potentially obvious items is needed to cover the bases.

 

I had some time to burn yesterday (took someone to a doctor appt) so I looked at the DSR a bit more carefully. There are a few spots where it isn't clear (to me) if the transfer leaves buffers open. As I may have mentioned before, the DSR does not buffer sectors or files in the traditional disk controller sense. It does however manage a set number of 'buffers' above 0x4d00. The stack is also located in this area. It is possible that one or both are being corrupted. I'll poke around some more this weekend.

 

Thanks for the explanation regarding DSK2PC. Now I understand what you were driving at.

[+InsaneMultitasker]

TurboForth only does level 3 disk access. When reading or writing blocks to/from disk it opens the file, does what it needs to do, and closes the file. Then it does the next block (open, read/write, close). This is slow but the disk access to blocks in TF is rock solid so I live with it.

 

This always caused problems with HDX and I had to give up using it. It would work for maybe 10 minutes then stop working. IIRC rather than crash it would just become increasingly erratic.

 

I get the feeling that if you're using it for just straight-up loading of programs then it's pretty good, but I was doing 100% file i/o (DF128) and it wasn't reliable long-term.

You might have been running into the same problem that I encountered with the IDE DSR. When I tried to install and operate my BBS with the IDE card, I had a lot of trouble with fixed record files. Wrong records, corrupt files, and occasional disk corruption were common themes. Loading/saving files and using level 2 direct IO worked just fine.

 

I passed along those findings to Fred with some good test cases. He wrote a number of DSR test scenarios (which are available on his site) and corrected his IDE DSR. If I recall correctly, some of those same problems were present and fixed in the HDX DSR and/or server; perhaps not all of them were addressed, adding to the challenges found within this thread.

[TheMole]

Just a user here, so I can't provide much valuable input on the technical side, but I would suggest it could be good to make any new hardware design in such a way that you can have both a side-car and PEB variant. With the nanopeb out of production, a replacement disk-system might be useful to have.

[ckoba]

I haven't used HDX at all, so I can't feed too much useful data into that part of the conversation, but building a PEB HDX2 board using the Protoboards looks like it might be a good starting point for some real hardware goodness. I can't put that into my queue right now (too many other projects grinding forward with glacial slowness), But I may be able to contribute some tips to anyone else taking a stab at it. . .

 

And I like the methodical way you've been testing this to destruction, Chris--hardware projects need that kind of rigorous testing to flush out the bugs that would otherwise only serve to annoy the unlucky end user if they actually get into a production product. . .and I will definitely be bumping some of my near-term hardware projects up against you (if you are interested in doing such) to make sure they are ready for prime time.

 

I'll do the design. It looks like my project priorities have been overtaken by events -- I was running with the assumption that the HDX hardware itself actually worked, even though it wasn't what I'd consider consumer-ready.

 

Now that *that* particular assumption has been proven false (if Willsy says it, then I believe him without question, because he's authoritative in this area), there is no known good portion of the HDX to use as a starting point. My first impression was correct -- it's poorly designed in all aspects.

 

That being the case, I'll get ahold of a few 16550s (TI still makes the DIP part as the PC16550N) and prototype something within a few weeks (it takes Mouser awhile to ship here). The board will be EPROM for the DSR and RAM for the buffers, as ${DEITY} intended. If Willsy was serious about writing a DSR for it, then I'll send a unit to him.

 

In any case, development is stalled until stable hardware with a DSR exists.

 

Thanks for your faith in my thorough corner-case testing, and I'd be happy to rip apart anything you toss my way

 

I'll finish by reiterating and amplifying my initial statement about the HDX.

 

I believe that it is poorly designed, due to its dependence on hardware that is incapable of any sort of hardware flow control. I believe that it is poorly implemented, because the DSR is in battery-backed RAM. I've heard the arguments "it works for lots of people!", "it'll last years before I need to change the battery!", and I'm sorry but they just aren't persuasive. Power-cycling a device is supposed to return it to a known good state, and a DSR in battery-backed RAM completely thwarts that.

 

It might be tolerable if the DSR itself was stable, but it has demonstrated itself to be spectacularly unstable -- not just to me, but to Willsy (and, by extension, anyone making heavy use of "normal" L3 commands).

 

The author was invited to participate, or at least comment, in the first post on the subject -- and either doesn't live here anymore (contrary to Arcadeshopper's belief), doesn't care to see his creation ripped to shreds, or doesn't care period. Willsy says that the author was aware from the very beginning of serious issues with the HDX, but only fixed some of them. I conclude that the HDX is, at this time, unsupported by the author.

 

(I'm also amused that no credit was given by the HDX author to InsaneMultitasker for the BASIC test case suite. That's rather rude, but in keeping with the mental image I've formed of the author over the past few weeks)

 

Therefore: I cannot in good conscience recommend the HDX to any TI user, because it is not reliable and can not be made reliable as long as it uses the TI serial hardware. I'm sorry if this dissuades people from purchasing HDX boards/installation from your store, Arcadeshopper, but the HDX is junk. If it were a Commodore device, it would have been a Jim Drew product.

 

I'll design a proper replacement when I have parts and time.

Edited February 20, 2016 by ckoba

[ckoba]

Just a user here, so I can't provide much valuable input on the technical side, but I would suggest it could be good to make any new hardware design in such a way that you can have both a side-car and PEB variant. With the nanopeb out of production, a replacement disk-system might be useful to have.

 

This would be trivial to accomplish. Sure.

[ckoba]

You might have been running into the same problem that I encountered with the IDE DSR. When I tried to install and operate my BBS with the IDE card, I had a lot of trouble with fixed record files. Wrong records, corrupt files, and occasional disk corruption were common themes. Loading/saving files and using level 2 direct IO worked just fine.

 

... which would be the common use case for folks like Omega, I guess. That would be why they don't understand why I dislike the HDX -- they don't see the problems because they don't use the (to be generous) 30% that doesn't work. It makes sense that Fred would share code between his IDE DSR and the HDX.

[Willsy]

The HDX is a great idea and Fred is a great guy. I haven't heard from him in a long time. He's kind of dropped off the radar a bit. I think with the HDX he just took it as far as he was able to within the normal constraints of a hobbyist project. And he never sold them; he just made everything available on his site -there's no warranties or guarantees. Don't be too hard on the guy. If you met him you'd like him. And we need talented guys like Fred around!