Detecting Various Types of Drives via SIO

[Zolaerla]

27 minutes ago, phaeron said:

I can also reproduce this on my NTSC 800XL + Happy 1050 v2, it is detected as a Tygrys 1050 instead of a Happy 1050.

 

The problem appears to be with the PERCOM tests being run at high speed. The command frame is going out fine and getting ACKed:

 

However, the data frame is NAKed by the drive:

 

Based on what I'm seeing in the emulation trace, it looks like the high-speed SIO routines are sending the data frame too quickly after the ACK -- the computer is already sending the start bit by the time the drive is able to turn around and start its high-speed receive routine. Combination of micro-timing and analog signal effects probably explains the reliability issues as in emulation you get ideal clocks with perfect signal quality, vs. the slower rise times above.

 

The timing above is about 250us from end of ACK to start of data frame, which is too short and a SIO spec violation, which specifies 1.0-1.8ms. Things get fuzzy with high-speed SIO where there isn't a clear spec, but SpartaDOS X does sends with ACK-to-data delays in the ball park of 1.2-1.5ms.

 

I couldn't determine if that was true with the docs I have, and it doesn't appear necessary in any of the devices I've tested via emulation in Altirra, though it might be a bit fuzzy and why the Tygrys was failing and the Happy 1050 rev 2 wasn't. SIO2PC devices don't care about the delay either, which doesn't help me since that's the high speed hardware that I have. Anyways, will have to update, test, and post the revised version.

 

Thank you for the in-depth test!

 

 

 

[E474]

Hi @Zolaerla,

 

   OK, I forgot to mention the hardware. It was a stock PAL 800XL with a SysCheck II card plugged in, with the stock 800XL OS selected. No hardware modifications to the 800XL, and nothing else (powered up) on the SIO chain (I had stopped the atariserver process on the Pi).

 

   I made the boot disks by booting up MyDos from a Raspberry Pi B1 using a homemade SIO2Pi connector with a logic level shifter board to handle voltages, and running atariserver to handle serving the MyDos and your bootdisk ATRs. I formatted each disk using MyDos single density format, and copied the bootdisks to floppies using MyDos option J, something like: J,2,1(1-720)/N

 

   Actual testing was done using write protected TDK 48 TPI disks.

 

   I was wondering if you could strip out the disk test routines into a DOS executable, but it sounds like this would complicate things. Can you add a title to each results screen, saying what the test was, as a lot of the results look the same, and otherwise I have to take a photo of the screen announcing the test about to be performed, then the results. It's much easier if the results screen clearly mentions what is being tested.

 

   Good luck with updating your code (I appreciate this is a lengthy edit, assemble, test, debug cycle)!

  

[Zolaerla]

I've added a ~1.5ms delay before writing, changed the clear screens to newlines and removed half the "press a keys" when displaying output (now they say "Starting SIO Status Test" and immediately display the results), added displaying output of the final Percom write (useful for drives that disable themselves like the Percom RFD and ATR-8000), and it seems everything still works just fine on my end. I've attached the updated ATR files.

 

I don't know anything about Atari DOSes. All of the games I've ever played auto booted, which is what I am planning on doing myself going forward... detecting drives is just a side project on top of making a custom game OS. I may end up adding some means of booting the custom OS through the more modern DOSes like MyDOS and SDX, but that isn't a goal at the moment. The disk tester is extensively using a script language for output that is a part of my custom OS. It would be a significant job to port the scripting language to work with the Atari OS (which I don't even know how to use other than loading my own OS and replacing it)... or to manually code all of the output to work in a XEX or the like.

 

For example, displaying the received status data looks like this:

	QSS_Newline
	QSS_Newline
	QSS_Output "Received Status Data:"
	QSS_Output "\nStatus:         $"
	QSS_Int8 DIO_ReadBuffer, "H"
	QSS_Output "\nFDC Status:     $"
	QSS_Int8 DIO_ReadBuffer + 1, "H"
	QSS_Output "\nFormat Timeout: $"
	QSS_Int8 DIO_ReadBuffer + 2, "H"
	QSS_Output "\nExtra [+3]:     $"
	QSS_Int8 DIO_ReadBuffer + 3, "H"
	QSS_Output "\n"
	QSS_Output "\nLast ACK:       $"
	QSS_Int8 QuickDriveIO::LastACK, "H"
	QSS_Output "\nLast Complete:  $"
	QSS_Int8 QuickDriveIO::LastComplete, "H"
	QSS_End

 

This becomes very compact once the macros change everything to raw data.

 

 

By the way, at least through Altirra, I found an anomaly with Happy Rev 2 when sending the absurd and all $FF Percom data, which does not correct itself without power cycling. It happens on the Happy Rev 1 as well, but it doesn't cause it to be "broken". There are two things that happen:
1) It stops pausing after each track change on load. This is really weird and unexpected, but it technically makes it load faster. Works on both versions (Warp commands for Rev 1, US Doubler for Rev 2).
2) However, on the Rev 2, every 3rd byte loaded from a sector is corrupted, as you can see in the screen shot from loading the Danson image. It does *NOT* cause this when loading at normal speed, it only happens when using high speed I/O. (you can see every 3rd byte of each sector highlighted in red at the bottom right in the debug data, as well as the obvious image glitches).

It would seem that the weird Percom settings cause the Happy to change some of its settings to compensate, and changing the Percom data back doesn't correct that.

 

FullDetect.atr NoDetect.atr

Edited May 28, 2023 by Zolaerla

[phaeron]

30 minutes ago, Zolaerla said:

I've added a ~1.5ms delay before writing, changed the clear screens to newlines and removed half the "press a keys" when displaying output (now they say "Starting SIO Status Test" and immediately display the results), added displaying output of the final Percom write (useful for drives that disable themselves like the Percom RFD and ATR-8000), and it seems everything still works just fine on my end. I've attached the updated ATR files.

Works now with my Happy 1050.

 

30 minutes ago, Zolaerla said:

By the way, at least through Altirra, I found an anomaly with Happy Rev 2 when sending the absurd and all $FF Percom data, which does not correct itself without power cycling. It happens on the Happy Rev 1 as well, but it doesn't cause it to be "broken". There are two things that happen:

1) It stops pausing after each track change on load. This is really weird and unexpected, but it technically makes it load faster. Works on both versions (Warp commands for Rev 1, US Doubler for Rev 2).

Check if you have accurate sector timing enabled in Altirra. You should actually be seeing slower loads after running "test various commands" as that test is disabling track buffering. Flip the slow/fast switch to slow and back again to enable track write buffering and it will load much faster.

 

30 minutes ago, Zolaerla said:

2) However, on the Rev 2, every 3rd byte loaded from a sector is corrupted, as you can see in the screen shot from loading the Danson image. It does *NOT* cause this when loading at normal speed, it only happens when using high speed I/O. (you can see every 3rd byte of each sector highlighted in red at the bottom right in the debug data, as well as the obvious image glitches).

It would seem that the weird Percom settings cause the Happy to change some of its settings to compensate, and changing the Percom data back doesn't correct that.

 

Correct:

This is due to a severe bug in the Happy 1050 v2 firmware's USD emulation. We nearly lost Silent Butler 80 because of this bug:

https://forums.atariage.com/topic/291352-after-32-years-silent-butler-80-is-found-by-curt-call-for-help/

 

Enabling fast writes works around this bug.

 

 

[_The Doctor__]

That title never made it into the a8sp initiatives online database. It's a worry.

55 minutes ago, phaeron said:

This is due to a severe bug in the Happy 1050 v2 firmware's USD emulation. We nearly lost Silent Butler 80 because of this bug:

https://forums.atariage.com/topic/291352-after-32-years-silent-butler-80-is-found-by-curt-call-for-help/

 

Enabling fast writes works around this bug.

 

[Zolaerla]

1 hour ago, phaeron said:

Works now with my Happy 1050.

 

Check if you have accurate sector timing enabled in Altirra. You should actually be seeing slower loads after running "test various commands" as that test is disabling track buffering. Flip the slow/fast switch to slow and back again to enable track write buffering and it will load much faster.

 

Correct:

This is due to a severe bug in the Happy 1050 v2 firmware's USD emulation. We nearly lost Silent Butler 80 because of this bug:

https://forums.atariage.com/topic/291352-after-32-years-silent-butler-80-is-found-by-curt-call-for-help/

 

Enabling fast writes works around this bug.

 

 

 

So what you're saying is that using really weird Percom blocks disables the write buffering? Because that's what I pass to test the Happy command. I'll have to test if sending it again after doing the Percom test fixes it up.

[phaeron]

16 minutes ago, Zolaerla said:

So what you're saying is that using really weird Percom blocks disables the write buffering? Because that's what I pass to test the Happy command. I'll have to test if sending it again after doing the Percom test fixes it up.

Yes, this is what I'm seeing in the debugger. The firmware flag byte is at $9702 on the drive and I can see it flipping from $00 (track buffer read/write) to $60 (track buffering disabled) after this command:

 

SIOCMD: Device 31 | Command 4F | 20 00 (Disk: Write PERCOM block) (high-speed command frame)

 

The firmware does this on any density change from a Write PERCOM block command. Looks like it is expecting a format command to follow, which reinitializes track buffering.

[Zolaerla]

7 minutes ago, phaeron said:

Yes, this is what I'm seeing in the debugger. The firmware flag byte is at $9702 on the drive and I can see it flipping from $00 (track buffer read/write) to $60 (track buffering disabled) after this command:

 

SIOCMD: Device 31 | Command 4F | 20 00 (Disk: Write PERCOM block) (high-speed command frame)

 

The firmware does this on any density change from a Write PERCOM block command. Looks like it is expecting a format command to follow, which reinitializes track buffering.

 

If I change my Happy test to send Aux1 of $60 and Aux2 of 0 and rerun it after writing the weird Percom blocks out, this fixes the problem. I tried just $20 and 0, but that didn't fix it on its own. Attached is the corrected FullDetect that has a second Happy call after doing the Percom tests.

 

Thanks again!

FullDetect.atr

[E474]

Hi @Zolaerla,

 

    I just tried the latest FullDetect, but it failed to load after the loading menu. See: images.

 

   Also, I managed to mess up which ATR I was using, e.g. FullDetect.atr, not FullDetect(1).atr (I deleted the old versions and re-downloaded, recopied, etc., once I had figure out my mistake), but can you up the version number on the title screen going forward?

 

    Hope this helps!

[Zolaerla]

55 minutes ago, E474 said:

Hi @Zolaerla,

 

    I just tried the latest FullDetect, but it failed to load after the loading menu. See: images.

 

   Also, I managed to mess up which ATR I was using, e.g. FullDetect.atr, not FullDetect(1).atr (I deleted the old versions and re-downloaded, recopied, etc., once I had figure out my mistake), but can you up the version number on the title screen going forward?

 

    Hope this helps!

 

 

 

This is still alpha-level software, which I don't generally increment versions on (usually they hang out on a development server in my other life until ready to launch a version, but there is no such thing here), but I can start maintaining alternate versions going forward since that would help with other people using it.

 

The error code of $92 is still a framing problem... I can see it gave a nonsense $A6 for the "Complete" byte which generally means it's out of sync with the drive, as tends to happen with framing issues. I wonder if sending the Happy commands to enable the buffering causes some other sort of problem with this drive. I'm not really certain what else to try here since phaeron's Happy 1050 rev 2 works with the existing code... I'm not exactly certain WHAT should change, though. I can try to detect "mystery Happy 1050 rev 2 clone" drive you have and forge on with some "Here, try this" list of things. I don't know if you'd be up for that sort of back and forth to try to get this drive to work at high speed.

[E474]

Hi @Zolaerla,

 

   I don't mind carrying on with the Happy clone, is there any chance it's a PAL/NTSC issue? Maybe someone with a PAL setup and genuine Happy can test to see if it is?

 

   I can test next with a Mini Super Speedy and a BitWriter/Super Archiver (both from @tf_hh) and  a Lazer and US Doubler combination drive, but ithat's also a clone  - see: 

 

[Zolaerla]

4 hours ago, E474 said:

Hi @Zolaerla,

 

   I don't mind carrying on with the Happy clone, is there any chance it's a PAL/NTSC issue? Maybe someone with a PAL setup and genuine Happy can test to see if it is?

 

   I can test next with a Mini Super Speedy and a BitWriter/Super Archiver (both from @tf_hh) and  a Lazer and US Doubler combination drive, but ithat's also a clone  - see: 

 

 

If it's a PAL issue, then it will never run full speed properly on a PAL system (random framing errors) so far as I'm aware. I'm at the mercy of the timing of POKEY here and what the US Doubler is telling me to use for the divisor ($0A or 10). It might run at Happy 1050 Warp Speed which is a bit slower (divisor of $10 or 16) and would need different commands, so I would have to detect that drive specifically and use that instead. Since the status is returning an FDC status of $9B (which would be $DB if not write protected), I can test for those odd bits ("Deleted Sector" and "Lost Data" bits) to determine this isn't a standard Happy which isn't a great test, but it's something. I'll also increment the minor version numbers going forward, since the new detector has the write delay added to it and the old one didn't.

 

Would you be ok with continuing this via PM and when/if we get this figured out, we can post the results here?

[E474]

Hi @Zolaerla,

 

   Yes, no problem (though you might be missing out on contributions from way more knowledgeable people on AA).

[Zolaerla]

After a week of disassembling the 810 Archiver ROMs since I had basically NO documentation on it, then another week making a load of rewrites to the underlying system, adding more timing precision/control, etc., I have finally got the detection nearly as good as I can possibly get it with Altirra's emulation since just above everything can be detected now.

 

At this point, with caveats, the full routine can detect/distinguish:
Atari 810 (If a disk is in: Rev B/C vs Rev E -- it says Rev E without since FDC Status is $00 in all versions in that case)
810 Archiver (it tests for secret code $1001 vs $9999 to enable Archiver mode then disable it)
Happy 810 (pre-Rev 7 vs Rev 7)
810 Turbo
Atari 1050 (All 5 known revs are distinguished, unless there isn't a disk, then it's unknown)
1050 Duplicator
Happy 1050 (rev 1 vs rev 2)
I.S. Plate
Tyrgrys 1050 (also reinitializes it after testing or it becomes a brick from a basic Happy test)
Speedy 1050 (not sure how to test vs Super Speedy)
Super Archiver (I haven't found a way to tell it from a Bitwriter yet)
TOMS 1050
1050 Turbo (I haven't found a way to tell it from the Top 1050 yet)
General US Doubler
Atari XF551 (standard and Hyper+ XF)
Amdek AMDC
Astra 1001
Percom RFD (<v2.1 vs v2.1)
Percom AT88 (can't tell SPD from the AT88 that I have found)
ATR8000
Indus GT
TOMS Turbo
General SIO2PC/Emulation

The detection is big and complicated now, not highly surprising. It explicitly tests Archiver (via Set Trace Buffer, command $5A), Duplicator (via Set RPM, command $6A), AMDC (via Get Version 2, Command $6D), and 1050 (Diag Commands $23, $24) to double check that they match when the limiter makes sense to test for them. I tried to sort stacked tests so the more expensive ones (1050's test is slow, at least the 810s NAK it) are later, but there's only so much that can be done for that.

I will have to completely restart with the flow chart for this. I have it in text and obviously in code... meh.

 

I also need to retest/revamp the high speed SIO on all of the devices that support it. Trying to get it working on TOMS Turbo (SIO CLOCK) and Amdek AMDC (a weird implementation) will be... interesting. I'm probably also going to try to dynamically load in Super Synchromesh for Indus GT drives.

 

While exploring disks, I've found more and more ways to brick certain drives until a reset. They just stop functioning properly or are soft-locked or the like... and I'm not even trying to. I've accidentally found crashes for I.S. Plate, Tygrys 1050, Percom RFD (and Astra 1001), Percom AT88 (and SPD), and ATR 8000.

 

Anyways, a disk that does the full detection so far for whatever drives are connected (up to 8 drives) is attached for anybody that's curious.

QuickOS-0.1.1.0-FullDetect.atr

[_The Doctor__]

You've made some great progress, very nice reading about it.

possibly some command overlap or activation for one drive type and modification class gets another to lock up, so it might be some of the slower checks will have to come first to avoid annoying some of the faster ones and vice versa. Looks like you will be having a Happy time flow charting and diagramming them out to avoid the locked up/locked out drive situations at least until the last of the tests. It all starts out so nicely and blossoms into thicket doesn't it.

[drac030]

10 hours ago, Zolaerla said:

At this point, with caveats, the full routine can detect/distinguish:

What is the sense of this? Why are you detecting drives, which are a legion, instead of protocols, which are about 5? And, what about parallel drives?

Edited June 19, 2023 by drac030

[Zolaerla]

1 hour ago, drac030 said:

What is the sense of this? Why are you detecting drives, which are a legion, instead of protocols, which are about 5? And, what about parallel drives?

 

As I said in the beginning of the thread, there isn't REALLY a reason, it's just a weird project I decided to do. The basic detection done in the High Speed SIO patch by Hiassoft gave me the idea, to be honest.

 

I don't really know much of anything about parallel devices on the Atari. Since I am aiming to make games and not deal with HDDs or printers or the like, it hasn't been a topic of interest for me. It hasn't helped that whenever I try to find out the actual hardware-level methods to communicate with a PBI device, I can never get enough information to even try reading a single sector off of one, let alone determine much else. It's very similar to trying to ever find the hardware-level methods of communicating with hard drives back when I did MS-DOS assembly in the 90s... everybody just wants you to use their drivers instead, and I like working with hardware directly because I'm a masochist. =)~

 

Honestly, I expect most data transfers will be handled through an SIO2PC/SIO2USB/SIO2Whatever instead of even any of the specific drives I've been detecting, but I already knew this wasn't a practical exercise! I'd probably even completely forgo using SIO whatsoever if cartridges supported saving in any sort of consistent manner since they're pretty much instantaneous (close enough anyways).

Edited June 19, 2023 by Zolaerla
always... typos

[flashjazzcat]

On 6/19/2023 at 8:57 PM, Zolaerla said:

don't really know much of anything about parallel devices on the Atari.

Fortunately, hardware abstraction means you don't necessarily have to. The DEVINFO and DISKINFO SIO commands described in the document below (which is based heavily on Konrad's IDE Plus documentation) are not exclusive to APT (HDD) devices; they could conceivably apply to SIO-connected mass storage devices too.

 

https://atari8.co.uk/wp-content/uploads/APT-API-Programmers-Reference-Manual.pdf

 

IDE Plus, Incognito, 1088XEL/XLD, U1MB/SIDE2/SIDE3 all support the API.

[Zolaerla]

1 hour ago, flashjazzcat said:

Fortunately, hardware abstraction means you don't necessarily have to. The DEVINFO and DISKINFO SIO commands described in the document below (which is based heavily on Konrad's IDE Plus documentation) are not exclusive to APT (HDD) devices; they could conceivably apply to SIO-connected mass storage devices too.

 

https://atari8.co.uk/wp-content/uploads/APT-API-Programmers-Reference-Manual.pdf

 

IDE Plus, Incognito, 1088XEL/XLD, U1MB/SIDE2/SIDE3 all support the API.

 

Thank you! I'll add in APT detection as well since it's actually pretty easy, and you even have information on how to tell various APT devices apart in this doc, which is great, though I don't see how to detect SIDE3/SIDE3.1 (which is what I actually have) only SIDE1/SIDE2.

Edited June 21, 2023 by Zolaerla

[flashjazzcat]

3 hours ago, Zolaerla said:

I don't see how to detect SIDE3/SIDE3.1 (which is what I actually have) only SIDE1/SIDE2.

Good point. I need to update the docs.

[drac030]

On 6/19/2023 at 9:57 PM, Zolaerla said:

Since I am aiming to make games and not deal with HDDs or printers or the like, it hasn't been a topic of interest for me

Hum. A printer is of little use for a game, probably, but for a HDD I would say the opposite: it is a data storage and it can load games very efficiently (even by ATR emulation). Considering that I do not think that anyone really uses floppy disks nowadays, yes, parallel floppy drives are now a rarity. But HDDs not so.

[_The Doctor__]

If nobody uses floppies, why do I have a stack of drives and 1,000's of floppies... what's worse, my kid uses a cassette for language tapes, and uses a 1050 for games, jumpan series, canyon climber etc.

Mind you I use sdrive max for this or that, AVGcarts, ultimate carts, MIO, BlackBox, IDE+, MyIDE, etc when testing for whatever development help that goes on.

BBS stuff is always on the hard drive/mass storage and is Parallel. Most games and what not are still grabbed from some SIO feeding device or another, and many time a floppy at that. Oddly I have a FujiNet again, but it's flaky, don't know if it's yet another bad build or the firmware as it's always in flux... I always go back to floppies to ensure a program isn't broken, and then work up through the pile until something doesn't work so well and go from there.

Edited June 24, 2023 by _The Doctor__

[Zolaerla]

I don't disagree that HDDs are definitely important, but in the world of Atari gaming, nobody passes around actual HDD images... we pass around .car (Cartridge) or .atr (Disk -- even the 16MB ones act just like a disk) or .xex (executables which work almost anywhere) files. I'm glad the HDDs and SD cards (like SIDE3) and FujiNets and SIO2PCs hide the actual humongous storage devices (WiFi or internet connection as the case may be) from the Atari (yes, I'm aware multiple options are available to select which .atr/.car/etc. file is in use to allow selecting things on the Atari), and allows a unified communication format with SIO.

[drac030]

7 minutes ago, Zolaerla said:

in the world of Atari gaming, nobody passes around actual HDD images.

As long as there are no games which take up 400 MB, indeed, nobody passes HDD images. But the point is somewhere else: everybody passes around ATR images, and these can be run from a HDD as long as they use the standard I/O. Tying a program to SIO, worse, to a known set of floppy drives, is impairing its possible success. And anyways, if it is worth, it will be cracked to run from any storage, and if it is not - it will be forgotten. Just saying.

[_The Doctor__]

running from any storage is the goal, sadly very little does that, this is why so many devices in order to be successful must emulate and provide paths for ATR, CAR, BIN, CAS, etc.