a8rawconv, a new raw disk conversion utility

[telengard]

29 minutes ago, phaeron said:

Hey, sorry for the delay in answering, glad that you got it working in the meantime. There isn't supposed to be a difference between using -if do and .dsk file, and renaming that .dsk file to .do. In both cases, they should interpret the file with Apple II DOS 3.3 ordering.

 

If you still have a case where this is an issue, I'd like to see diagnostics on what's going on, such as whether the program reports it is reading a different format than "Apple II disk image (DOS 3.3 ordering)" or if a8rawconv can't read back its own result in both cases. I checked the current version with output to an .scp file and couldn't see a difference in the output, aside from the timestamp at the end of the file.

 

Sorry for the confusion, and thank you for responding!

 

The renaming from .dsk to .do was necessary for the HxC application to load it and be able to convert it to an .scp.  This wasn't something I was doing with a8rawconv.  I was just trying stuff out, and it happened to work.

 

I had initially tried writing a .dsk using just a8rawconv with the -if do flag and was unsuccessful. 

I then tried converting the .dsk to a flux image via the HxC app (but that only works if you rename it to .do).  I was able to successfully write the flux image using a8rawconv.

 

Then I was just wondering if there was something I was doing wrong in the more direct method of the .dsk straight to floppy via a8rawconv.

 

If I can gather any logs/diags etc I can do that, just not sure how.

Edited March 7, 2023 by telengard

[_The Doctor__]

not that at makes a great deal of difference but what revisions of a8rawconv have you used or tried?

[telengard]

I am using version 0.94, which seems to be the latest?

[telengard]

If it is of any help, this is the image I was using.

 

I've done it a few more times tonight and it is pretty repeatable (using the same physical floppy).  The converted to scp writes correctly, going straight from .do does not.

 

 

mecc_computer_inspector.do

[TZJB]

I have just hooked up my Mitsubishi MF503B 40/80 track from my Atari XF551 to my Greaseweazle V4 and it seems to be working using the Drive 0 parameter with a straight cable.

 

Can you indicate the latest version of a8rawconv that I should use please @phaeron? So far I have a8rawconv-0.94. Is there any advance on that one? Many thanks.

[_The Doctor__]

I think two version are needed of a8rawconv- but I can't think of the exact reason, but the older version was still needed for certain disks?

[BillC]

10 hours ago, _The Doctor__ said:

I think two version are needed of a8rawconv- but I can't think of the exact reason, but the older version was still needed for certain disks?

The following is for flux dumps made by no-flip modified drives.

 

v0.92 is preferred for decoding sideB of Kryoflux RAW dumps, while v0.94 is capable of decoding sideB of SCP dumps after some modification to the Track DATA Header Table.

Edited April 29, 2023 by BillC

[TZJB]

10 hours ago, BillC said:

The following is for flux dumps made by no-flip modified drives.

 

v0.92 is preferred for decoding sideB of Kryoflux RAW dumps, while v0.94 is capable of decoding sideB of SCP dumps after some modification to the Track DATA Header Table.

 

Thanks for the reply. I will stick to V0.94 then as I have SCP dumps. Not worked out what I need to do for side Bs yet apart from copy them to another disk side A.

[BillC]

7 hours ago, TZJB said:

Thanks for the reply. I will stick to V0.94 then as I have SCP dumps. Not worked out what I need to do for side Bs yet apart from copy them to another disk side A.

I make SCP dumps of sideB using a Greaseweazle F7 Lightning Plus and a modified JU475-5 drive, then modify the TDHT so that v0.94 can access the flux DATA streams. The issue is that v0.94 only reads from the sideA locations in the TDHT, so the DATA in the sideB locations must be moved/copied from the sideB location to the sideA location. Once modified any flux streams for sideA are inaccessible.

 

I am attaching a python script which copies the sideB offsets to the side A offset locations, it's a modification of a script by @Savetz on the a8sp forums.

To decode SideB from the modified SCP to ATR/ATX use a8rawconv v0.94 with the -r parameter added to the end of the command line.

 

SCPmodSideB.py

[TZJB]

19 hours ago, BillC said:

I make SCP dumps of sideB using a Greaseweazle F7 Lightning Plus and a modified JU475-5 drive, then modify the TDHT so that v0.94 can access the flux DATA streams. The issue is that v0.94 only reads from the sideA locations in the TDHT, so the DATA in the sideB locations must be moved/copied from the sideB location to the sideA location. Once modified any flux streams for sideA are inaccessible.

 

I am attaching a python script which copies the sideB offsets to the side A offset locations, it's a modification of a script by @Savetz on the a8sp forums.

To decode SideB from the modified SCP to ATR/ATX use a8rawconv v0.94 with the -r parameter added to the end of the command line.

 

SCPmodSideB.py 1.54 kB · 2 downloads

Thanks, but I am not sure what to do with that script. Unfortunately for me my knowledge of Python is nil.

 

Incidentally, what settings are you using for Greaseweazle? I have found that I need to copy tracks 0-40 to get a valid track 39.

 

 

[BillC]

53 minutes ago, TZJB said:

Thanks, but I am not sure what to do with that script. Unfortunately for me my knowledge of Python is nil.

 

Incidentally, what settings are you using for Greaseweazle? I have found that I need to copy tracks 0-40 to get a valid track 39.

You just need to install the Python interpeter for your OS, then run the script in a terminal window and it will create a modified scp file, you can examine the script by opening it with notepad, I added comments to make the procedure easier to follow, I had no prior experience with Python before working on this script, but I did start with a script by Savetz that moved the SideB flux DATA offset from the SideB to the SideA locations in the TDHT, my script writes the SideB offsets to both locations. I wrote it for single-sided SideB SCP flux dumps, but it does work on double-sided dumps as well.

The usage is:

"python scriptname scpname"

 

I have always dumped tracks 0-40 since a8sp wants submitted flux dumps to contain an extra track or 2 beyond the first 40 tracks, so I never encountered an invalid track 39.

[+DjayBee]

On 4/29/2023 at 5:01 AM, BillC said:

v0.92 is preferred for decoding sideB of Kryoflux RAW dumps

Yep, Kryoflux software by default names trackdumps for side A *.0.raw and for side B *.1.raw. This is also the naming scheme expected by v0.92.

v0.94 changed this to *.0.raw for both sides. Therefore you must rename all trackdumps for side B if you want/need to convert them using a8rawconv.

 

So unless you need some specific feature of v0.94^*), it is best to stay with v0.92 for Kryoflux dumps as BillC already said.

 

^*) Examples:

• Snokie contains a sector which has both a bad CRC and is marked "missing". Converting it with v0.92 results in a really missing sector which fails the protection check.
• Certain Enhanced Density disks need it as well.
  I could not find the exact cases in my documentation but IIRC one case are disks containing both FM- and MFM-tracks.

 

Edited May 1, 2023 by DjayBee

[TZJB]

On 4/30/2023 at 9:05 PM, BillC said:

You just need to install the Python interpeter for your OS, then run the script in a terminal window and it will create a modified scp file, you can examine the script by opening it with notepad, I added comments to make the procedure easier to follow, I had no prior experience with Python before working on this script, but I did start with a script by Savetz that moved the SideB flux DATA offset from the SideB to the SideA locations in the TDHT, my script writes the SideB offsets to both locations. I wrote it for single-sided SideB SCP flux dumps, but it does work on double-sided dumps as well.

The usage is:

"python scriptname scpname"

 

I have always dumped tracks 0-40 since a8sp wants submitted flux dumps to contain an extra track or 2 beyond the first 40 tracks, so I never encountered an invalid track 39.

Thanks @BillC. However I am going to avoid your Python script unless I really have to use it which would be when using head 1 of the disk drive.

 

Maybe it's because I am using a 40/80 track drive in 40 track mode, but just dumping tracks 0-39 does not produce a valid track 39 so 40 is the answer. Plus I have found a Greaseweazle setting that simulates an index pulse for the flipped B side of a disk so no data offset is required. I just tried it with an unprotected disk as it turns out, now I will need to make a protected disk B side to verify it.

 

4 hours ago, DjayBee said:

Yep, Kryoflux software by default names trackdumps for side A *.0.raw and for side B *.1.raw. This is also the naming scheme expected by v0.92.

v0.94 changed this to *.0.raw for both sides. Therefore you must rename all trackdumps for side B if you want/need to convert them using a8rawconv.

 

So unless you need some specific feature of v0.94^*), it is best to stay with v0.92 for Kryoflux dumps as BillC already said.

 

^*) Examples:

• Snokie contains a sector which has both a bad CRC and is marked "missing". Converting it with v0.92 results in a really missing sector which fails the protection check.
• Certain Enhanced Density disks need it as well.
  I could not find the exact cases in my documentation but IIRC one case are disks containing both FM- and MFM-tracks.

 

 

Thanks for clarification, but I am imaging disks with Greaseweazle V4 using v1.9 software tools directly to SCP files. I don't have any DSDD 5.25" disks to image so I will just use v0.94 for now. Incidentally I have just noticed v1.10 Greaseweazle tools are available.

[ijor]

1 hour ago, TZJB said:

Plus I have found a Greaseweazle setting that simulates an index pulse for the flipped B side of a disk so no data offset is required. I just tried it with an unprotected disk as it turns out, now I will need to make a protected disk B side to verify it.

 

Note that this doesn't work with all drives. Also some protections would work, others will not. But even when it works, it is not recommended for preservation purposes because the original index alignment is not recorded in the image. Please try to avoid this method expect for user formatted disks.

[TZJB]

9 hours ago, ijor said:

 

Note that this doesn't work with all drives. Also some protections would work, others will not. But even when it works, it is not recommended for preservation purposes because the original index alignment is not recorded in the image. Please try to avoid this method expect for user formatted disks.

 

Thank you for your advice. I now understand your requirement, but I am struggling to understand it. Atari 810/1050 drives don't use the index alignment. There is no index sensor. I am a bit ignorant here and you have much more experience at this, so I am guessing that you think an original disk B side may have been produced with a single or double sided disk drive with two index sensors?

 

So far I haven't found any original disks with data on the B side worthy of preservation anyway. All the popular ones have already been done. I am just going through what I have to see if there is anything missing from current online collections. There are very few original disks at the moment.

[ijor]

6 hours ago, TZJB said:

 

Thank you for your advice. I now understand your requirement, but I am struggling to understand it. Atari 810/1050 drives don't use the index alignment. There is no index sensor. I am a bit ignorant here and you have much more experience at this, so I am guessing that you think an original disk B side may have been produced with a single or double sided disk drive with two index sensors?

 

Original disks were duplicated with all sorts of ways. Some with modified or special drives, some with disks with two index holes, and some were not recorded aligned to the index hole at all. Doesn't matter too much for this purpose. The original index alignment is usually a valuable piece of information for preservation purposes, even when not required by the protection or even when the disk is not copy protected. The relative track alignment is certainly important because it will affect the timing, again, even when not strictly required by the copy protection.

 

[TZJB]

5 minutes ago, ijor said:

 

Original disks were duplicated with all sorts of ways. Some with modified or special drives, some with disks with two index holes, and some were not recorded aligned to the index hole at all. Doesn't matter too much for this purpose. The original index alignment is usually a valuable piece of information for preservation purposes, even when not required by the protection or even when the disk is not copy protected. The relative track alignment is certainly important because it will affect the timing, again, even when not strictly required by the copy protection.

 

 

Thanks for the explanation. It makes some sense now. Do you then compare images from different sources? What if they are different?

[ijor]

On 5/2/2023 at 1:32 PM, TZJB said:

Do you then compare images from different sources? What if they are different?

Sure we do. If they are different, then that it is interesting. It could be for several reasons and it requires analysis and sometimes multiple images to be able to find out the actual reason.

[+DjayBee]

While converting @SoulBuster's SCP image of Forms Generator (SCP is contained in the linked post's big ZIP file), a8rawconv 0.92 and 0.94 detect sector 12 on track 4 as a valid and good one while it does not even have any data block.
We have two more flux-dumps from this company which show the same behaviour.

 

Reading from the original disk followed by a SIO STATUS command DVSTAT returns $04 and DVSTAT+1 $BF.
@ijor I completely failed in trying to provoke $BF with an ATX file and was only able to receive $CF, $DF or $EF.

 

Found track 4, sector 12 at position 0.05
Decoded FM track 4, sector 12: 128 bytes, pos 0.049-0.084, DAM FB, CRC 2FB7 (OK)

 

[phaeron]

10 hours ago, DjayBee said:

While converting @SoulBuster's SCP image of Forms Generator (SCP is contained in the linked post's big ZIP file), a8rawconv 0.92 and 0.94 detect sector 12 on track 4 as a valid and good one while it does not even have any data block.
We have two more flux-dumps from this company which show the same behaviour.

I need to limit the search length for the DAM to match the 1771/2793 behavior, it is probably scanning all the way into the next sector (it doesn't stop on an IDAM due to interleaved sectors).

 

10 hours ago, DjayBee said:

Reading from the original disk followed by a SIO STATUS command DVSTAT returns $04 and DVSTAT+1 $BF.
@ijor I completely failed in trying to provoke $BF with an ATX file and was only able to receive $CF, $DF or $EF.

Was that on an 810? You shouldn't be able to get $BF on a 1050 because the 2793/2797 only supports one of the data address mark bits. Unfortunately, ATX's encoding is modeled after the 1050 and reuses bit 6 for another purpose, so it cannot encode both bits either.

[+SoulBuster]

I used a standard XF551 to record the sectors @DjayBee asked for.

[phaeron]

Interesting, so the WD1772 also returns both record type bits and the XF551 can also read the full record type.

[ijor]

3 hours ago, phaeron said:

Interesting, so the WD1772 also returns both record type bits ...

It doesn't, and anyway, unless I miss something, we are talking about a sector header alone without any data mark at all. Not even an 810 should set status bit 6 in this case, for that matter, no WD FDC should return that status here.

Edited August 4, 2023 by ijor

[phaeron]

2 hours ago, ijor said:

It doesn't, and anyway, unless I miss something, we are talking about a sector header alone without any data mark at all. Not even an 810 should set status bit 6 in this case, for that matter, no WD FDC should return that status here.

Ah, you're right, read the chart on a PDF too quickly at lunchtime, it's transposed compared to normal FDC datasheets. Well, in that case I'm confused as to how that status would come from the read sector command too, time to test on the XF551.

[ijor]

1 hour ago, phaeron said:

Well, in that case I'm confused as to how that status would come from the read sector command too, time to test on the XF551.

Perhaps it's a firmware artifact. And we don't know which firmware he is using. He might be using a third party custom firmware as far as we know. But might me something else. I don't know. There is not enough information to be sure. Also the SCP dump was performed in Splice mode (instead of the correct Index mode). That makes difficult to perform a precise low level analysis.