MAME Geneve - creating MFM HD images

[JJB]

I am a Myarc noob so this has probably been done before but I couldn't find any info on creating your own HD (MFM) images. I had a crack at it and made a quick guide along the way (see attached). 

 

It uses the CHDMAN MAME tool (attached) and you will need an existing HD image to start with. The latter can be found on whtech or see the latest MDOS release.

 

Again, there may be much better info out there; let me know and I'll update this post.

Myarc Geneve 9640 - MAME MFM HD images.pdf chdman.zip

[+mizapf]

chdman and imgtool's deficiencies were the reason why I created TIImageTool. You should try it.

 

https://www.mizapf.de/en/ti99/timt

[JJB]

Awesome, will try ASAP. 

[hloberg]

here is a complete 3 .hd setup for the geneve with all software MDOS & everything.

https://ti99resources.wordpress.com/emulation/

goto MAME packages then Geneve.

[JJB]

49 minutes ago, hloberg said:

here is a complete 3 .hd setup for the geneve with all software MDOS & everything.

https://ti99resources.wordpress.com/emulation/

goto MAME packages then Geneve.

Thanks! I started my emulation journey there and it's the reason I wanted to create my own images: sorting stuff differently, creating a clean boot HD with the latest MDOS version etc.

[JJB]

5 hours ago, mizapf said:

chdman and imgtool's deficiencies were the reason why I created TIImageTool. You should try it.

 

https://www.mizapf.de/en/ti99/timt

Wow that's a great piece of software. Super!

[hloberg]

42 minutes ago, JJB said:

Thanks! I started my emulation journey there and it's the reason I wanted to create my own images: sorting stuff differently, creating a clean boot HD with the latest MDOS version etc.

cool. note thou, I had some file weirdness happen with the Geneve if I used .HD sizes more than twice the size I'm using with my .hds.

also after you create the new .HDs using TIImageTool it's easy copying files from my .HDs to yours using TIMT or from any .DSK or TIFILES.

oh and I just uploaded a new set of .HDs with the latest MDOS files.

 

[+9640News]

One thing to be careful if you mix Geneve and 4A images. MDOS creates subdirectories according to the specs in the HFDC manual. The 4A does not with MDM5 or when using MDM5 in Rompage mode on the Geneve. 
 

With the Geneve, either use the MD or MKDIR command or a MDOS program such as Clint’s DM or Fred’s GDM2K. Otherwise, any directory created with MDM5 can not be deleted with the RD RMDIr command on the Geneve. MDM5 has a bug where it leaves out a recursive sector vector in the directory FDR.

 

beery

[+mizapf]

BTW, directory creation in TIImageTool follows the specs of GeneveOS / Myarc HFDC, so they should be created and used without any problems on the Geneve.

[Brufnus]

Apropos TIImageTool... I think that one could be useful once I'm ready to throw my software collection. Can these be read somehow by a Tipi as well?

[+mizapf]

Do you want to read a MFM HD image with the TIPI? I don't think that TIPI can read that; at least you have to save the HD image in RAW format. This is basically the same as the DSK image, just for a hard disk.

 

Otherwise, do a mass export of your HD image to your file system, then upload everything to TIPI.

[+arcadeshopper]

2 hours ago, mizapf said:

Do you want to read a MFM HD image with the TIPI? I don't think that TIPI can read that; at least you have to save the HD image in RAW format. This is basically the same as the DSK image, just for a hard disk.

 

Otherwise, do a mass export of your HD image to your file system, then upload everything to TIPI.

tipi doesn't do disk images, it uses the native linux os drive and saves everything in TIFILES format.. any floppy files copied to it will be automaticlly unpacked into a folder with the disk name and the dsk file deleted hard disk images are not supported, export in tifiles format and copy/upload to the TIPI drive directly as @mizapf suggests

[Brufnus]

Hehe, okay... I know next to nothing about the TIPI (for good reasons, since I don't have one). 

So, uploading to a TIPI here is impossible. 😕

 

All right then... any good suggestions as to how I can make the collection available? I don't think the emulator images will be of much use unless there's an emulator in the receiving end as well. c".)

 

An option could of course be to use Archiver III with each disk, transfer them to a PC and then zip the lot of them?

[RickyDean]

21 minutes ago, Brufnus said:

Hehe, okay... I know next to nothing about the TIPI (for good reasons, since I don't have one). 

So, uploading to a TIPI here is impossible. 😕

 

All right then... any good suggestions as to how I can make the collection available? I don't think the emulator images will be of much use unless there's an emulator in the receiving end as well. c".)

 

An option could of course be to use Archiver III with each disk, transfer them to a PC and then zip the lot of them?

Are you're using a HD emulator such as a Drem or a MFM Hard Disk Reader/Emulator from PDP8online? if so you just take the image from the device and post it. Then we use TIImageTool or our own HD emulator device and use the files as we see fit.

If you're using a real MFM drive, do you have an IDE card, SCSI card, or HDX? If so you can ise the first two on a pc and create an image using TIImageTool or using the HDX(modified rs232) you can use a folder on the pc as hard drive storage. These would seem to be the easiest ways. You can also use Fred Kaal's software on a normal RS232 and do the same as the HDX.

 

If doing it a software way then you maybe can use PCtransfer or a terminal emulator, or Archiver?

Edited September 1 by RickyDean
spelling

[Brufnus]

55 minutes ago, RickyDean said:

Are you're using a HD emulator such as a Drem or a MFM Hard Disk Reader/Emulator from PDP8online? if so you just take the image from the device and post it. Then we use TIImageTool or our own HD emulator device and use the files as we see fit.

If you're using a real MFM drive, do you have an IDE card, SCSI card, or HDX? If so you can ise the first two on a pc and create an image using TIImageTool or using the HDX(modified rs232) you can use a folder on the pc as hard drive storage. These would seem to be the easiest ways. You can also use Fred Kaal's software on a normal RS232 and do the same as the HDX.

 

If doing it a software way then you maybe can use PCtransfer or a terminal emulator, or Archiver?

That's it then... I'm using the Emulator from PDP8online, so if that image is usable with TIImageTool etc, it will definitely be the easiest option. Those I can transfer directly to a PC via network and then upload them.

Edited September 1 by Brufnus

[RickyDean]

21 minutes ago, Brufnus said:

That's it then... I'm using the Emulator from PDP8online, so if that image is usable with TIImageTool etc, it will definitely be the easiest option. Those I can transfer directly to a PC via network and then upload them.

I would ask @mizapf to see if those images are directly readable by the Tool, but I think even if it weren't that by the the use of the chdman tool in Mame it could be converted to a raw image that could.

Edited September 1 by RickyDean
spelling

[+mizapf]

Are the images from PDP8online sector dumps of the hard disk, or are they true MFM recordings? MAME does not handle MFM images of hard disks, and neither does TIImageTool. The reason is that this would require too much performance to keep up with real time (in contrast to floppy disk images, which are all treated as (M)FM recordings).

 

This was at least the case 10 years ago. Computers have increased their performance, and probably, MAME would be able to properly emulate MFM images of hard disks, but this would also mean that you had to run it on a recent CPU.

[Brufnus]

16 minutes ago, mizapf said:

Are the images from PDP8online sector dumps of the hard disk, or are they true MFM recordings? MAME does not handle MFM images of hard disks, and neither does TIImageTool. The reason is that this would require too much performance to keep up with real time (in contrast to floppy disk images, which are all treated as (M)FM recordings).

 

This was at least the case 10 years ago. Computers have increased their performance, and probably, MAME would be able to properly emulate MFM images of hard disks, but this would also mean that you had to run it on a recent CPU.

Okay, I see. Well, AFAIK the images are true MFM recordings, since the size is almost three times the size a raw sector dump would be. 

Anyway, I need to ready the archive first, but in the mean time it would be nice to find the best option for distributing it.

 

I do have an ISA MFM controller at hand, so I could throw everything on a single MFM disk (or two if need be), read a raw image with, say, dd in Linux, and then upload that one. If everything else fails, I'll create a bunch of ARK files and upload these, eventually zip'ed into fewer files.

[RickyDean]

25 minutes ago, mizapf said:

Are the images from PDP8online sector dumps of the hard disk, or are they true MFM recordings? MAME does not handle MFM images of hard disks, and neither does TIImageTool. The reason is that this would require too much performance to keep up with real time (in contrast to floppy disk images, which are all treated as (M)FM recordings).

 

This was at least the case 10 years ago. Computers have increased their performance, and probably, MAME would be able to properly emulate MFM images of hard disks, but this would also mean that you had to run it on a recent CPU.

Here is the website link:https://www.pdp8online.com/mfm/mfm.shtml

I have also asked David the creator this question.

[Brufnus]

Just now, RickyDean said:

Here is the website link:https://www.pdp8online.com/mfm/mfm.shtml

I have also asked David the creator this question.

Yep, I know that page, but thanks anyway. He really need tidying up that web page, haha  Anyway, it's probably possible to create a raw image somehow; perhaps the emulator software can do a trick in order to do that (I haven't figured out how or if, though...) c".)

[RickyDean]

24 minutes ago, Brufnus said:

Okay, I see. Well, AFAIK the images are true MFM recordings, since the size is almost three times the size a raw sector dump would be. 

Anyway, I need to ready the archive first, but in the mean time it would be nice to find the best option for distributing it.

 

I do have an ISA MFM controller at hand, so I could throw everything on a single MFM disk (or two if need be), read a raw image with, say, dd in Linux, and then upload that one. If everything else fails, I'll create a bunch of ARK files and upload these, eventually zip'ed into fewer files.

 

 

Here's some information from one of his pages: https://www.pdp8online.com/mfm/code/mfm/ext2emu_doc.html 

 

Ext2emu converts an extracted data file to an emulator file.

 

--cylinders -c #

The number of cylinders.

--emulation_file -m filename

File name to write emulation bit data to.

--extracted_data_file -e filename

File name to read decoded data from.

--format -f formatName

The track format. Use --format help to list all currently supported formats.

--heads -h #

The number of heads.

--interleave -i #[,#]

The sector interleave and track interleave. The first parameter is the value to increment sector number by between sectors on the same track. If interleave is not specified default of 1 is used. The second parameter is the value to increment the first sector by for next track on the same cylinder. Default is zero.

You can use –analyze on an existing disk or image to determine the proper value. Subtract the first two number printed for interleave and use that value. For example below use 7.

Interleave (not checked): 0 7 14 4 11 1 8 15 5 12 2 9 16 6 13 3 10

If you get a message like “Interleave mismatch previous entry...” it may indicate the second parameter should be non zero. You would need to use mfm_read/mfm_util –quiet 1 to see the sector values and manually determine the proper value. Getting the second parameter wrong will only have a small impact on performance. Getting the first value wrong can significantly reduce performance.

--mark_bad -M #,#,#:#,#,#...

The cylinder,head,sector to mark bad by complementing the CRC/Check code. Use to force sectors that are known to be bad to be seen as bad by the host system.

--note -n “string”

String is stored in header of emulation file for information about image. mfm_util will display.

--quiet -q #h

Bit mask to select which messages not to print. 0 is print all messages. Default is 1 (no debug messages). Higher bits are more important messages in general.

--version -v

Print program version number.

 

Long options can be abbreviated to the shortest unique name. Option values can't have spaces unless quoted as a string.

 

# is a number. #h is a number which may be decimal, octal if starts with a 0, or hex starting with 0x.

 

Cylinders, heads, format, emulation_file and extracted_data_file are required. The cylinders, format, and heads parameters can be determined by mfm_util if an emulator or transitions file is available.

 

NOTE: Some computers use different format on different tracks. This program can only handle one format.

 

Example:

 

ext2emu --extracted_data_file extracted_data --emulation_file emulation_file --cylinders 300 --heads 4 --format OMTI_5510 --interleave 3,1 –mark_bad 3,2,6:6,1,3:200,4,14 –note “Regenerated file”

 

This would generate sectors 0, 3, 6… for first track, 1, 4, 7... for second track, 2, 5, 8... for third. The start of the next cylinder will start back with sector 0.

 

If the extracted data file size calculated using the specified cylinders and tracks doesn't match the actual file size the warning message “Calculated extract file size # bytes, actual size #'” will be printed. Verify the parameters are correct for the file being converted.

 

 

and then here is some more info: https://www.pdp8online.com/mfm/code/emu/mfm_emu_doc.html

Edited September 1 by RickyDean
added content

[RickyDean]

@mizapf, @Brufnus,

David got back with me with this information:

The extracted data file --extracted_data_file is the sector data in
the order specified by the sector header. If the sector header indicates
where bad sectors have been relocated to they are put back in the proper
location. Some controllers use part of the disk for storing the disk
geometry or bad sector relocation data etc. For those controllers the
extracted data file has what is on the disk not how the sectors are
seen by the host system. There is no metadata in the file.

Note that for MYARC_HFDC I found some sector headers are marked in header
with deleted data 0xf8. If that information is needed it does not exist in the
 extracted data file since it is only the sector data. I don't know if
the controller uses part of the disk for its usage.

The --emulation_file is the entire track of mfm encoded data with metadata
giving disk geometry etc.

The --transitions_file is the raw flux transition timing. Also has metadata.

I would think you would want to use the extracted data file. I think that
is similar to how most emulators store disk images.

This emulator does support using the emulation file directly.
http://48k.ca/trs80gp.html

[Brufnus]

Actually, the MFM emulator creates not one, but two images. I believe the smaller of these is a raw replica of the drive; I'll put it to the test in MAME and if that works properly, then I'll make such images available. c",)

[Brufnus]

5 minutes ago, RickyDean said:

@mizapf, @Brufnus,

David got back with me with this information:

The extracted data file --extracted_data_file is the sector data in
the order specified by the sector header. If the sector header indicates
where bad sectors have been relocated to they are put back in the proper
location. Some controllers use part of the disk for storing the disk
geometry or bad sector relocation data etc. For those controllers the
extracted data file has what is on the disk not how the sectors are
seen by the host system. There is no metadata in the file.

Note that for MYARC_HFDC I found some sector headers are marked in header
with deleted data 0xf8. If that information is needed it does not exist in the
 extracted data file since it is only the sector data. I don't know if
the controller uses part of the disk for its usage.

The --emulation_file is the entire track of mfm encoded data with metadata
giving disk geometry etc.

The --transitions_file is the raw flux transition timing. Also has metadata.

I would think you would want to use the extracted data file. I think that
is similar to how most emulators store disk images.

This emulator does support using the emulation file directly.
http://48k.ca/trs80gp.html

Yep, it seems that it should work that way.... it's evolving faster than I can keep up with it, haha  Just downloaded and installed the latest MAME as well, just need to find that bloody boot ROM (I'm sure I have it somewhere).

 

 

[Brufnus]

Okay, got MAME working now.... it does complain about the HDD image being invalid however, so I'll have to investigate this.

Edited September 1 by Brufnus