Myarc HFDC Bug?

[Brufnus]

On 6/18/2019 at 1:08 PM, 9640News said:

I've been working with Oleksandr who developed the DREM to emulate MFM drives.  Everything has been working fine with drives up to 8 heads x 1024 cylinders (64 MB's).  Our Myarc HFDC is supposed to support 2048 cylinders.

 

First question, has anyone formatted a MFM hard drive that used more than 1024 cylinders?  If so, did you use MDM5 or did you possibly use Mike Maksimik's CFORM program?  If MDM5, do you know what version you are likely to have used?

 

I have used MDM5 V1.30 to format a hard drive.  Whenever there is an attempt to copy a file (under MDOS using XCOPY) to the image, a cylinder 1024 I get an error.  I've been providing Oleksandr with a number of log files of the formatting process and I have enclosed his following comments below.  Under MAME when I have used a 8 head x 2048 cylinder, I do not get an error.  What I do not know is how MAME handles the sector gap information.  Both MAME and the DREM do not  include the gap information, but how they get to the same point may be a bit different????  Hopefully, someone will have some ideas????

 

 

From Oleksandr:

 

I guess it is a bug and real 2048 track HDD will not work too.

Controller should form an id byte as:

0xFE ^ (track >>

 

However for tracks 1024+ it always stay 0xFD I guess it is causing read error.

 

SMC datasheet describe ID Field Ident byte as FE 0 FF 256 FC 512 FD 768 Custom Ident byte is possible, but if we will use  the same scheme:

0xFE ^ (track >> 8) it will produce 0xFB and cause the conflict with data field ident byte.

 

It seems I've done something that shouldn't be possible. c".)

 

Using the H3-mod'ed HFDC and my new MFM emulator from David Gesswein, I tried pushing the limits as far as I could.

MDM5 V1.50 seemed to format properly with 16 heads and 2048 cylinders, but failed when trying to use it. 

 

Then I remembered there's a 250 MB partition limit with the WHTech SCSI card, so I tried again with only 1984 cylinders - and now it works! I have yet to verify how it will behave when addressing above a certain cylinder count (1024 or whatever), but I've copied quite a few files to that image now and booted from it... seems to work fine for now.

 

I guess the original 128 MB limit (134 according to my manual, perhaps if done with 34 sectors per track?) were merely because of the head # limit. If so, not surprisingly a >8 head mod would break that barrier.

 

Anyway, now I'll test it thoroughly, and - if everything goes well - there's an option there for anyone wanting half a gig disk space with the HFDC... in my world, this would be absolutely mind blowing.... c".= 

Edited October 20, 2022 by Brufnus

[+InsaneMultitasker]

Just because MDM5 let you do this, doesn't mean that it worked    If your HFDC is not modded to support the extra heads, data will almost certainly be overwritten.  The directory and file descriptors may not overlap right away due to the formatted capacity but I would guess your file data is compromised.

 

With 8 heads, the maximum addressable sectors is  (31) total bitmap sectors  x the (256) bytes/sector x (8) allocation units/byte * (8) sectors/allocation unit =  31x256x8x8 = 507904 sectors.   With 16 heads, the sectors/allocation unit doubles:  31x256x8x16 = 1015808 sectors.

 

There are some cases where formatting fewer sectors is preferable. For example, formatting a 128MiB platter at 8 sectors/AU versus 16 sectors/AU provides nearly double the total AUs, and allow for more files on the platter.

[Brufnus]

1 minute ago, InsaneMultitasker said:

Just because MDM5 let you do this, doesn't mean that it worked    If your HFDC is not modded to support the extra heads, data will almost certainly be overwritten.  The directory and file descriptors may not overlap right away due to the formatted capacity but I would guess your file data is compromised.

 

With 8 heads, the maximum addressable sectors is  (31) total bitmap sectors  x the (256) bytes/sector x (8) allocation units/byte * (8) sectors/allocation unit =  31x256x8x8 = 507904 sectors.   With 16 heads, the sectors/allocation unit doubles:  31x256x8x16 = 1015808 sectors.

 

There are some cases where formatting fewer sectors is preferable. For example, formatting a 128MiB platter at 8 sectors/AU versus 16 sectors/AU provides nearly double the total AUs, and allow for more files on the platter.

Yes I know there are certain limitations, which I needed to find out one way or another (your figures such as 1015808 sectors are most useful with respect to this). Anyway, as I mentioned, the HFDC has the H3 mod.

 

Anyway, calculating back from the 1015808 divided with 16 heads, 32 sectors per track... that gives exactly the 1984 cylinders that I used. May be either pure luck or my well developed intution, haha  

 

Since it matches your figures, I'll continue with this setup and test how it behaves when I fill it entirely. Hopefully my discoveries will turn out useful for others as well, if they wish to improve on their HFDC setup. 

[+InsaneMultitasker]

6 minutes ago, Brufnus said:

Anyway, as I mentioned, the HFDC has the H3 mod

Ah hah!  I read this but did not realize you were referring to the 16-head mod     Very good, and excellent guesswork!

[Brufnus]

3 hours ago, InsaneMultitasker said:

Ah hah!  I read this but did not realize you were referring to the 16-head mod     Very good, and excellent guesswork!

Okay, hehe

 

Just for good measure, I won't use it for anything important of course, until I've made absolutely sure no corruption takes place (making large dir's redundant in subdirs, and such).

 

If need be to go as low as 64 MB, it's still quite valuable for archiving my many floppies. Swapping images is as easy as editing the config file and restarting the service... thus, it can be done even with the TI/Geneve up and running.

 

Once I have the results, I'll give it a review - thumbs up or down. In any case, there's no doubt this emulator is an excellent piece of hardware, that makes a good competitor to the DREM. 

Edited October 21, 2022 by Brufnus

[RickyDean]

I'll have to get started on mine in the not to distant future. Money has been the main issue and if I remember correctly some of the parts had gone up in price. But I've been working some overtime and money is not quite as much of an issue. The Avalanche has been a time consumer too.

Still will need about thousand or so to finish it, but it will have all the bells a whistles I want, without the giant price tag of even another good condition Avalanche  or other vehicle.

I need to get my PFM+ Geneve done and work some of my other TI projects, so I can examine the Myarc HFDC Bug 😂

[Brufnus]

Yeah, such car and motorcycle projects are real time and money consumers. c".)

 

Luckily I avoided an excessive price tag on the emulator and beaglebone; the inflation here in Denmark is enough already as it is... 😕

Anyway, I'll be testing my setup with 8x2048 heads/cylinders as well as 16x1984 (I'm working on filling the latter with dsk emulation files right now). Hopefully I'll be able to report back that no file corruption has taken place.

I'm at 216,000 sectors used now...  

Edited October 21, 2022 by Brufnus

[+InsaneMultitasker]

@Brufnus - when I test HFDCs I execute a simple batch file that chain-copies MDOS/GeneveOS to fill up the hard drive.  It looks like this: 

 

COPY SYSTEM-SYS 01

COPY 01 02

COPY 02 03

and so on...

 

When complete, I use CRCOS to confirm the final file is valid.  If it is valid, the assumption is that all previous copies were completed without error. If it is not valid, I then check other files for errors, and of course try to discern if the HFDC is at fault.  CRCOS is included in the MDOS 7.30 distribution package. Each MDOS file is 500+ sectors so you can pretty quickly fill up a lot of space.  you might want to try this with your newly-formatted storage. 

 

 

[Brufnus]

1 hour ago, InsaneMultitasker said:

@Brufnus - when I test HFDCs I execute a simple batch file that chain-copies MDOS/GeneveOS to fill up the hard drive.  It looks like this: 

 

COPY SYSTEM-SYS 01

COPY 01 02

COPY 02 03

and so on...

 

When complete, I use CRCOS to confirm the final file is valid.  If it is valid, the assumption is that all previous copies were completed without error. If it is not valid, I then check other files for errors, and of course try to discern if the HFDC is at fault.  CRCOS is included in the MDOS 7.30 distribution package. Each MDOS file is 500+ sectors so you can pretty quickly fill up a lot of space.  you might want to try this with your newly-formatted storage. 

 

 

That's a good idea. I intended to copy a lot of dsk emulation files, some even as large as 1,44 MB, then testing if they emulates as expected... but your approach is definitely less time consuming. Actually I could do that with some emulation files, too, just to fill the drive as fast as possible.

 

With respect to the sector # limit - I guess that's what's causes the 250 MB limit on the WHTech SCSI card as well - thus, a file system/OS limitation rather than a controller limitation? If so, I feel confident in getting this working with the HFDC as well. 

Edited October 21, 2022 by Brufnus

[Brufnus]

That's crazy, by the way... emulating a floppy drive on an emulated hard drive.  

[+InsaneMultitasker]

42 minutes ago, Brufnus said:

With respect to the sector # limit - I guess that's what's causes the 250 MB limit on the WHTech SCSI card as well - thus, a file system/OS limitation rather

Correct, the limitation is at the file system level which is still decent considering the hard drive sizes at the time the HFDC was produced.  For reference, the same file system is used by the HFDC, SCSI, and IDE cards.  Additionally, the Geneve-based DSR can use the Horizon ramdisk as a 'hard drive' with the same file system, though capacity is limited to ~16MiB by hardware.

[Brufnus]

7 hours ago, InsaneMultitasker said:

Correct, the limitation is at the file system level which is still decent considering the hard drive sizes at the time the HFDC was produced.  For reference, the same file system is used by the HFDC, SCSI, and IDE cards.  Additionally, the Geneve-based DSR can use the Horizon ramdisk as a 'hard drive' with the same file system, though capacity is limited to ~16MiB by hardware.

Okay, I learn something new all the time, even after having owned it since the mid 80's.  I agree; back then it was more than adequate for the drives actually available... and even today, even a single 250 MB drive is huge for these computers. 

 

[Brufnus]

Oh dear... something went wrong. Have to edit this post and check out what happened. c".) 

(Why can't I delete it?)

Edited October 22, 2022 by Brufnus

[Brufnus]

250 MB with the HFDC...

 

Well, something went wrong, and I'll have to investigate that. Anyway, I successfully filled the entire drive with no apparent corruption taking place.

Then I deleted the test files with xdel, still apparently okay - but when I started deleting the dirs with DM, it corrupted the root dir for some reason (all subdirs seemed to be okay, though).

 

I'll try one more time and instead delete with MDM5 - luckily I can easily make a backup of the image and restore it, if need be.

As soon as I've done some thorough testing, I'll share my experiences, as well as uploading some photos of the setup.

 

[Brufnus]

While the HFDC is working like mad to fill my emulated drive, I found this in an old thread. I'm not sure, but I think I used both MDOS and MDM5 to create dir's, which is probably why I had some drive corruption after removing the dirs (I just had a little chat with Tim, and he also mentioned this issue, so that confirms it).

 

Thus, my last post was based on a wrong assumption - any file operations should  be done from MDOS alone and MDM5 be used for formatting only.

I'm confident this approach will be successful and confirm our HFDC is indeed capable of using 250 MB drives. That would be awesome!

 

(Probably no need to mention that I've encountered no bad sectors during format, either... hehe).

 

By the way, there are so many other advantages with using this emulator. It's fast and easy to backup using SCP, for example... perhaps one of the greatest advantages of using such an emulator.

 

It's also possible to make a new HDS1 image bootable even without using a floppy/RAM disk:

- attach a drive as HDS2 with the system files on it,

- boot from an image on the emulator, load MDM5 then swap images in the config file and restart the service,

- format the new HDS1 and copy the system files from HDS2, and

voila!

 

P.S. Excuse me for being giddy, but I cannot help it! c".) 

Edited October 23, 2022 by Brufnus

[+9640News]

Yeah, definitely an issue with using MDM5 and MDOS to create and remove directories.  As I recall, MDM5 did not follow the complete specifications for the subdirectory header name.  I believe it was missing the recursive pointer to the previous higher level directory.

 

 

[Brufnus]

That was indeed the reason, problem solved! I couldn't remember the exact problem before, but at least I hadn't forgotten that I couldn't remember.  🇩🇰

 

I had a single 1.44 MB image file that I couldn't delete from DOS for some reason; however, MDM5 did it and caused no apparent problems; thus, I was able to get rid of the remaining subdirs... and everything still works, including a test run of CFORM cloned from another clone in the testdirs. It shows the disk geometry correctly in that one, by the way.

 

Thus... 

750 MB WITH THE HFDC IS INDEED POSSIBLE!   (Although I think I can dispense with the 3rd - read-only - drive myself). 😛 

 

In turn, that also sort of answers the initial post in this thread; unless I possess a unique HFDC myself, it seems the DREM is responsible for the >1024 cylinder issue, not the controller itself.

 

Here are a couple of images; I apologize for the mess and bad quality of the first one... I think I'll just shove the emulator into the PEB so it will look nice and tidy and no external components sitting on the table.

 

 

This is an excellent piece of hardware that works amazingly with the HFDC, David definitely deserves some credit for creating this board!

I never thought I would see that amount of disk space on my Geneve with merely an HFDC... c".) 

 

Edited October 24, 2022 by Brufnus

[Brufnus]

There... when I came home yesterday, I prepared a cable with a header connector in both ends so that I could setup yet another 250 MB drive on the emulator. Thus, now I have around half a gig disk space with my HFDC. I think that should be enough for some time...  

[RickyDean]

10 hours ago, Brufnus said:

There... when I came home yesterday, I prepared a cable with a header connector in both ends so that I could setup yet another 250 MB drive on the emulator. Thus, now I have around half a gig disk space with my HFDC. I think that should be enough for some time...  

I'll have to get busy on mine again, when I got them, finances were slimmer. I already have the beagle bone black, and a white one, which may or may not work.

[Brufnus]

5 hours ago, RickyDean said:

I'll have to get busy on mine again, when I got them, finances were slimmer. I already have the beagle bone black, and a white one, which may or may not work.

I don't know the white one, but I suppose it'll work too, if it's got the same basic components. I've noticed one slighty annoying issue by the way; it doesn't always activate the Ethernet interface - I looked it up and it's simply a design flaw with the boards. Solution is to power cycle it or reset it on the microswitch. But as soon as it's configured and up and running, there's less need for the LAN connection of course.

Perhaps those with wi-fi aren't as prone to this problem...

 

I hope you'll get it up and running soon and enjoy it as much as I already do!

[Brufnus]

During my intensive floppy - and flippy - disk massage on the Geneve, I've noticed something weird. It's a long story why, but I have two separate PSU's for my emulator and disk drives, external to the PEB.

 

It seems that my NIC problem vanishes if I turn on the emulator first, then the disk drives and PEB - when I turn on the floppies first, it usually fails. So perhaps it's not a big deal after all, but rather a result of my strange configuration. I'll try reconfiguring it when I have a spare moment and see, if that solves the problem. c",)

 

Darn is it tedious copying dozens and dozens of floppies to the emulator, by the way... 😞 haha  

Edited November 20, 2022 by Brufnus

[RickyDean]

"Darn is it tedious copying dozens and dozens of floppies to the emulator, by the way..."

 

But it will make your labor of love, long lasting. ☺

[Brufnus]

On 11/21/2022 at 4:07 AM, RickyDean said:

"Darn is it tedious copying dozens and dozens of floppies to the emulator, by the way..."

 

But it will make your labor of love, long lasting. ☺

Indeed! It's worth the efforts, and once everything is copied I'll have everything directly at my disposal on a single or two hdd images.

I've completed 7 boxes by now - probably about 400 disks or so - and halfway through.