ATRFS: Mount ATR files as local directories under Linux or Mac

[pcrow]

8 hours ago, Piotr D. Kaczorowski said:

A small request that will enable debugging under different systems with different versions of libraries. It shouldn't be too complicated and probably won't take much time, but it can help many times.

 

Please consider to add a "--dummy" option, e.g. "atrfs --dummy /mnt/atrdisk"?

 

The goal is to test-install a virtual ATR, which will return a short list of files, or for files, it will return the content "hello world", allow copying a file to the virtual disk - but only this will create a file that will contain "hello world". Such a thing will make it easier to add DEBUG and determine incompatibility with a given system or fuse library.

I don't quite follow what the request here is.

 

You can create a new image with the --create option.

 

I could have it copy the ATR file into memory instead of doing a memmap of the file itself so that nothing is changed in the original file, which would be more convenient than copying the ATR file before using it.

 

I could combine the two to create a new image in memory only.

[Piotr D. Kaczorowski]

@pcrow,

 

Alright. Everything is going great, and we can probably refine and add new systems. The only drawback right now is that it simply doesn't work for me - Input/Output Error 5. So, we need to determine the next steps. Do you have any ideas, or should I try to set up a tunnel and expose the host so you can check it, or should I add debugging in my free time and see what's going on?

[pcrow]

4 minutes ago, Piotr D. Kaczorowski said:

Alright. Everything is going great, and we can probably refine and add new systems. The only drawback right now is that it simply doesn't work for me - Input/Output Error 5. So, we need to determine the next steps. Do you have any ideas, or should I try to set up a tunnel and expose the host so you can check it, or should I add debugging in my free time and see what's going on?

Start by sending me an image that you're trying in a private message.  Hopefully I'll see the same thing.

[Piotr D. Kaczorowski]

2 minutes ago, pcrow said:

Start by sending me an image that you're trying in a private message.  Hopefully I'll see the same thing.

I've checked 3 images... staring from two standard atrs from LiteDOS dist. archive and 3rd was SDX.. Same results.

[pcrow]

2 minutes ago, Piotr D. Kaczorowski said:

I've checked 3 images... staring from two standard atrs from LiteDOS dist. archive and 3rd was SDX.. Same results.

How far does it get?  Run with: -f -s --atrdebug

That will keep it from forking to the background and add all the debugging.

Also check if it's just the directory listings that are broken.  If your mount point is ./mnt, then:

cat mnt/.fsinfo

cat mnt/.bootinfo

hexdump -C mnt/.sector1

cp mnt/DOS.SYS .

(for the last, use any file you know exists on the image)

Don't try to change the working directory into the mount point until it's stable for you.

[Piotr D. Kaczorowski]

34 minutes ago, pcrow said:

How far does it get?  Run with: -f -s --atrdebug

That will keep it from forking to the background and add all the debugging.

Also check if it's just the directory listings that are broken.  If your mount point is ./mnt, then:

cat mnt/.fsinfo

cat mnt/.bootinfo

hexdump -C mnt/.sector1

cp mnt/DOS.SYS .

(for the last, use any file you know exists on the image)

Don't try to change the working directory into the mount point until it's stable for you.

 

 

 

It doesn't go too far...

 

 

 

Edited September 14, 2023 by Piotr D. Kaczorowski

[pcrow]

That's strange.  The debugging output shows it's going through the code flow to look up the main directory, and it looks like it's giving a good result  After that, reading .sector1 shouldn't be even touching the litedos.c code, and it shouldn't have an EIO getting returned.  The EBADFD is probably a bug in hexdump's error handling causing a secondary error, so I would ignore that.

 

MacOS seems to be doing things a little differently, as I don't see the lookups on the main directory when doing the operations I described.  It could also just be the older version of FUSE is behaving differently.  I installed FUSE 2.9, and building with that gives me the same working behavior as FUSE 3.

 

I wonder if MacOS is expecting something different from the file system?  I'm suspecting that it's something that isn't needed on Linux, so I didn't implement it, but on MacOS it's calling it and getting EIO before letting you do anything.

 

Here's an idea: Use -oro to mount read-only.  I've seen MacOS .zip files with ._MacOS or something like that, and if it's trying to create those, that will fail, causing EIO.  If it's read-only, it shouldn't try to create anything.

 

Side note: I just pushed the ability to specify --atrdebug=# for higher debug levels.  Use '2' for more verbosity on a few things.

[Piotr D. Kaczorowski]

12 minutes ago, pcrow said:

That's strange.  The debugging output shows it's going through the code flow to look up the main directory, and it looks like it's giving a good result  After that, reading .sector1 shouldn't be even touching the litedos.c code, and it shouldn't have an EIO getting returned.  The EBADFD is probably a bug in hexdump's error handling causing a secondary error, so I would ignore that.

 

MacOS seems to be doing things a little differently, as I don't see the lookups on the main directory when doing the operations I described.  It could also just be the older version of FUSE is behaving differently.  I installed FUSE 2.9, and building with that gives me the same working behavior as FUSE 3.

 

I wonder if MacOS is expecting something different from the file system?  I'm suspecting that it's something that isn't needed on Linux, so I didn't implement it, but on MacOS it's calling it and getting EIO before letting you do anything.

 

Here's an idea: Use -oro to mount read-only.  I've seen MacOS .zip files with ._MacOS or something like that, and if it's trying to create those, that will fail, causing EIO.  If it's read-only, it shouldn't try to create anything.

 

Side note: I just pushed the ability to specify --atrdebug=# for higher debug levels.  Use '2' for more verbosity on a few things.

 

Left window:

$ ./atrfs -f -s --atrdebug=2 -oro --name=a.atr atr1

DEBUG: dos4_sanity: Cluster size: 6  VTOC Clusters: 66-67 (sectors 349-360)  VTOC Sector count: 1  First VTOC sector: 360 Max DIR entries: 88 Max Cluster: 128

DEBUG: atr_preinit detected LiteDOS image

DEBUG: atr_statfs

DEBUG: atr_statfs

DEBUG: atr_statfs

DEBUG: atr_statfs

DEBUG: atr_getattr /

DEBUG: litedos_getattr: /

DEBUG: litedos_path: /

DEBUG: litedos_getattr: / lookup 0

DEBUG: litedos_getattr: / mode 040755

...

...

...

 

DEBUG: atr_getattr /

DEBUG: litedos_getattr: /

DEBUG: litedos_path: /

DEBUG: litedos_getattr: / lookup 0

DEBUG: litedos_getattr: / mode 040755

 

 

Right window:

$ cat atr1/.fsinfo

cat: atr1/.fsinfo: Input/output error

 

$ cat atr1/.bootinfo

cat: atr1/.bootinfo: Input/output error

 

$ hexdump -C atr1/.sector1

hexdump: atr1/.sector1: Input/output error

hexdump: atr1/.sector1: Bad file descriptor

 

 

 

 

[Piotr D. Kaczorowski]

@pcrow,

 

By the way, on this computer, I normally use Macfusion with sshfs, which uses fuse.

[pcrow]

Hmmm.  Grasping at straws here.  Could MacOS behave badly based on statfs saying the max name length is 12?  Change that to 128 in litedos_statfs() on line 1306 of litedos.c.

 

It's looking like the EIO is coming from MacOS itself or the FUSE library, not my code.

 

You could also: sed -i -e s/-EIO/-EINVAL/ *.c

That would change the errnos that my code sends, so you would presumably get a different error message if it's coming from those error codes.  I don't expect that to change the output, but if it does, it gives us something to go on.

[Piotr D. Kaczorowski]

57 minutes ago, pcrow said:

oking like the EIO is coming from MacOS itself or

Still the same...

 

look at this:   "?atr1" (not /atr1):

 

 

 

[Piotr D. Kaczorowski]

@pcrow,

 

This example works: https://github.com/MaaSTaaR/SSFS

[pcrow]

11 minutes ago, Piotr D. Kaczorowski said:

This example works: https://github.com/MaaSTaaR/SSFS

That looks exactly like what I'm doing, only I've got more stuff.  Just to eliminate anything that might be confusing it, in atrfs.c at line 594 remove everything from atr_oper except getattr, readdir, and read.  When I do that, I can still list the directory and do the hexdump on .sector1.  That will either say something else is confusing things on the Mac, or something is wrong in those functions.  If that works, then add them back in until you find the culprit.  I would think .init or .statfs would be the only other ones getting called.

[Piotr D. Kaczorowski]

21 minutes ago, pcrow said:

That looks exactly like what I'm doing, only I've got more stuff.  Just to eliminate anything that might be confusing it, in atrfs.c at line 594 remove everything from atr_oper except getattr, readdir, and read.  When I do that, I can still list the directory and do the hexdump on .sector1.  That will either say something else is confusing things on the Mac, or something is wrong in those functions.  If that works, then add them back in until you find the culprit.  I would think .init or .statfs would be the only other ones getting called.

Ok.  During Friday I should have time to debug it. 

[+x=usr(1536)]

On 9/13/2023 at 4:05 PM, Piotr D. Kaczorowski said:

atrfs@osxfuse0 on /Users/piotr/Downloads/LiteDOS-SE/atr1 (osxfuse, nodev, nosuid, synchronous, mounted by piotr)

 

As you can see, I'm using FUSE version 3.10.4 for OSX (FUSE for macOS 3.10.4).

Unfortunately, the mounted directory isn't functioning. I receive an 'Input/Output Error (5)' when I attempt to access it.

 

 

Do you have any suggestions on how to resolve this issue?

Hang on a sec - are you using osxfuse or macFUSE?  osxfuse has been deprecated for a long, long time, and was replaced by macFUSE.

 

There were known (in)compatibility issues between them, which may be the cause of what you're seeing here.

[Piotr D. Kaczorowski]

2 hours ago, x=usr(1536) said:

Hang on a sec - are you using osxfuse or macFUSE?  osxfuse has been deprecated for a long, long time, and was replaced by macFUSE.

 

There were known (in)compatibility issues between them, which may be the cause of what you're seeing here.

So it happened that I work on a Mac Pro 3,1 (8xXeon, 32GB, SSD RAID 0) + Apple Cinema Display 30", which hardware-wise is not terrible, but thanks to Apple's policy, I have the OS X El Capitan operating system in version 10.11.6. On this system, I have the version that I have, and it works. Subsequent ones don't work. I normally use SSHFS together with Fuse. Everything works. I also provided an example, which also works without any problems. When the prices drop a bit, I plan to buy a Mac Pro Xeon W, but for now, it's still several thousand USD/EUR. For now, it might be cheaper for me to rewrite a piece of this software

[pcrow]

I just pushed a change altering the syntax for creating new images.  Now it's:  --create --fs=mydos  --sectors=720 --sectorsize=128

Or whatever.  This makes the code simpler, as I can put the file system name in a table and not mess with the parameters every time I add support for a new type.  I also added "blank" as an option.

[+x=usr(1536)]

13 hours ago, Piotr D. Kaczorowski said:

For now, it might be cheaper for me to rewrite a piece of this software

Understood, and having had an iMac of that generation (2008-ish), I completely understand why you're topped out at OS X 10.11.

 

Thing is, most of the rest of the world is on macFUSE by now.  And while I have no issues with you wanting to do an osxFUSE build of atrfs, please remember that you are building for an obsolete (and unsupported) target.  If it works, great, but you are kind of a corner case on this one 

 

(And FWIW, I feel your pain re: upgrading.  I'm typing this on an early 2018 MBP, and the next macOS release will probably be the last one that will support this machine.  Not thrilled about being forced into that when this one still does what I need.)

[Piotr D. Kaczorowski]

14 minutes ago, x=usr(1536) said:

Understood, and having had an iMac of that generation (2008-ish), I completely understand why you're topped out at OS X 10.11.

 

Thing is, most of the rest of the world is on macFUSE by now.  And while I have no issues with you wanting to do an osxFUSE build of atrfs, please remember that you are building for an obsolete (and unsupported) target.  If it works, great, but you are kind of a corner case on this one 

 

(And FWIW, I feel your pain re: upgrading.  I'm typing this on an early 2018 MBP, and the next macOS release will probably be the last one that will support this machine.  Not thrilled about being forced into that when this one still does what I need.)

You know.. in Poland, I connected the first school to the Internet in 1993, and probably even earlier, for 2-3 years, I was working on Linux and Coherent/Xenix/Minix/Solaris/HP-UX. So I'm used to the idea that if something doesn't work, I can compile and fix it myself. Back in the day, as you might remember, a Unix Administrator = System Programmer. So, I'm kind of stuck on the old OSX. The interface is fine, my main tools are still iTerm2/tmux/joe/vim/brew, etc... Unfortunately, I'm reaching the point where OSX is becoming a distortion of its former self. Generally, the idea used to be that everything works, and you focus on the work, not on preparing to work. Let me put it this way.. I'm not really convinced about computers on M1/M2/etc... because I feel like it's working on a mobile phone processor dressed up as a computer...

Edited September 15, 2023 by Piotr D. Kaczorowski

[pcrow]

In any case, since the simple test FUSE program works, I would like to understand why mine doesn't.  If something is broken on an older FUSE version, then it would be great to work around it so that someone else doesn't hit it later.

[Piotr D. Kaczorowski]

15 minutes ago, pcrow said:

In any case, since the simple test FUSE program works, I would like to understand why mine doesn't.  If something is broken on an older FUSE version, then it would be great to work around it so that someone else doesn't hit it later.

 

That's why I mentioned adding "--dummy" to the current version. It should be very simple for you, and it will make the transition from one library to the other quite easy.

[Piotr D. Kaczorowski]

@pcrow,

 

Proposal for a Change Release.

 

(1)

 

Look at the directory of LiteDos FS.

 

From the Unix filesystem, we should have some different filename accessors.

For example:

1. "LITERD  DRV" should be accessed by "LITERD  DRV" or "LITERD.DRV"

2. "LITEINITXEX" should be accessed by "LITEINITXEX" or "LITEINIT.XEX"

 

(2)

 

In the future version, when mounting the ATR to the system, there should be an option for unix/crlf/cr/etc...

 

After verifying that a given file is text-based, with conversion enabled, it should recode characters and line endings to the host system, and when copying from the host, convert to the Atari convention.

 

If the above already exists, I apologize for the confusion

 

[pcrow]

6 minutes ago, Piotr D. Kaczorowski said:

From the Unix filesystem, we should have some different filename accessors.

For example:

1. "LITERD  DRV" should be accessed by "LITERD  DRV" or "LITERD.DRV"

2. "LITEINITXEX" should be accessed by "LITEINITXEX" or "LITEINIT.XEX"

I had to pick one for the directory listing, and treating the files as 8.3 file names seems like the most standard option.

For creating new files, you can just treat it as an 11-character file name, provided you don't use any '.' characters.  I believe that works for accessing existing files, too, as the lookup code takes what you enter and splits it into the 11 characters as best it can (skipping to the last three if it sees a '.', otherwise just copying byte by byte).

 

I think I block creating files with directory-separation characters on file systems that use subdirectories, but otherwise I avoid mangling them.

 

This will be a mess for files with reverse-video file names, though it's just cosmetic.  In Unix, the only characters you can't have in a file name are '/' and 0x00.  I suppose someone out there loves to put zero-valued characters in their file names, and that's going to interact badly with atrfs.  I don't ♥ dealing with that problem.

[pcrow]

15 minutes ago, Piotr D. Kaczorowski said:

In the future version, when mounting the ATR to the system, there should be an option for unix/crlf/cr/etc...

 

After verifying that a given file is text-based, with conversion enabled, it should recode characters and line endings to the host system, and when copying from the host, convert to the Atari convention.

I anticipate the most common use case is moving files around that originated on the Atari and will end up on an Atari.  I don't want to have files getting modified when copying them.

 

However, it would be fine to have a magic extension that does text conversion from ATASCII to ASCII.  Something like adding ".text" or ".ascii" to the file name when reading it.  As long as the magic extension is four or more characters, it wouldn't interfere with any Atari files.  I already do that for ".info" which shows the binary load blocks or a summary of the BASIC structure of applicable files.

[Piotr D. Kaczorowski]

4 minutes ago, pcrow said:

I had to pick one for the directory listing, and treating the files as 8.3 file names seems like the most standard option.

For creating new files, you can just treat it as an 11-character file name, provided you don't use any '.' characters.  I believe that works for accessing existing files, too, as the lookup code takes what you enter and splits it into the 11 characters as best it can (skipping to the last three if it sees a '.', otherwise just copying byte by byte).

 

I think I block creating files with directory-separation characters on file systems that use subdirectories, but otherwise I avoid mangling them.

 

This will be a mess for files with reverse-video file names, though it's just cosmetic.  In Unix, the only characters you can't have in a file name are '/' and 0x00.  I suppose someone out there loves to put zero-valued characters in their file names, and that's going to interact badly with atrfs.  I don't ♥ dealing with that problem.

 

  it was just proposal.... For now, there are probably more important things to do. However, for instance, in OSX there are systems that consider uppercase letters and treat both lowercase and uppercase the same. This is a design pattern that says a file can have a certain physical name, but many accessors.