AspeQt: Yet another SIO2PC/APE clone

[w1k]

that same hmm

[Disk 1] Mounted 'contiki-irc-email.atr' as 'ED Diskette (130k)'.
[Disk 9] Mounted 'sdx-veci.atr' as '65535 sector DD HardDrive (16383k)'.
Serial port speed set to 19200.
Emulation started through standard serial port backend on 'COM1' with DSR handshaking.
Emulation stopped.
Serial port speed set to 19200.
Emulation started through standard serial port backend on 'COM1' with DSR handshaking.
Serial port speed set to 19200.
[Disk 1] Speed poll.
[Disk 1] Get PERCOM block (ED Diskette (130k)).
[Disk 1] Read sector 1 (128 bytes).
[Disk 4] command: $3f, aux: $0000 ignored. [x16]
[Disk 3] command: $3f, aux: $0000 ignored. [x16]
[Disk 2] command: $3f, aux: $0000 ignored. [x16]
[Disk 1] Speed poll.
[Disk 1] command: $48, aux: $0020 NAKed. [x16]
[Disk 1] Get status.

[Roydea6]

Interim Release:

 

Added Folder Image boot support for DOS XL and SmartDOS. Just download the archive and extract the contents into your AspeQt application directory.

 

This is only the Windows executable, I will commit the source to SF for anyone who is interested to compile it for other OS.

Reports missing QTNETWORK.DLL.

[atari8warez]

Reports missing QTNETWORK.DLL.

 

Missing file qtnetwork4.dll is now included, however I have also made a small change in the main executable so please extract both files: qtnetwork4.dll and aspeqt.exe.

 

Release also includes some preliminary network code. if you get the message "Network connection cannot be opened" but you actually do have an active WIFI or Ethernet connection to a network please inform me and tell me about your net configuration (i.e. your OS, name/model of your network interface)..

AspeQt v0.8.7 (interim).zip

Edited February 3, 2013 by atari8warez

[atari8warez]

that same hmm

 

Victor, is the problem happening only with SDX, did you try it with other DOS? (i.e. SpartaDOS 3.x, MyDos etc..).

Are you using the same Atari 800XE you recently had problems with?

Edited February 3, 2013 by atari8warez

[w1k]

tommorow i downgrade to sdx 4.45 and i try it

i using another atari 800xe, only with u1mb+sio2pc

Edited February 3, 2013 by w1k

[atari8warez]

tommorow i downgrade to sdx 4.45 and i try it

i using another atari 800xe, only with u1mb+sio2pc

 

Try reflashing SDX 4.46 first, 4.46 works in my 130XE with U1MB + SIO2PC. If it still doesn't work you can try downgrading to 4.45.

Edited February 3, 2013 by atari8warez

[w1k]

thanks.. reflash 4.46 helps..

strange

[atari8warez]

Glad it's working now, the previous flash might have gone wrong for some reason.....

[Roydea6]

 

Window 7 64bit Via Linksys cable/Router...

Edited February 3, 2013 by rdea6

[atari8warez]

Thank you, I will soon add an option to "not to show the message again", so that i won't bug you all the time...

[atari8warez]

This removes the annoying "Network Connection can not be opened" message from the interim release. I found that for some reason the app can only open a net connection on the development computer but not on target computers, so the message makes no sense for now, thus removed.

AspeQt v0.8.7 (interim, no net message).zip

[flashjazzcat]

How difficult (I hope not very) would it be to add to Aspeqt the facility to directly mount a partition on the PC's hard disk for unfiltered access by the Atari? By this I mean map Atari sector requests directly to the PC HDD partition sectors. I can see an immediate use for this... As well as something for the future.

[atari8warez]

Are you thinking of something like mounting a partition on SIDE (or similar devices) only it happens in PC, through an SIO2PC connection.

 

Even though, I haven't ever written an app to manipulate partitions on PC, I suppose it's not terribly difficult to accomplish that using the Disk Management Control API functions like:

 

IOCTL_DISK_SET_PARTITION_INFO_EX

IOCTL_DISK_GET_PARTITION_INFO_EX

IOCTL_DISK_SET_DRIVE_LAYOUT_EX

IOCTL_DISK_GET_DRIVE_LAYOUT_EX

IOCTL_DISK_GET_DRIVE_GEOMETRY_EX

 

etc....

 

And directly accessing disk sectors using CreateFile() API with elevated access rights (must be an administrator). However I've also read somewhere that under Win Vista and Win 7 (most likely also in Win 8 ), this may not be possible even with an admin account. This last bit I am no quite sure about and needs to be investigated more.

 

So if you're asking me if I can do it, I probably can (given that Win Vista/7/8 limitation is not true), how soon? not very soon as I have a lot of other stuff to keep me busy for a while. Currently I am struggling to migrate AspeQt code to Qt 5.0.1, it's almost done but due to some API changes in the new release I have to do some good testing to make sure everything that worked before still does.

 

 

EDIT: The last word from Microsoft:

Corruption or system instability can occur when a program writes directly to a volume that is mounted by a file system without first obtaining exclusive access to the volume. This is because the write to the volume may collide with the file system writes. When such collisions occur, the contents of the volume may be left in an inconsistent state.

 

To reduce the effect of this issue, changes were made to the file system and to the storage stack to restrict direct access to the disk or to a volume.

A file system can write to a volume handle only if the following conditions are true:

• Condition 1: The sectors that are being written to are boot sectors.
  Note This condition exists to support programs such as antivirus programs, Setup programs, and other programs that have to update the startup code of the system volume. The system volume cannot be locked.
  
• Condition 2: The sectors that are being written to reside outside the file system space.
  Note The region between the end of the file system space and the end of the volume space is not under the control of the file system. Therefore, there is no reason to require the volume to be locked to write to it.
  
• Condition 3: The volume has been locked implicitly by requesting exclusive write access.
  
• Condition 4: The volume has been locked explicitly by requesting a lock request or an unmount request.
  
• Condition 5: The write request has a SL_FORCE_DIRECT_WRITE flag that indicates that Condition 2 must be bypassed.
  

Edited February 8, 2013 by atari8warez

[flashjazzcat]

Pretty much getting direct access to a FAT32 formatted Windows partition via a filesystem driver on the A8.

 

Looks like there may be some technical obstacles to overcome. While it'll be a few weeks or months before I need this for my own purposes, I wondered if current and in-the-pipleline FAT drivers for SDX would be able to use it in the meantime. Really, the client understanding the host file system removes a lot of headaches (especially regarding write support). I think the possibilities are intriguing and numerous.

 

If you could look into the practicalities whenever you get time, anyway, I'd really appreciate it. I know how it is to be faced with a queue of feature requests when there are many other pressing priorities.

[atari8warez]

What I understand from the Microsoft blurb is that we pretty much have to exclusively lock the disk/partition to be able to do direct sector I/O, we can not share the volume used by the OS. We either have to put exclusive lock on the volume or to dismount the partition first which practically means we have to reserve that volume for A8 and it can not be shared by Windows for other purposes.

 

Phaeron might have more information on this and perhaps he may want to share his knowledge here.

Edited February 8, 2013 by atari8warez

[flashjazzcat]

OK - I watch with interest.

[HiassofT]

How difficult (I hope not very) would it be to add to Aspeqt the facility to directly mount a partition on the PC's hard disk for unfiltered access by the Atari? By this I mean map Atari sector requests directly to the PC HDD partition sectors.

One issue would be that SIO sector numbers are limited to max 65535 - so you would only be able to access the first 32MB of a partition (unless you extend the SIO protocol, for example add a command to set the higher value sector bits).

 

Raw device access shouldn't be too complicated, I've done this in myidetool (but no partition mapping/scanning, just pure raw device access). Of course getting windows out of the way and locking / unmounting the partition could be somewhat tricky.

 

so long,

 

Hias

[flashjazzcat]

One issue would be that SIO sector numbers are limited to max 65535 - so you would only be able to access the first 32MB of a partition (unless you extend the SIO protocol, for example add a command to set the higher value sector bits).

 

Good point. We have the XDCB (designed by KMK) which extends the DCB for 32-bit addressing, but that's aimed at PBI drivers (and used by the U1MB, SIDE, etc), and I'm not sure how serial transmission will deal with that. But hell - 32MB is enough to be getting on with for a start (will Windows even allow you to set up a 32MB partition these days?).

Edited February 8, 2013 by flashjazzcat

[atari8warez]

Good point. We have the XDCB (designed by KMK) which extends the DCB for 32-bit addressing, but that's aimed at PBI drivers (and used by the U1MB, SIDE, etc), and I'm not sure how serial transmission will deal with that.

SIO can not deal with that as is, however with a new device handler on the Atari side, some new SIO commands, and a bit of cooperation from AspeQt would be enough to handle 32 bit addressing through SIO.

 

But hell - 32MB is enough to be getting on with for a start (will Windows even allow you to set up a 32MB partition these days?).

 

Yes it will, however the actual minimum size depends on how the partition is formatted. (FAT32 requires at least 36MB)

[phaeron]

What I understand from the Microsoft blurb is that we pretty much have to exclusively lock the disk/partition to be able to do direct sector I/O, we can not share the volume used by the OS. We either have to put exclusive lock on the volume or to dismount the partition first which practically means we have to reserve that volume for A8 and it can not be shared by Windows for other purposes.

 

Phaeron might have more information on this and perhaps he may want to share his knowledge here.

 

You do need administrator access in order to access volumes directly. You do not, however, need an exclusive lock on a volume in order to read it, even if it is mounted. I've regularly had mounted FAT32 drives also mapped for read in Altirra. Write access is a different story, as the OS does attempt to prevent you from accidentally destroying partitions and according to the docs you've already found it does require an exclusive lock request. I haven't tried doing raw write access.

 

The other tricky part is finding the volume name for the volume you want. It's not necessarily mounted as a partition, so you can't use drive letters. The way Altirra does it is to go through the SetupDi*() APIs to enumerate all disk and volume objects in the system, then it attempts to map the long and ugly volume paths and display names for display purposes so the user can select the drive. This works but is an unmitigated pain in the posterior. The less correct but saner way to do this is to just enumerate all of the PhysicalDisk symbolic links.

 

Regardless of how you choose, you'll want to be 100% certain that AspeQt is always either asking the user which drive to use or being absolutely certain to remount the correct drives. What you would not want to happen is to accidentally auto-mount the wrong drive for write access because the physical disk numbers shifted since the last time the drive was mounted, especially if it is a removable drive. With regard to future write access in Altirra, I'm thinking of preventing mounting of any fixed volume or volume bigger than a threshold like 16GB unless the user absolutely confirms it is intended. I'll probably also do IPC to a small helper program so that only the helper program has to be elevated to admin access instead of the whole program.

 

There's one other factor to be aware of, and that's physical sector size. Newer hard drives are coming with physical sectors that are 4KB instead of 512b. That could be a problem for physical disk access in Windows because a volume handle has to be accessed using FILE_FLAG_NO_BUFFERING mode, which means sector aligned reads and writes. I haven't seen any problems in Altirra with accessing a 512e drive (4KB sectors with 512b emulation in drive firmware), but there could be problems trying to read and write 512 byte sectors on a 4K-only drive, which Windows 8 supports. No one is going to bother putting a 2TB drive directly on an Atari but it's not out of the realm of possibility that someone tries mounting a small FAT32 partition on one through AspeQt.

 

Before trying physical access, I would implement image support first. I find the .VHD format particularly convenient for testing as various tools and OSes support it for hard disk virtual images. For instance, Windows 7 has native support for creating and mounting .VHDs, and the disk2vhd utility can image a disk. This lets me copy files into a FAT32 partition in Windows and then remount it in Altirra. The .VHD format also supports dynamic allocation to save space on sparsely populated images.

[atari8warez]

You do need administrator access in order to access volumes directly. You do not, however, need an exclusive lock on a volume in order to read it, even if it is mounted.

 

That's right, exclusive lock is needed for write access

 

Before trying physical access, I would implement image support first. I find the .VHD format particularly convenient for testing as various tools and OSes support it for hard disk virtual images. For instance, Windows 7 has native support for creating and mounting .VHDs, and the disk2vhd utility can image a disk. This lets me copy files into a FAT32 partition in Windows and then remount it in Altirra. The .VHD format also supports dynamic allocation to save space on sparsely populated images.

 

Great info as usual..... I should definitely do some reading on .VHD (which I suppose stands for Virtual Hard Drive/Disk) format.

 

Thank you Phaeron

[flashjazzcat]

Bug I noticed since I started using the official builds with SDX: underscore is filtered out of filenames, while this character is legal in SpartaDOS filenames.

[atari8warez]

Is this related to Folder Images or just any image, SDX only or any DOS, and how does it affect the operation?

 

You can submit a ticket here

 

Note: I will no longer be accepting requests and bug reports unless officially submitted through a ticket. Sorry but I have to keep a tab on these things.

Edited February 14, 2013 by atari8warez

[flashjazzcat]

No problem... I suppose it can get a bit overwhelming. Do I answer your first question here, or in the ticket?

 

OK - ticket submitted.

Edited February 15, 2013 by flashjazzcat

[atari8warez]

No problem... I suppose it can get a bit overwhelming.

 

It's more like things are getting spread all over the thread, much easier to keep track of them if they're submitted through the ticket system.

Edited February 16, 2013 by atari8warez