BASIC and IOCB PUT BYTE pointer?

[tschak909]

@phaeron and anyone else who might know:

 

Am I hallucinating, or does Atari BASIC actually NOT do a full CIO call to PUTCHR or PUTREC, but rather calls the routine attached in the PUT BYTE ptr in the IOCB directly?

 

If so, WHY?!

 

-Thom

[Alfred]

Likely to save space. Only passing one byte, so simplest just to jump straight to byte putter. Sets X and Y to proper values, shouldn’t be an issue.

[phaeron]

Because CIO is slow for short strings and painfully slow for single character I/O. Whoever documented ICPTL/H in the OS Manual didn't like it:

Quote

The pointer was provided to accommodate the people writing the ATARI BASIC cartridge, and has no legitimate use in the system.

One thing that is sometimes missed when using this pointer: ICAX1Z needs to be set from ICAX1,X or some handlers will fail permission checks.

 

[Rybags]

Fairly sure the reason that inconsistent vector exists was for the benefit of Basic.

My suspicion is that Basic was likely being developed in parallel with a half baked OS, so with the full CIO interface not worked out the easy way would be to just assume there's a put character vector which would have meant minimal code change once the OS was finalised.

 

Of course, the same doesn't apply for get character so maybe it was just a speed hack.

[thorfdbg]

On 4/23/2020 at 5:50 AM, tschak909 said:

Am I hallucinating, or does Atari BASIC actually NOT do a full CIO call to PUTCHR or PUTREC, but rather calls the routine attached in the PUT BYTE ptr in the IOCB directly?

You are correct, it does not, and it is pretty dangerous that it does not. As for "why" - probably a historical artefact, probably to cut down the complexity. The Basic PRINT command, for example, prints character by character, counting characters to implement the comma operator, so in a sense it "needs" single byte-IO (could be done differently, of course, counting upfront).

 

The danger is that this requires an extra safety check in the handler to disregard bursting (bypassing CIO, i.e. the FMS does that to speed up block transfer) if the call comes through the put-one-byte vector as IOCBLen is not set and only a single byte is to be put. The typical check for that is that the PUT-function of the handler checks where the call is coming from, by checking the stack, and if it comes from the cart area, bursting is disabled as the IOCBLen values cannot be trusted, as they are not set by Basic. This works, but is stupid as it disallows bursting for any call coming from $a000 and above, no matter whether it is Basic or not, and no matter whether it is coming through the vector.

 

Os++ changes this in the sense that the Open function of the handler can replace the Put-one-byte function to ensure that the handler is aware that the PUT function was called via the vector, and not via CIO, so this special weird fixup in the handler is not required, and bursting continues to work for calls from $A000 *not* coming into the handler through the put-one-byte vector.

 

Basic should really do the right thing and go through CIO, but its a "a bit too late" to fix this.

Edited April 24, 2020 by thorfdbg

[tschak909]

This is depressing as hell. Seriously.

 

At least GET works correctly, but still, fuck me....

 

I want to compile a good amount of information and tutorial on writing CIO drivers, as this is something that the Atari community really fell short on, and CIO really was under-served with unique devices. 

 

-Thom

[thorfdbg]

3 hours ago, tschak909 said:

I want to compile a good amount of information and tutorial on writing CIO drivers, as this is something that the Atari community really fell short on, and CIO really was under-served with unique devices.

The DeReAtari book contains pretty much everything you need to know. This bursting problem is unfortunately troublesome, but it is an advanced technique to improve the performance of handlers.

 

I was not quite correct, though - another and better strategy is to allow only bursting if the call comes from the OsROM. It still requires a stack analysis. Kludgy.

[tschak909]

I disagree. I think there needs to be more concrete examples.

 

Mine is somewhat unique in that it does additional SIO communication to cut down on the amount of code required in the handler (the handler does not need to know every single possible command/network protocol supported), and the prototype is written in C.

 

Currently being written here:

https://github.com/FujiNetWIFI/atariwifi/tree/master/n-handler/prototype/src

 

-Thom

[phaeron]

4 hours ago, tschak909 said:

I want to compile a good amount of information and tutorial on writing CIO drivers, as this is something that the Atari community really fell short on, and CIO really was under-served with unique devices.

I started one, but haven't gotten around to finishing it:

Section 7.5, Custom device handlers.

 

In my experience, burst I/O is a pain but only an issue if you actually support it and your device is commonly used for bulk transfers, which is mainly limited to DOS. It did make a significant difference when I implemented it for an H: handler. Otherwise, supporting ICPTL/H is not that bad, especially if you don't use AUX2-5.

 

34 minutes ago, tschak909 said:

Mine is somewhat unique in that it does additional SIO communication to cut down on the amount of code required in the handler (the handler does not need to know every single possible command/network protocol supported), and the prototype is written in C.

 

Currently being written here:

https://github.com/FujiNetWIFI/atariwifi/tree/master/n-handler/prototype/src

How are you handling page zero usage? CIO device handlers aren't allocated page zero memory and have to save/restore what they use; even this is risky in the $80-FF region if DLI handlers are involved. I use heavy amounts of self-modifying code in my handlers to work around this. It looks like the handler is configured to use the math pack area at $E5-FF, which risks overwriting the DEG/RAD flag at $FB.

 

[tschak909]

1 hour ago, phaeron said:

I started one, but haven't gotten around to finishing it:

Section 7.5, Custom device handlers.

 

In my experience, burst I/O is a pain but only an issue if you actually support it and your device is commonly used for bulk transfers, which is mainly limited to DOS. It did make a significant difference when I implemented it for an H: handler. Otherwise, supporting ICPTL/H is not that bad, especially if you don't use AUX2-5.

 

How are you handling page zero usage? CIO device handlers aren't allocated page zero memory and have to save/restore what they use; even this is risky in the $80-FF region if DLI handlers are involved. I use heavy amounts of self-modifying code in my handlers to work around this. It looks like the handler is configured to use the math pack area at $E5-FF, which risks overwriting the DEG/RAD flag at $FB.

 

I deliberately haven't implemented any code to preserve page zero values, mostly because this is a prototype, and the intent here is to get functionality working so that it can be rewritten in assembler when ready. It works well enough at the moment so that I can work out the functionality. I have been going back and forth with @StefanD on this, for the last few months, and it will reach the point where we'll be able to make an assembler driver.

 

Since the N: device handler is intended to be used in both programming languages _AND_ DOS, I absolutely need to implement burst transfers.

 

I intend to make the N: device work in as many places as humanly possible, and any help that  can be given here would be most appreciated. This is where the vast majority of my #FujiNet dev time is going at the moment, and I am busting my arse to make this thing appear to "just work" when someone uses it, and this knowledge needs to be preserved for others to write drivers.

 

So the answer to "can you write a CIO handler in CC65? is, YES." the answer to "should you write a CIO handler in CC65? is, HELL NO!!! but I suck at assembler!"

 

-Thom

Edited April 24, 2020 by tschak909

[phaeron]

4 hours ago, tschak909 said:

Since the N: device handler is intended to be used in both programming languages _AND_ DOS, I absolutely need to implement burst transfers.

You don't need to implement burst transfers to work in any environment. The point of a burst transfer is to speed up a CIO Get Bytes or Put Bytes operation so it doesn't involve transferring one byte at a time between CIO and the handler. This is purely between the handler and CIO. DOS does this because otherwise the transfer is too slow, delaying the next sector read/write and causing the disk drive to take an extra revolution for each sector. This is orthogonal to whether your handler needs or does burst I/O. If your handler doesn't implement burst I/O, it'll still work, it'll just take a bit longer to process the command.

 

[tschak909]

I want to provide the best performance I can, so burst mode is indeed a good idea, but I want to get both the SIO and CIO layers working well enough, first.

 

The N: device is intended not only to be used within languages like BASIC, but also in DOS (e.g. with the TNFS protocol), this will become more evident, as I release more videos. 

 

-Thom

[thorfdbg]

12 hours ago, phaeron said:

I CIO device handlers aren't allocated page zero memory and have to save/restore what they use; even this is risky in the $80-FF region if DLI handlers are involved.

That is a bit harsh to say, and not entirely correct. There are a couple of zero-page variables that are reserved for the FMS other handlers can use as well. These are all temporaries, i.e. they are not supposed to hold their data *between* calls to the handler, but *within* a handler call, one can make use of them without any problem. For example, $43 through $48 are available for the handler, and $15 through $19 as well.

 

The FMS, in particular, has its own "extension" of the IOCB, that holds data that needs to be preserved.

[phaeron]

10 hours ago, thorfdbg said:

That is a bit harsh to say, and not entirely correct. There are a couple of zero-page variables that are reserved for the FMS other handlers can use as well. These are all temporaries, i.e. they are not supposed to hold their data *between* calls to the handler, but *within* a handler call, one can make use of them without any problem. For example, $43 through $48 are available for the handler, and $15 through $19 as well.

 

The FMS, in particular, has its own "extension" of the IOCB, that holds data that needs to be preserved.

These bytes are officially reserved for the FMS and not for other handlers. From Section 9 of the OS Manual:

 

Quote

Zero-Page RAM

Zero-page RAM has no spare bytes, and even if there were, there is no allocation scheme to support multiple program assignment of the spares. Therefore, the nonresident handler must save and restore the bytes of zero-page RAM it is going to use. The bytes to use must be chosen carefully, according to the following criteria:

The bytes cannot be accessed by an interrupt routine.

The bytes cannot be accessed by any noninterrupt code between the time the handler modifies the bytes and then restores the original values.

Examples follow to save and restore persistent Display Handler variables (COLCRS, in particular). There is no mention or example of reusing a zero-page location without first saving it.

 

It is true that using a variable that is volatile during the handler's execution is a useful and generally safe technique. The temporary locations assigned to SIO, in particular, should be safe to use since SIO is explicitly intended to be used from handlers and Atari's own resident and non-resident handlers do not save/restore either the temporaries or DCB when using it, so applications must consider those areas volatile when calling CIO. The main caveat is that they will get trashed when the handler uses them as well. The areas used by CIO can also be reused, though save/restore is required.

 

The FMS areas, on the other hand, only have minimal documentation in Appendix L and it is not mentioned whether they are persistently or temporarily used by FMS, and usage may vary with different DOSes. In addition, if DOS is not loaded, these bytes are arguably available for use by the running program. This is a valid scenario for a poll-based handler load for a cartridge. Thus, I would not recommend using the FMS areas.

 

Self-modifying code, while more difficult to write and sometimes less efficient, avoids these issues entirely. It is particularly useful for non-resident handlers that can make less assumptions about OS database usage than resident handlers, which can assume more by virtue of being part of the OS.