meta A new language for the Atari

[Kaj de Vos]

I see RespeQt is a drive emulator on the other end of a serial line. If that wants to serve up host files, emulating a disk image would be logical, even though fraught with problems, as you said. But that's because there is only an SIO connection. There would be no H:, only D.

 

If RespeQt has H:, it must implement that with a custom protocol over the serial line, and no disk emulation would be necessary anymore. The logical reason to do this would be exactly to get rid of the problematic disk emulation.

 

If Atari800 and Altirra would do disk emulation without a serial line inbetween, that would be really backwards.

[phaeron]

10 hours ago, Kaj de Vos said:

 

@phaeron, it turns out Altirra corrupted the SpartaDOS 1 image. This seems to have happened when I used the Disk Editor to rename *.XEX to *.COM for SpartaDOS. This does not explain how it happened to the DOS 2.5 ATR. Just importing to the SpartaDOS 1 and MyDOS ATR's, and renaming in SpartaDOS itself, works.

 

SpartaDOS 1.x is prone to overly caching sectors and seeing or causing disk corruption if you modify disks on another computer or in the emulator's Disk Explorer while SD1.x is running. This is because the SpartaDOS 1 filesystem lacks a change number in the superblock and thus there isn't a way for the emulator to tell it that the disk has changed. This was fixed in SpartaDOS 2.0 and I don't recommend using SpartaDOS 1.

 

10 hours ago, Kaj de Vos said:

Now my programs work in SpartaDOS 1.1 and MyDOS! So the crashes are caused by the Atari800 H: device. This is really weird, because it works with all the other DOSes I tried. So both the Altirra and Atari800 H: devices are broken. But on Altirra it's fairly consistent due to an unimplemented CIO function that usually yields an error, while on Atari800 it only crashes SpartaDOS 1.1 and MyDOS in unexpected places. It's still possible that the code parts in CC65 that I identified trigger this, but the incompatibility is due to Atari800.

This is the limitation of the way that some DOSes interact with CIO devices like H:, namely that they expect the device itself to implement EXE loading. This is not possible in the general case for a couple of reasons.

 

First, DOSes use conflicting XIO numbers for such services. MyDOS 4.5 uses XIO 47 to load executables, while SpartaDOS 1.1 uses XIO 48. But XIO 47 in SpartaDOS is Get File Length. The H: device does not know what DOS is running.

 

Second, for this to work the H: device has to reimplement the EXE loader, and this isn't possible in all cases. SpartaDOS has an extended file format that requires access to some memory allocation variables that aren't public and change between versions.

 

8 hours ago, Kaj de Vos said:

The H: device works on Altirra with DOS 2.5, and on Atari800 also with SpartaDOS 3.2 and BW-DOS, at least. So it's clear that this works and SpartaDOS 3.2 and BW-DOS could also work with Altirra. I don't see why MyDOS and SpartaDOS 1.1 could not also be made to work, and I don't see why that wouldn't be desirable. For one thing, it would have saved us many days. The biggest archeological site of all is Altirra. It has enormous functionality. Why should there be a taboo on adding this?

Please read the CIO docs in the Atari Operating System Manual and inspect the CIO calls being made to the H: device by these DOSes before making a judgment like this. You are making unfair statements without full knowledge of the limitations of the CIO interface.

 

[drac030]

2 hours ago, phaeron said:

This was fixed in SpartaDOS 2.0 and I don't recommend using SpartaDOS 1.

ICD also highly recommended giving up SpartaDOS 1.1 already in 1988.

 

2 hours ago, phaeron said:

SpartaDOS has an extended file format that requires access to some memory allocation variables that aren't public and change between versions.

Memory allocation variables in SpartaDOS X are pointed to by symbol H_FENCE, mainly. It is true, though, that the memory management mechanisms have a bit evolved since 4.20 and 4.47 was subject to major changes in this respect (backward compatible, though). The whole idea of symbols is however such that when they are in use, data structures and procedures pointed to by them do not need to keep the same addresses from revision to revision (which greatly facilitates development).

Edited March 27, 2021 by drac030

[Kaj de Vos]

As I said, I only use SpartaDOS 1.1 for quick testing, because it's the fastest. And for compatibility testing, for which it is clearly significant. Productivity and compatibility are important to me.

 

Don't worry, I was a big SDX user at the time, and I will return to it when I will work natively on the Atari again. I found the new SpartaDOS X project and am really impressed by it.

[Kaj de Vos]

@phaeron, thanks for the deep insights!

 

8 hours ago, phaeron said:

SpartaDOS 1.x is prone to overly caching sectors and seeing or causing disk corruption if you modify disks on another computer or in the emulator's Disk Explorer while SD1.x is running. This is because the SpartaDOS 1 filesystem lacks a change number in the superblock and thus there isn't a way for the emulator to tell it that the disk has changed. This was fixed in SpartaDOS 2.0 and I don't recommend using SpartaDOS 1.

I made sure to unmount the disk in the emulator before I used the Disk Explorer, and I didn't do any write operations in SpartaDOS in that session, so that should not have been the cause.

 

8 hours ago, phaeron said:

First, DOSes use conflicting XIO numbers for such services. MyDOS 4.5 uses XIO 47 to load executables, while SpartaDOS 1.1 uses XIO 48. But XIO 47 in SpartaDOS is Get File Length. The H: device does not know what DOS is running.

Ah, indeed I saw some double implementations in MyDOS to be more compatible with SpartaDOS. It seems to me, then, that Atari800 implements the SpartaDOS interface, so it can work with SpartaDOS 3.2 and BW-DOS. But MyDOS just gets a file length, which is why the program isn't executed but MyDOS does not report an error.

[ivop]

17 hours ago, Kaj de Vos said:

I see RespeQt is a drive emulator on the other end of a serial line. If that wants to serve up host files, emulating a disk image would be logical, even though fraught with problems, as you said. But that's because there is only an SIO connection. There would be no H:, only D.

True. If you select a H device, you specify a local path that that will be an emulated Dx: drive.

 

17 hours ago, Kaj de Vos said:

If RespeQt has H:, it must implement that with a custom protocol over the serial line, and no disk emulation would be necessary anymore. The logical reason to do this would be exactly to get rid of the problematic disk emulation.

That's PCLINK.

 

[+CharlieChaplin]

14 hours ago, phaeron said:

First, DOSes use conflicting XIO numbers for such services. MyDOS 4.5 uses XIO 47 to load executables, while SpartaDOS 1.1 uses XIO 48. But XIO 47 in SpartaDOS is Get File Length. The H: device does not know what DOS is running.

 

 

 

Think it is/was XIO 39 vs. XIO 40 for binary load...?!?

(https://atariwiki.org/wiki/Wiki.jsp?page=XIO Command Codes DOS Handler)

 

[phaeron]

7 hours ago, Kaj de Vos said:

@phaeron, thanks for the deep insights!

 

I made sure to unmount the disk in the emulator before I used the Disk Explorer, and I didn't do any write operations in SpartaDOS in that session, so that should not have been the cause.

Unmounting the disk is not enough as the caching is in DOS and not in the disk drive. DOS does not even know that a disk change has occurred, it can only see changes in the disk contents, and SpartaDOS 1.x does not adequately check if the disk contents have changed before reusing cached directory and file structures. SpartaDOS 2.0 added the volume change serial number to fix this and the Disk Explorer increments that to avoid the problem on SDFS 2.0+ volumes.

 

7 hours ago, Kaj de Vos said:

Ah, indeed I saw some double implementations in MyDOS to be more compatible with SpartaDOS. It seems to me, then, that Atari800 implements the SpartaDOS interface, so it can work with SpartaDOS 3.2 and BW-DOS. But MyDOS just gets a file length, which is why the program isn't executed but MyDOS does not report an error.

It implements a binary load XIO call for Atari DOS format executables. Try copying a SpartaDOS 1.1 executable like MEMLO.COM to the H: drive and running it there and it will crash. This is because SpartaDOS's own binary loader can't be used with the H: device due to SpartaDOS's design.

 

MyDOS doesn't report an error because the H: device didn't report one, it succeeded at getting the file length.

 

1 hour ago, CharlieChaplin said:

 

Think it is/was XIO 39 vs. XIO 40 for binary load...?!?

(https://atariwiki.org/wiki/Wiki.jsp?page=XIO Command Codes DOS Handler)

 

Sorry, you're right, I messed up converting hex to decimal.

[Kaj de Vos]

Hello again!

 

Here is Hello World Next Generation:

 

https://language.meta.frl/examples/platforms/Atari/8-bit/HELLO.XEX
https://language.meta.frl/examples/platforms/Atari/8-bit/hello.list
https://language.meta.frl/examples/platforms/Atari/8-bit/hello.map

 

Here is the VICE labels file, to support @dmsc's emulator:

 

https://language.meta.frl/examples/platforms/Atari/8-bit/hello.labels

 

The compiler now analyses whether a program writes to standard output, and if it does, code is compiled in to detect at run-time whether it was called from a menu DOS or a command-line DOS. If it finds no command-line, it ends with a request to press a key and waits for that, to prevent the DOS menu or an emulator from overwriting the output prematurely.

 

This is the invisible code that is added:

also

    last program expression

    unless system/command-line? [

        print {

Press a key to end}

        input

    ]

 

INPUT is a shorthand for take system/ports/keyboard. This is a REBOL-style input/output abstraction.

system/ports is like CIO.

system/ports/keyboard is like K:.

As you can see, REBOL I/O abstractions map very nicely to the Atari OS. The REBOL setup is different, though; in some cases I'm only using the principles. REBOL doesn't have system/ports/keyboard. The model of the Atari OS is better, so I'm implementing that in REBOL style.

 

REBOL has a powerful SERIES! abstraction, that it applies to almost all data types. I/O is abstracted as PORT! data type, which is also a SERIES!. However, the SERIES! abstraction for PORT! is inconsistent and incomplete. In my language, I am improving these deficiencies in REBOL.

 

Also, most REBOL SERIES! are REBOL data types. PORT! is the exception, used to access external data, lazily. Other SERIES! are internal data, managed in REBOL memory. This is an expensive and complex approach, which makes REBOL a memory-hungry and slow language. In my language, any data type can be an abstraction over external data, leading to simple, razor-thin, fast interfaces.

 

Getting a key is implemented through conio.h. The needed function is unfortunately CC65 specific. The bad thing about this is that CC65 is incompatible with the Borland "standard", the good thing is that my language has now passed the mark of being more cross-platform than C.

 

The key press request was initially printed from column 1, while the "Hello, world!" is printed from column 3, respecting the left margin as expected. I'm using a variety of print functions to optimise for the situation, which led me to suspect at first that the lower-level functions were writing to S: instead of E:. Following the complex path of I/O initialisation showed that stdin, stdout and stderr are all opened to E:, as expected. It turned out that CC65 sets LMARGN to 0, while failing to correct the first printed line.

 

The margin is a user setting in the external environment, that a toolchain should not mess with, so I needed to get rid of this. I found it deep in CC65 in crt0.s. To change it, I needed to remove my CC65 installation and install it from source. The instructions for this have disappeared from the CC65 documentation. My language now requires a patched CC65.

 

It was a lot of work to improve compatibility and user-friendliness, but I am glad we worked through it. I thank everyone here for the help. It is important: it will improve the experience of all programs for everyone.

[sanny]

1 hour ago, Kaj de Vos said:

My language now requires a patched CC65.

 

Can you show a diff?

 

regards,
chris

[Kaj de Vos]

https://language.meta.frl/tools/CC65/Atari/8-bit/leave-margin.diff

[sanny]

Hmm.

 

You don't like that the indentation of 2 is reset to 0?

 

Is this important to be happening in the crt0.s startup code? You could always set it back to 2 in your program.

 

regards,
chris

[Kaj de Vos]

No, I don't like it. I expect a programming language to leave it alone unless I change it myself in my user program.

 

The Unix philosophy of C is most like command-line DOSes such as SpartaDOS. So I expect to be able to write programs that run seamlessly from the command line. The Atari OS already abstracts this with the E: device. I expect the user to set the margin to his or her liking, and command-line programs to respect it.

[ivop]

Hmm, setting the margins, setting cursor inhibit, etc..., all take effect after a character is printed. That's the Atari OS behaviour.

[sanny]

2 hours ago, Kaj de Vos said:

No, I don't like it. I expect a programming language to leave it alone unless I change it myself in my user program.

 

The Unix philosophy of C is most like command-line DOSes such as SpartaDOS. So I expect to be able to write programs that run seamlessly from the command line. The Atari OS already abstracts this with the E: device. I expect the user to set the margin to his or her liking, and command-line programs to respect it.

This is in the cc65 startup code since (I'd say) 20 years. You are the first one who doesn't like it. Sorry, I won't change it just for you.

 

But you can change LMARGN in your C code at the start of main(). I don't see a need to modify the cc65 library for that.2

 

regards,

chris

[Wrathchild]

From what I can vaguely recall from discussions^* around the time there would have been a consensus that this was the better option to have implemented.

 

Going down the path of making your own CC65 variant to distribute with your code seems overkill but seeing, for example, how many toolsets install their own version of Eclipse, its not uncommon.

 

I think I read the @dmsc was adding a build to Fast Basic distros but (I'll let him confirm) I believe that was in order to simply use a known/given release of CC65 with FB and not a variant?

 

^* the mail group archives are possibly online somewhere

[drac030]

9 hours ago, sanny said:

You could always set it back to 2 in your program.

And who said that the margin was 2 in the first place? Maybe it was set differently by an OS variant or by the user to own liking? The point, as I understand it, is to leave the setting untouched. If I want my left margin to be 1 and setup the system so, then I certainly do not want it to be 0 or 2.

[sanny]

7 hours ago, drac030 said:

And who said that the margin was 2 in the first place? Maybe it was set differently by an OS variant or by the user to own liking? The point, as I understand it, is to leave the setting untouched. If I want my left margin to be 1 and setup the system so, then I certainly do not want it to be 0 or 2.

 

This shouldn't be a problem. The startup code remembers the old setting. I just need to export that variable to C, so you can set it back.

[Kaj de Vos]

13 hours ago, sanny said:

But you can change LMARGN in your C code at the start of main(). I don't see a need to modify the cc65 library for that.

As @drac030 said, I should restore the original value, but I can't get at that because it is hidden in crt0.s, so I would have to change CC65, anyway. But that would generate code twice for something backwards, a hack upon a hack. The goal of the project is to approach an ideal situation, so I prefer to patch it out and save the space.

 

12 hours ago, Wrathchild said:

Going down the path of making your own CC65 variant to distribute with your code seems overkill but seeing, for example, how many toolsets install their own version of Eclipse, its not uncommon.

I will release the language as a remote client to a build server. This will prevent the problems of people needing to install compiler toolchains. Installing a toolchain can be quite difficult, especially for cross-compiling or when it needs to be patched. I expect that more toolchains for other platforms will need to be patched in the future.

[sanny]

I've exported the old value of LMARGN in the cc65 startup code. Use revision 200b42056216457a4f79097bf76195ef5bbb3da2 or later.

 

You can do it in C or use the following assembler version:

 

.include "atari.inc"
.import __LMARGN_save
.constructor reset_LMARGIN,20
.code
reset_LMARGIN:
lda __LMARGN_save
sta LMARGN
rts
.end

 

[ivop]

IMHO crt0.s should only setup the required parameters to support the CC65 API ABI. It should not mess with console settings, like its left margin or shift lock status.

 

Edit: I meant ABI, not API. API is the C standard.

Edited April 1, 2021 by ivop

[Kaj de Vos]

I agree. It reminds me of some changes that REBOL and Red make to their environment, which have always made it hard to use them to write common command-line programs.

[sanny]

42 minutes ago, ivop said:

IMHO crt0.s should only setup the required parameters to support the CC65 API. It should not mess with console settings, like its left margin or shift lock status.

 

This is done to be cross-platform compatible. Ideally, compile a program for C64 and Atari, and the output is the same.

[ivop]

4 minutes ago, sanny said:

This is done to be cross-platform compatible. Ideally, compile a program for C64 and Atari, and the output is the same.

I understand what you mean, but a cross-platform program in C just cannot rely on how the console "should" be. Imagine if every Linux command line utility would mess with my vt100/xterm settings. That's not acceptable. If your program needs shift lock pressed or unpressed, you need to do that in your program (like Pokey Explorer does). If you need printing to E: with a left margin of 0, you set that in your program (midimon does that). A compiler/library combination should not do that behind your back. And neither should a compiler add hidden code to detect whether you were called from a command line DOS or not. That's up to the program itself.

 

[sanny]

6 minutes ago, ivop said:

I understand what you mean, but a cross-platform program in C just cannot rely on how the console "should" be. Imagine if every Linux command line utility would mess with my vt100/xterm settings. That's not acceptable. If your program needs shift lock pressed or unpressed, you need to do that in your program (like Pokey Explorer does). If you need printing to E: with a left margin of 0, you set that in your program (midimon does that). A compiler/library combination should not do that behind your back.

 

I see and in general agree with you. But changing this now changes behaviour as it was for the last 20 years. It's difficult. I don't want to piss off people because their program suddenly behaves different.

 

6 minutes ago, ivop said:

And neither should a compiler add hidden code to detect whether you were called from a command line DOS or not. That's up to the program itself.

 

 

I disagree. If there are command line arguments I want to have them in argc/argv and don't care how they ended up there,