A collection of Atari BASIC benchmarks

[drpeter]

4 minutes ago, Faicuai said:

This is important because, in the case of Bench64 (my specific contribution to the repo), all bets were thrown off as soon as its original author formally included the MinZ configuration and operating environment as a real contender.

Yeah, but bear in mind the the purpose of Bench64, as far as one can divine it (the relative speed of various 8-bit systems in running a strangely arbitrary subset of CBM BASIC 2- how many BASIC programs include calculating arc-tangents?) is entirely different to that of @Maury Markowitz which is to do with comparing BASIC implementations on the Atari 8-bit systems specifically. As is evident from the point that the author of Bench64 states Atari BASIC is specifically excluded from his definition of the benchmark as it can't run the required subset of CBM BASIC 2. Redefining the benchmark so that something similar can be run in Atari BASIC doesn't really get around that, if that's how the benchmark is- like all benchmarks- arbitrarily defined. That doesn't mean Atari BASIC is defective. Few would say that CBM BASIC 2 is anything other than a dog, particularly for a BASIC shipped as stock alongside hardware very capable in both graphics & sound departments. It does reflect that finding sensible cross-platform performance comparators is a difficult- some would say impossibly pointless- exercise.

[Maury Markowitz]

Just now, drpeter said:

Few would say that CBM BASIC 2 is anything other than a dog

It is not clear what "dog" refers to in this case. Normally that would refer to performance, as in "dog slow", but your following list is mostly feature related.

 

If we are talking about performance, there's really nothing dog-like about it. As it runs basically the same underlying code as practically every other platform of the era (Atari and TI-99 being the obvious exceptions), I would not say "dog" but instead "baseline". One could also say that applies to the features. To be dog slow, one would have to run much slower than MS.

 

I recall when I first started reading the original 6502 MS code. I understood how BASICs worked from descriptions of Atari found in magazines, so I simply assumed all BASICs worked that way. When I read the MS code I was gobsmacked - it doesn't tokenize anything other than keywords? It re-parses constants every time through the code? It leaves things in a weird mash of pre-interpreted and original source code? What madness is this?!

 

The fact that MS has to, for instance, re-parse the "100" in "GOTO 100" every time through a loop and that it still runs 2 to 3x faster than  Atari BASIC is downright breathtaking. Now AB... that is the definition of "dog slow".

[drpeter]

39 minutes ago, Maury Markowitz said:

Normally that would refer to performance, as in "dog slow"

Sorry, was not very specific was I? ?

 

I meant totally inappropriate to its standard operating environment, i.e. lack of features to support graphics and sound programming, or input controllers come to that. IMHO. As others have suggested, more appropriate to a PET than a C64.

Edited June 11, 2021 by drpeter

[Faicuai]

26 minutes ago, drpeter said:

is entirely different to that of @Maury Markowitz which is to do with comparing BASIC implementations on the Atari 8-bit systems specifically

Yes, you are right. The only additional point  would be knowing the reason for pre-setting any particular CPU+GPU mix, which on the A8 is always our choice. It really helps knowing up-front what motivates it.

 

26 minutes ago, drpeter said:

Redefining the benchmark so that something similar can be run in Atari BASIC doesn't really get around that

Well, Bench64 author declared that "any feature [NOT INCLUDED] in CBM basic will NOT be tested".

 

It should be noted that the current  A8 version of Bench64 essentially runs in CBM basic as is, except for for the typical dialect-translation (string indexing) in line 960, IIRC. 

 

This was reason enough (for me) to believe that Atari-basic exclusion of this test was borderline arbitrary. It can perfectly accomplish the intended tasks, although with a performance penalty on emulation of DEF FN cases. That's about it.

[drpeter]

13 minutes ago, Faicuai said:

emulation of DEF FN

This is what the author (rightly or wrongly) decided excluded Atari BASIC. One could argue whether that ruling was necessary. But given the vagueness of the defined purpose of this benchmark in the first place, it would I think be pointless.

[drpeter]

26 minutes ago, Faicuai said:

It really helps knowing up-front what motivates it.

Exactly.

 

If the purpose is to compare relative speed of different BASICs on the Atari like for like, there is little point is turning DMA off. The relative gain will be identical in every case.

 

If the purpose is to see how quickly a given 8-bit CPU-system can calculate 50 bitcoin hashes in BASIC without resorting to machine language but otherwise by any means 'fair or foul', optimising the BASIC code as much as you like, that's something else entirely.

 

Or, perhaps more fun and relevant, how many fps can be achieved in Star Raiders space combat...

Edited June 11, 2021 by drpeter

[drpeter]

40 minutes ago, Maury Markowitz said:

Atari and TI-99 being the obvious exceptions

And BBC

[Faicuai]

4 minutes ago, drpeter said:

If the purpose is to compare relative speed of different BASICs on the Atari like for like, there is little point is turning DMA off. The relative gain will be identical in every case.

Absolutely, no argument there.

 

That would be unquestionable and irreductible.

[drpeter]

1 hour ago, Maury Markowitz said:

The fact that MS has to, for instance, re-parse the "100" in "GOTO 100" every time through a loop and that it still runs 2 to 3x faster than  Atari BASIC is downright breathtaking. Now AB... that is the definition of "dog slow".

I'm pretty sure you've read these,  but for others interested in the fascinating trade-offs involved in cramming BASIC into 10K when working to a short deadline, and specifically those involved in developing Atari BASIC, the series of Insight Atari articles by Bill Wilkinson provide exactly that.

Edited June 11, 2021 by drpeter

[Maury Markowitz]

3 hours ago, drpeter said:

This is what the author (rightly or wrongly) decided excluded Atari BASIC. One could argue whether that ruling was necessary. But given the vagueness of the defined purpose of this benchmark in the first place, it would I think be pointless.

It was because of the DEF FNs. He had no interest in trying to code around versions that lacked it. I don't think it's because he wanted to test DEFs that badly, but didn't have any interest in learning new dialects.

2 hours ago, drpeter said:

the fascinating trade-offs involved in cramming BASIC into 10K when working to a short deadline

I don't think the short deadline or size constraints fully explains it. One of the biggest problems in AB is the FOR/NEXT loops, which write down a 16-bit line number instead of a 16-bit address, and then calls the (slow) line-lookup macro every time through the loop. It is exactly the same code in the end, you simply have to move one macro call from A to B in the code. According to his later articles, this was done deliberately so you could CONT after breaking in a loop, but honestly, that always sounded like a post-facto explanation.

 

What's amazing about that particular issue is that when I was in university my computer languages book had a list of 10 commandments on the inside cover. Among them was "distributed costs", which was essentially don't make a statement do something general if the 99% use is very specific. This is precisely what AB did - by adding a feature that might be used 1 in 1000 times, they slowed down every program every time it ran.

 

One can give them a bye on the GOTO issue, as that would require more code, but the FOR/NEXT is simply a wrong decision that was never fixed.

Edited June 11, 2021 by Maury Markowitz

[drpeter]

24 minutes ago, Maury Markowitz said:

According to his later articles, this was done deliberately so you could CONT after breaking in a loop, but honestly, that always sounded like a post-facto explanation.

 

Yeah. I didn't really buy that 'it's not a bug, it's a feature' pleading either.  To be fair, he didn't quite go so far as to openly state it was a deliberate & well-thought-through design decision.

 

Although claiming this as a 'feature' remains slightly more plausible than celebrating CBM BASIC 2.0's drawbacks (as some have done) as no bad thing since they encourage programming in assembly language instead....

[Mazzspeed]

1 hour ago, drpeter said:

 

Yeah. I didn't really buy that 'it's not a bug, it's a feature' pleading either.  To be fair, he didn't quite go so far as to openly state it was a deliberate & well-thought-through design decision.

 

Although claiming this as a 'feature' remains slightly more plausible than celebrating CBM BASIC 2.0's drawbacks (as some have done) as no bad thing since they encourage programming in assembly language instead....

I remember the very first time I ported a simple bit of code from BASIC to assembly to poke the screen/screen boarder color registers in order to change the color of the screen in succession, and saw for the first time ever just how fast it ran, I never went back to BASIC - It was literally about 200x faster and CBM BASIC was actually anything but slow as far as interpreters are concerned (yes, it wasn't the fastest either).

 

From that point on, the lack of graphics and sound support under CBM BASIC wasn't even a consideration. Furthermore, both Commodore and Atari relied on DOS wedges loaded off disk to provide for more disk commands.

 

Nowadays, most coding on C64 is done via software such as CBM Studio on PC and ported directly to the memory of a real C64 via Ethernet and the C64's unique DMA feature using CodeNet, which must have it's advantages as there is quality software being released for the C64 almost daily - Although some of us still like using assemblers on real hardware, and for that purpose Turbo Macro Pro running from the REU to allow for maximum memory space when banking everything out but IO is fantastic.

 

I mentioned earlier on the issue was a result of DEF/FN, as the C64 is used as the basis for the benchmark and it has been decided by the author that everything therefore much conform to CBM BASIC in order for the software to produce meaningful results, but certain individuals probably didn't read that. As stated, tweaks (including DMA off) aren't allowed and that includes BBC owners regarding tweaks of their own.

 

I am purely impartial. I own a what I consider to be a fully expanded A8 and a fully expanded C64 and I enjoy the merits of both as neither on their own is the ideal 8bit machine - However, in using both machines daily as fully expanded devices I can see a number of advantages to the C64's newer design, and that's not to detract from the design of the A8 as it is very impressive considering it's age - It's a shame it never saw any substantial upgrades beyond increased memory and more colors. I read a lot of negativity on these forums surrounding the C64 that is outright rubbish, which is a shame as this isn't an Atari BBS or the schoolyard, the days of C64 vs A8 should be well behind us. The user group I'm a part of meets once a month as a Commodore user group, we also have Atari users as part of our group and we never get into anything even remotely like Commodore vs Atari discussion, we crowd around and see just what such devices are capable of with outright interest and nothing more.

 

There was never anything stopping anyone from tweaking the situations surrounding the benchmark based purely on curiosity, if anything I usually find such experiments very interesting. But it has to be understood that such experimentation is proof of concept only and invalid in the context of the benchmark. It's not difficult to understand, nor is it worth pages and pages of round and round rubbish in blind defense of one platform.

 

Edited June 11, 2021 by Mazzspeed

[Faicuai]

 

Time now for updating and reviewing our thread's pivotal stats (AA-search facility):

 

"Atari": (64 results)

 

Dr. Peter:                     7

JKK:                            1

Maury:                         3

(non-contributor) :        2 [did acknowledge including all DMA on/off results]

Thor:                           3

The Doctor:                  9

(Mazz-C64):                 19 [Contributions to repository (main-topic): ZERO]

Faicuai:                        20 

 

"C64": (36 results)

 

Dr. Peter:                     3

Faicuai:                        11

(Mazz-C64):                 22 [Contributions to reposotry (main-topic): ZERO]

 

The original trend remains intact.  No surprises. 

 

 

[Mazzspeed]

20 minutes ago, Faicuai said:

 

Time now for updating and reviewing our thread's pivotal stats (AA-search facility):

 

 

"Atari": (64 results)

 

Dr. Peter:                     7

JKK:                            1

Maury:                         3

(non-contributor) :        2 [did acknowledge including all DMA on/off results]

Thor:                           3

The Doctor:                  9

(Mazz-C64):                 19 [Contributions to repository (main-topic): ZERO]

Faicuai:                        20 

 

"C64": (36 results)

 

Dr. Peter:                     3

Faicuai:                        11

(Mazz-C64):                 22 [Contributions to reposotry (main-topic): ZERO]

 

The original trend remains intact.  No surprises. 

 

 

What, exactly, are you hoping these statistics you dig up prove? You harp on about 'contributions to repository' and as evidenced by the above post you're the one not adding anything of value to this thread.

 

Because what I see in relation to my own statistics (19 vs 22) is a fairly balanced perspective, what I see regarding your statistics (20 vs 11 in favor of Atari) is an expected biased perspective. As always, your posts are actually working against you.

 

You're acting like a child and it's becoming tiring.

Edited June 11, 2021 by Mazzspeed

[drpeter]

1 hour ago, Mazzspeed said:

It's a shame it never saw any substantial upgrades beyond increased memory

That's definitely something to agree on.  Many would still take an 800 over any of the later models, which were predominantly exercises in reducing manufacturing costs.

[drpeter]

1 hour ago, Mazzspeed said:

and more colors

I don't think this is right. All models had a palette of 128 colours (NTSC) or 120 colours (PAL).  Very early 800s/400s produced early in 1980 for the US market had the CTIA chip, which lacked the 4-bit-per-pixel 80x192 modes of all later GTIA chip models... is that what you were thinking of?

[Mazzspeed]

15 minutes ago, drpeter said:

That's definitely something to agree on.  Many would still take an 800 over any of the later models, which were predominantly exercises in reducing manufacturing costs.

I'm not interested in the 800 TBH, the XL range better suits my needs. Having said that, there's no doubting the 800 was built like a tank - Unfortunately that also meant it was expensive and therefore classed as a luxury many couldn't justify, especially outside the US. I have no issue with the build quality of my XL or my C64.

 

As far as cost cutting regarding latter models, such tactics were bound to happen as manufacturers realized what was needed to push computing to the masses. It doesn't change the fact that the architecture of the C64 is actually quite impressive (considering 1982) once you strip away all the Atari bias. The Atari way isn't always the best way.

Edited June 11, 2021 by Mazzspeed

[Mazzspeed]

6 minutes ago, drpeter said:

I don't think this is right. All models had a palette of 128 colours (NTSC) or 120 colours (PAL).  Very early 800s/400s produced early in 1980 for the US market had the CTIA chip, which lacked the 4-bit-per-pixel 80x192 modes of all later GTIA chip models... is that what you were thinking of?

CTIA had only 8 luminance values, GTIA had 16 luminance values. Therefore CTIA was limited to a 128 color palette while GTIA supported a 256 colour palette considering NTSC.

[drpeter]

6 minutes ago, Mazzspeed said:

The Atari way isn't always the best way.

You misunderstand me. I meant any of the later Atari 8-bit models. Not that there's anything wrong with the XL series. FCC regulations of the time were responsible for a lot of the costly build design decisions of the original 400/800 series.

[Mazzspeed]

1 minute ago, drpeter said:

You misunderstand me. I meant any of the later Atari 8-bit models. Not that there's anything wrong with the XL series. FCC regulations of the time were responsible for a lot of the costly build design decisions of the original 400/800 series.

I know what you were implying.

 

I own both an A8 and a C64, certain judgements regarding the latter are but one rubbish point made by certain members on this forum. In all honesty, this forum is a fairly toxic place, even for Atari biased users.

[drpeter]

5 minutes ago, Mazzspeed said:

GTIA supported a 256 colour palette considering NTSC.

Ah. That's true for the relatively little-used (except in demos) 1 hue/16 luminance GTIA-specific mode. The rest were 120/128 colours as for CTIA.

[Faicuai]

 

1 hour ago, Mazzspeed said:

What, exactly, are you hoping these statistics you dig up prove?

 

1 hour ago, Mazzspeed said:

you're the one not adding anything of value to this thread.

 

1 hour ago, Mazzspeed said:

Because what I see in relation to my own statistics (19 vs 22)

 

Bingo!!

 

Not just the stats, but your own reaction when confronting your own factual trail:

 

1. You come up on top with 41 posts (now 42), topping "C64" mentions on this thread, by far?

 

2. You have contributed with NOTHING to the repository... A repository of ATARI benchmarks (!)

 

3. You just posted two (2) additional,  unsolicited C64 adds, in the span of the last few minutes....

 

4. You not only called my a child in the last post, and also a non-contributor. when I actually provided Bench64 ported to Atari, plus plenty of supporting material.

 

 

Do you realize what you are doing, and that you simply can't (and won't) help yourself? That your problem is you simply don't know when to stop even when confronting your own factual trail?

 

I will keep tracking the numbers.... let's see how far they go...

 

[Mazzspeed]

I'm not a member of any Commodore forum, I usually frequent certain general retro computing forums where everyone loves anything retro computing related and participates in relative harmony. What I see here is a bunch of apparent coders and developers (excluding a minority that have vast wealth's of real technical knowledge to offer the community and are open minded) that all garble on about how the A8 is the best at literally everything, with little knowledge on just what it is they're talking about - Everyone says C64 titles can be ported to the A8 no problem, no one does it! Anyone would assume that's because it isn't possible.

 

This place is outright toxic, there's Atari users attacking other Atari users, there's too many 'developers' with their developer attitude and not enough open minded enthusiasts. Furthermore, it's a shame, as it could be a really good community if everyone could just lift that huge chip off their shoulder.

Edited June 11, 2021 by Mazzspeed

[Mazzspeed]

3 minutes ago, Faicuai said:

 

 

 

 

Bingo!!

 

Not just the stats, but your own reaction when confronting your own factual trail:

 

1. You come up on top with 41 posts (now 42)

 

2. You have contributed with NOTHING to the repository...

 

3. You just posted two (2) additional,  unsolicited C64 adds, in the span of the last few minutes....

 

4. You not only called my a child in the last post, and also a non-contributor. when I actually provided Bench64 ported to Atari.

 

 

Do you realize what you are doing, and that you simply can't (and won't) help yourself?

 

 

 

 

 

 

 

Shut up. You talk rubbish.

[Faicuai]

7 minutes ago, Mazzspeed said:

Shut up. You talk rubbish.

 

?

 

And then cries because the place "toxic"...

 

Yeah, right.... ?