Ahl's Benchmark?

[+Larry]

David Ahl, published his benchmark in the November, 1983 issue of his Creative Computing magazine. (The first computer magazine I ever subscribed to.) That text article is posted at the Classic Computer Magazine Archive (November, 1983 issue):

http://www.atarimagazines.com/creative/v9n...arison_test.php

 

But the actual benchmark and the comparison times are not included. The benchmark is posted in several places on the internet, but I can't find the original times. Does anyone have the original article (or link) with the comparison times?

 

-Larry

Edited November 29, 2007 by Larry

[JamesD]

David Ahl, published his benchmark in the November, 1983 issue of his Creative Computing magazine. (The first computer magazine I ever subscribed to.) That text article article is posted at the Classic Computer Magazine Archive (November, 1983 issue):

http://www.atarimagazines.com/creative/v9n...arison_test.php

 

But the actual benchmark and the comparison times are not included. The benchmark is posted in several places on the internet, but I can't find the original times. Does anyone have the original article (or link) with the comparison times?

 

-Larry

I've been looking for the code for the benchmark for some time. If you have a link, please post it.

<edit>

Never mind... it didn't turn up on google last time I searched but now it's in several places.

 

http://www.floodgap.com/retrotech/mac/ahl/imp.html

http://setiathome.berkeley.edu/~korpela/ahl/basic.html

http://www3.sympatico.ca/maury/other_stuff/atari_basic.html

Edited November 22, 2007 by JamesD

[+Larry]

I've been looking for the code for the benchmark for some time. If you have a link, please post it.

 

If you Google "Ahl's Benchmark" you'll find several listings. Note it included as a sidebar in the article "Atari Basic -- The Good, the Bad, and the Ugly"

 

http://www3.sympatico.ca/maury/other_stuff/atari_basic.html

 

Some listings require very minor changes to run under Atari Basic.

 

-Larry

[Rybags]

Like the article says, Atari BASIC was/is embarassingly slow.

 

Mainly thanks to half-baked FP ROM, and inefficient line lookup.

 

In the day, I can remember it usually being between 3rd last to last in the common benchmarks.

 

 

Although, if we had something like "MadOnion 2DMark 1985", it would have crushed all comers.

[JamesD]

I looked all over the place and couldn't find the benchmark results. I thought it was actually a scan with the article so either the scan was removed from that page or whatever page it was on before is now gone.

[+Larry]

Like the article says, Atari BASIC was/is embarassingly slow.

 

Mainly thanks to half-baked FP ROM, and inefficient line lookup.

 

In the day, I can remember it usually being between 3rd last to last in the common benchmarks.

 

 

Although, if we had something like "MadOnion 2DMark 1985", it would have crushed all comers.

 

What kind of benchmark (?) was the "MadOnion 2D Mark 1985?"

 

-Larry

[+Larry]

I looked all over the place and couldn't find the benchmark results. I thought it was actually a scan with the article so either the scan was removed from that page or whatever page it was on before is now gone.

 

I'm speculating, but I'll guess that it was left out through a "zoning" error when the article was OCR'd. I've contacted Kevin to see if by chance he has the original article. IIRC, there were several updates to this article in later issues, but I *do* remember the times being published at some point.

-Larry

[JamesD]

We could just run the benchmark on as many machines as we can and post the results here.

 

If we do that... NO OPTIMIZATIONS!

The loops are spread over several lines intentionally.

[Rybags]

What kind of benchmark (?) was the "MadOnion 2D Mark 1985?"

 

-Larry

 

 

It wasn't one.

 

MadOnion is the old name of Futuremark.

 

They do PC Benchmarks, the famous ones starting with 3DMark (99, 2000, 01, 03, 05 and 06).

[JamesD]

Well, I haven't found anyone with the article yet but I have 1 benchmark that wasn't on the original list.

The Tandy Color Computer 3 in high speed mode:

time 1 min 12 sec

accuracy 5.96284867E-04

random 7.3876276 test1, 24.5601945 test2, 9.40813446 test3

 

Reruns yielded: 1 min 11 sec.

With the fast keyboard routine enabled: 1 min 9 sec

 

Random should *not* be evenly distributed (even = too random to really be random) and on TRS-80's you can see the random number generator for better results with:

A=RND(-TIMER)

 

Not bad results for a Microsoft version of BASIC.

I'm wondering what the faster Atari BASIC replacements managed.

Edited November 27, 2007 by JamesD

[+Larry]

Well, I haven't found anyone with the article yet but I have 1 benchmark that wasn't on the original list.

The Tandy Color Computer 3 in high speed mode:

time 1 min 12 sec

accuracy 5.96284867E-04

random 7.3876276 test1, 24.5601945 test2, 9.40813446 test3

 

Reruns yielded: 1 min 11 sec.

With the fast keyboard routine enabled: 1 min 9 sec

 

Random should *not* be evenly distributed (even = too random to really be random) and on TRS-80's you can see the random number generator for better results with:

A=RND(-TIMER)

 

Not bad results for a Microsoft version of BASIC.

I'm wondering what the faster Atari BASIC replacements managed.

 

Here are some other results:

 

Atari 8K Basic Time=405 sec. (!) Accuracy=.013959 Random=16.77, 8.80

 

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33

 

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

 

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9

 

Atari Microsoft Basic Time=101.4 Accuracy .150879 Random=2.06506, 2.06506 (5.60 when RANDOMIZEd)

 

Notes:

The FAST mode of BXL and BXE made essentially no difference in the times, presumably since the line numbers are low and very close together.

Atari Microsoft Basic would not run properly as the benchmark is written. RND(1) gives a random number between 1 and 1. RND(0) appears to work, but gives the same results each time. A RANDOMIZE statement must be added to "seed" the random number generator. The RANDOMIZE value can be taken from POKEY to provide a "random" seed from 0-255 IIRC. Also, the times for AMSB are single precision floating point (not integers).

As usual, TBXL is fastest, with BXE a pretty close second. And what can we say about the speed of Atari Basic...

 

Can anyone run this on the C64 for comparison?

 

-Larry

Edited November 29, 2007 by Larry

[Urchlay]

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

 

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9

 

Wow... I didn't think BXL and BXE were that different, would not have expected the huge speed increase in BXE.

 

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33

 

Is this interpreted or compiled Turbo BASIC?

[+Larry]

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

 

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9

 

Wow... I didn't think BXL and BXE were that different, would not have expected the huge speed increase in BXE.

 

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33

 

Is this interpreted or compiled Turbo BASIC?

 

Hi Urchlay-

 

I think this benchmark is all about the math package. BXL uses the regular Atari FP routines. BXE (with the extensions loaded) uses its own "fast math" routines.

I probably ought to run BXE without its extensions to compare that.

 

This was interpreted TBXL. I probably should run the compiler, also, but I suspect there would not be much difference.

 

-Larry

[JamesD]

The CoCo 3 with a 6309 running in native mode and with no 6309 BASIC patches ran it in 56 seconds. Not bad for an unoptimized Microsloth BASIC.

 

BTW, the fastest machine that had been benchmarked in the article was the Toshiba T100 with a time of 1:09. The T100 is based on a 4MHz Z80.

Link

<edit>

BTW, a 64180 (Z180) upgrade to the T100 should give about the same # as the 6309.

Edited November 29, 2007 by JamesD

[JamesD]

Here are some other results:

 

Atari 8K Basic Time=405 sec. (!) Accuracy=.013959 Random=16.77, 8.80

 

Turbo Basic XL Time=41.6 Accuracy=.013649 Random=12.29, 17.33

 

Basic XL Time=395.88 (!) Accuracy=.013959 Random= 7.95, 2.45

 

Basic XE Time=49.7 Accuracy=.014614 Random= 17.58, 16.9

 

Atari Microsoft Basic Time=101.4 Accuracy .150879 Random=2.06506, 2.06506 (5.60 when RANDOMIZEd)

I just noticed that the Accuracy is lower than other machines.

[JamesD]

The Apple IIc Plus completes this in around 31 seconds. I don't have a clock timer on my machine to be more exact.

[+Larry]

The Apple IIc Plus completes this in around 31 seconds. I don't have a clock timer on my machine to be more exact.

 

The IIC Plus was 4 MHz machine, wasn't it? I also wonder what a 65C816 cpu would do with its improved floating point facilities? And likewise a C64 with a SuperCPU?

Or (shudder) a modern PC @3 GHz? It probably would be over before it started...

 

-Larry

[Rybags]

The QBASIC language is on the Win98 install CD.

 

Pretty sure it works with Windoze XP.

 

10 rem Ahl's simple benchmark

20 for n = 1 to 100: a = n

30 for i = 1 to 10

40 a = sqr(a): r = r + rnd(1)

50 next i

60 for i = 1 to 10

70 a = a^2: r = r + rnd(1)

80 next i

90 s = s + a: next n

100 print "Accuracy ";abs (1010-s/5)

110 print "Random ";abs (1000-r)

 

Adapted for QBASIC, with an extra enclosing loop to run the whole thing 100 times (without the printing), does it in about 2 seconds.

Edited November 29, 2007 by Rybags

[JamesD]

The Apple IIc Plus completes this in around 31 seconds. I don't have a clock timer on my machine to be more exact.

 

The IIC Plus was 4 MHz machine, wasn't it? I also wonder what a 65C816 cpu would do with its improved floating point facilities? And likewise a C64 with a SuperCPU?

Or (shudder) a modern PC @3 GHz? It probably would be over before it started...

 

-Larry

It had a 4MHz clock but used cache like the Zip Chip if I remember right so it wouldn't be quite as fast as a full 4MHz machine. Still, it's probably the fastest 6502 based system that was made unless a Laser 128EX2 or Franklin can outrun it.

 

Applesoft Basic was never updated to support the 65816 so a standard speed IIgs will be slower than the IIc Plus. With one of the CPU upgrades it would be faster.

The SuperCPU (?) for the C64 running at 20MHz should be the fastest 65816 unless one of the IIgs upgrades was available at that clock speed.

[JamesD]

Well, using BASIC O9 under OS-9 on the CoCo 3 it runs in 6 seconds.

But the benchmark was only intended to compare built in BASICs, not compiled BASIC.

Still, it gives you an idea of what difference integers and not having to look up line numbers makes.

[JamesD]

The Commodore Plus/4 (PAL?) had these results:

 

Accuracy 1.04141235-03

Random 11.1208959

Time 1:49

 

Note: Time was calculated AFTER printing the rest of the results

[+Larry]

Well, using BASIC O9 under OS-9 on the CoCo 3 it runs in 6 seconds.

But the benchmark was only intended to compare built in BASICs, not compiled BASIC.

Still, it gives you an idea of what difference integers and not having to look up line numbers makes.

 

Did you revise the benchmark coding to account for integers? I tried Atari Microsoft Basic defining integers as N,I,S. But it had little effect on the times. I presume the accuracy would not be meaningful using integers.

 

-Larry

[Urchlay]

Did you revise the benchmark coding to account for integers? I tried Atari Microsoft Basic defining integers as N,I,S. But it had little effect on the times. I presume the accuracy would not be meaningful using integers.

 

I remember reading somewhere that Commodore 64 BASIC's integers are actually slower than its floating point, because the integers have to get converted back to FP any time they're used in calculations. Maybe Atari MS BASIC suffers the same problem?

 

(Being slower doesn't make them useless: integer variables still use less RAM than floating point variables)

[JamesD]

Well, using BASIC O9 under OS-9 on the CoCo 3 it runs in 6 seconds.

But the benchmark was only intended to compare built in BASICs, not compiled BASIC.

Still, it gives you an idea of what difference integers and not having to look up line numbers makes.

 

Did you revise the benchmark coding to account for integers? I tried Atari Microsoft Basic defining integers as N,I,S. But it had little effect on the times. I presume the accuracy would not be meaningful using integers.

 

-Larry

Someone over on CoCo3.com ran it. Revising the benchmark has good and bad points.

Technically, you aren't doing the same thing. But on the other hand, it shows what you can do in the real world.

<edit> Ahl himself may have modified some versions to use integers. We don't know since he didn't post every version.

Accuracy = 9.45091248E-04

That's less than regular MS BASIC but still better than the Atari.

 

BTW, the ABS() has limitations that required the use of the if/then/else.

 DIM i,n:INTEGER
s:=0 \ r:=0
a:=RND(-VAL(RIGHT$(DATE$,2))))
PRINT DATE$
FOR n=1 TO 100 \a:=n
  FOR i=1 TO 10
   a:=SQ(a) \r:=r+RND(0)
  NEXT i
  FOR i=1 TO 10
  a:=SQ(a) \r:=r+RND(0)
  NEXT i
  s:=s+a
NEXT n
PRINT "Accuracy ";
IF s<5050 THEN
PRINT 1010-s/5
ELSE
PRINT s/5-1010
ENDIF
PRINT "Random "; ABS(1000-r)
PRINT DATE$

Edited December 1, 2007 by JamesD

[+Larry]

A small update:

 

I compiled the benchmark with the Turbo Basic Compiler, and it runs in 33.1 seconds -- nearly 20% faster.

 

-Larry