What is a good benchmark for comparing the speed of languages?

+Larry · July 27, 2015

I want to do a benchmark that shows the speed of Action! versus compiled Turbo Basic XL. I also plan to show the added speed available from an accelerator, but that is easier to compare. There are many benchmarks, but some really don't show much about a language. For instance, Ahl's Simple Benchmark mostly shows the speed of the math pack. Benchmarks that mostly just write to the screen in loops largely show the speed of the E: or S: devices.

What are some other benchmarks that might better show the inherent speed of Action!?

Any ideas?

Thanks,

Larry

tebe · July 27, 2015

ACTION!

; ERATOSTHENES SIEVE BENCHMARK
 
MODULE
 
BYTE SIZE=[255]
 
CARD SIZEPL=[256]
 
BYTE ITER_MAX=[10]
 
;VAR:
 
BYTE ARRAY FLAGS(SIZEPL)
 
BYTE RTCLOK=20, RTP
, ITER, A
CARD I,K,COUNT
,PRIME
 
 
PROC BEGIN()
PRINT("ITERATIONS: ")
PRINTBE(ITER_MAX)
 
 
A=RTCLOK
DO
IF A=RTCLOK THEN EXIT FI
OD
 
RTCLOK=0
POKE(19,0)
POKE(18,0)
 
ITER=1
 
WHILE ITER<=ITER_MAX
DO
 
 COUNT = 0 
 I = 0 
 
 WHILE  I <= SIZE  DO
     FLAGS(I) = 1 
     I==+1
 OD
 
 I  = 0 
 WHILE  I <= SIZE  DO
 
  IF FLAGS(I)=1 THEN
 
        PRIME  =(I LSH 1)+ 3 
        K  = I + PRIME 
 
        WHILE  K <= SIZE  DO
          FLAGS(K)  = 0 
          K==+PRIME 
        OD
        COUNT==+1
  FI
  I==+1
  OD  
 
  ITER==+1
 
 OD
 
RTP=RTCLOK
PRINT("PRIMES: ") PRINTCE(COUNT)
 
PRINT("FPS: ") PRINTBE(RTP)
 
[96]

Mad Pascal

// Eratosthenes Sieve benchmark
 
const
 size = 255;
 sizepl = 256;
 
 iter_max = 10;
 
var 
  flags: array [0..sizepl] of boolean;
 
  iter: byte;
 
  i,k,count: word;
 
  prime: word;
 
 
function time: byte;
begin
asm
{ mva rtclok+2 Result
};
 
end;
 
 
begin
writeln(iter_max,' iterations');
 
asm
{
lda:cmp:req rtclok+2 ; reset clock
lda #0
sta rtclok
sta rtclok+1
sta rtclok+2
};
 
iter := 1;
while (iter <= iter_max) do begin
 
count := 0;
i := 0;
 
while (i <= size) do begin
flags[i] := true;
inc(i);
end;
 
i := 0;
while (i <= size) do begin
 
if flags[i] then begin
 
prime := i shl 1 + 3;
k := i + prime;
 
while (k <= size) do begin
flags[k] := false;
k := k + prime;
end;
inc(count);
end;
inc(i);
end;
 
inc(iter);
end;
 
 writeln(count, ' primes');
 writeln(time, ' fps');
 
 repeat until keypressed;
 
end.

ACTION! is 3x faster then Mad Pascal

Edited July 27, 2015 by tebe

GoodByteXL · July 27, 2015

I want to do a benchmark that shows the speed of Action! versus compiled Turbo Basic XL. I also plan to show the added speed available from an accelerator, but that is easier to compare. There are many benchmarks, but some really don't show much about a language. For instance, Ahl's Simple Benchmark mostly shows the speed of the math pack. Benchmarks that mostly just write to the screen in loops largely show the speed of the E: or S: devices.

What are some other benchmarks that might better show the inherent speed of Action!?

Any ideas?

Thanks,

Larry

A good start is the article about it in BYTE magazin, Sept. 1981: A High-Level Language Benchmark.

As far as I remember back then mostly the ERATOSTHENES SIEVE BENCHMARK was standard.

If you look at Draco's SysInfo maybe you get some ideas how to put such things to a high level language.

fujidude · July 27, 2015

The Action! language is very closely designed around the particulars of the 6502 specifically, so the object code it makes is often tighter, faster, and more efficient than other high level languages that compile for the 6502. On the other hand, Action does not even natively support real (floating point) numbers. There is a library to add the capability, but it is a bit unwieldy.

A lot of software really doesn't require real number support though, and unwieldy or not, if you do need it you can "bolt it on." In all, Action is a great language for 6502 based machines and is the one I always hope to see get the most attention and effort in enhancing. A cross (or native) compiler that could output object code directly to disk would be wicked for example. There is... anyway I'm digressing. For a lot of tasks, you will be hard pressed to beat Action! for speed of high level language.

Edited July 27, 2015 by fujidude

fujidude · July 27, 2015

If you look at Draco's SysInfo maybe you get some ideas how to put such things to a high level language.

Do you mean the "MACH" command?

GoodByteXL · July 28, 2015

Do you mean the "MACH" command?

No, not the external SDX command from the current SDX Toolkit but the test suite SysInfo -> http://drac030.krap.pl/en-sparta-pliki.php

JamesD · July 28, 2015

I want to do a benchmark that shows the speed of Action! versus compiled Turbo Basic XL. I also plan to show the added speed available from an accelerator, but that is easier to compare. There are many benchmarks, but some really don't show much about a language. For instance, Ahl's Simple Benchmark mostly shows the speed of the math pack. Benchmarks that mostly just write to the screen in loops largely show the speed of the E: or S: devices.

...

Actually, Ahl's benchmark was also designed to do a lot of looping because that is a performance issue with some BASIC interpreters.

But looping is kind of a given for benchmarks in general.

The biggest problem with Ahl's benchmark is that he ignored string handling entirely.

Microsoft BASIC wasn't the fastest one out there but it did have excellent string handling.

And when it came to formatted numerical output, a BASIC with PRINT USING required a lot less code.

...

ACTION! is 3x faster then Mad Pascal

On that benchmark. The difference may be different for another one.

Since Mad Pascal is new I'm sure there is room for improvement.

Edited July 28, 2015 by JamesD

JamesD · July 28, 2015

I think Dhrystone was a good benchmark for comparing compilers.

http://www.netlib.org/benchmark/dhry-pascal

+Larry · July 29, 2015

Yes, Sieve sounds like a winner. Thanks!

Larry

tebe · July 31, 2015

Sieve of Eratosthenes

https://en.wikipedia.org/wiki/Action!_(programming_language)

Edited July 31, 2015 by tebe

fujidude · July 31, 2015

Sieve of Eratosthenes

https://en.wikipedia.org/wiki/Action!_(programming_language)

I get these results:

1899 primes in 89 jiffies (Altirra 1.79MHz 65C816)

1899 primes in 7 jiffies (Altirra 21.48MHz 65C816)

A jiffie on my NTSC emulated system is 1/60th of a second. So 89 jiffies = 1.48S, and 7 jiffies = 0.11S.

GoodByteXL · July 31, 2015

Yes, Sieve sounds like a winner. Thanks!

Larry

As always ... ;-)

But it's a very syntetic benchmark.

In the 1980s I was playing around with tests more oriented to practical issues: Moving memory content, calculating print data for graphical printouts in A3 and A4, converting pictures in memory, kaleidoscopes, etc. All routines done in the respective high level language without any tricks or subroutines in assembler. To me back then this was a more realistic approach to benchmark our A8.

mike.2000 · July 25, 2018

Performance compare of various Atari 800 programming languages

The Task: Fill a GRAPHICS 7 or GRAPHICS 15 screen

All programs except the last use built-in Atari graphics routines like

Graphics N
Position
Plot

Basic program: Graphics 7 takes the computer 103 seconds

Pascal program: Graphics 7 takes the computer 26 seconds. Used Kyan Pascal

ACTION! program: Graphics 15 takes the computer 45 seconds. Converted to Graphics 7 transforms to 23 seconds

Assember version 1: Graphics 7 takes the computer 18 seconds. I used ATMAS II

Assember version 2: Graphics 15 takes the computer 2 seconds!!! I used ATMAS II without the libraries

Graphical display of the time in seconds:

------------------------------------------------------------------------------------------------------- (Atari Basic)

-------------------------- (KYAN Pascal)

----------------------- (ACTION!)

------------------ (Assember version 1)

-- (Assember version 2)

The listings for Basic, PASCAL and ACTION! aren’t shown here, as they are basically the same. You know…

set GRAPHICS mode,
two encapsuled FOR-Loops x,y
…
plot
….
END.

That’s a bit boring.

Maybe Assembler is more interesting

Here’s my lisiting: Assember version 1 program

**************************************

* MICHAEL SANDAU *

* PERFORMANCE ATARI 800 *

* TASK:FILL A *

* GRAPHICS 7 SCREEN *

* using a modified library *

* GRAFLIB.SRC mod is passing *

* x,y parameters in *

* X,Y registers direct *

* FILE: SPL.SRC *

***********************************************************************

* GRAFLIB.SRC

*

* Makro-Bibliothek

*

* GRAPHIK

*

* Fuer ATMAS-II

* PETER FINZEL

***********************************

ZPXPOS EQU $C0 * ZERO PAGE X

ZPYPOS EQU $C1 * ZERO PAGE Y

* IOCB-Struktur:

*

ICCOM EQU $342

ICSTA EQU $343

ICBAL EQU $344

ICBAH EQU $345

ICBLL EQU $348

ICBLH EQU $349

ICAX1 EQU $34A

ICAX2 EQU $34B

CIOV EQU $E456

* CIO-Befehle

COPEN EQU 3

CCLSE EQU 12

CGTXT EQU 5

CPTXT EQU 9

CGBIN EQU 7

CPBIN EQU 11

CDRAW EQU $11

* ATARI Graphik-Variable

ATACHR EQU $2FB

ROWCRS EQU $54 CURSOR-

COLCRS EQU $55 POSITION

*

* GRAPHICS-Befehl

*

* Aufruf: GRAPHICS <stufe>

*

* <stufe> 0 bis 15 (XLs)

* 0 bis 11 (400/800)

*

GRAPHICS MACRO STUFE

JMP GR1

@

DEV@ ASC "S:"

GR1@ LDX #$60

LDA #CCLSE ZUERST KANAL 6

STA ICCOM,X SCHLIESSEN

JSR CIOV

LDA #STUFE JETZT NEUE GRAPHIK

STA ICAX2,X STUFE ANWAEHLEN

AND #$F0

EOR #$10

ORA #$0C

STA ICAX1,X

LDA #COPEN

STA ICCOM,X

LDA #DEV@

STA ICBAL,X

LDA #DEV@/256

STA ICBAH,X

JSR CIOV

MEND

*

* Auswahl der Zeichenfarbe

*

* Aufruf: COLOR <farbe>

*

* <farbe> von 0 bis 255, je nach

* Graphikmodus, muss eine

* Konstante sein.

*

COLOR MACRO COL

LDA #COL

STA ATACHR

MEND

*

* Positionierung des Cursors

*

* Aufruf: POSITION <x>,<y>

*

* <x>,<y> je nach Graphikmodus, beide

* UEBERGEBEN IN X UN Y REGISTER

*

POSITION MACRO

* LDA #X

TXA

STA COLCRS

* LDA #X/256 ONLY 320 PIX MODE

LDA #$0

STA COLCRS+1

* LDA #Y

TYA

STA ROWCRS

MEND

* Graphik-Punkte setzen

*

* Aufruf: PLOT <x>,<y>

*

* <x>,<y> je nach Graphikmodus,

* VOR AUFRUF IN X UND Y REGISTER

* BEFINDEN

*

PLOT MACRO

LDX #$60 KANAL 6

LDA #CPBIN

STA ICCOM,X

LDA #0

STA ICBLL,X

STA ICBLH,X

LDA ATACHR

JSR CIOV

MEND

**************************************

* MICHAEL SANDAU *

* PERFORMANCE ATARI 800 *

* MAIN PROGRAM comes here: *

***************************************

ORG $A800

GRAPHICS 7

COLOR 2

LDY #$0

STY ZPYPOS

OUTERY LDX #$0

STX ZPXPOS

INNERX POSITION * position in x and y register

PLOT

* GET POSITIONS BACK

LDX ZPXPOS

LDY ZPYPOS

INX

STX ZPXPOS

CPX #159

BNE INNERX

INY

STY ZPYPOS

CPY #95

BNE OUTERY

GRAPHICS 0

Assember version 2 program :

- saves current display list

- creates new display list for GRAPHICS 15

- does the performance by writing the pixels directly to screen memory

- restores previos display list for GRAPHICS 0

The fastest performing program is written completely by me. The comparison to the other programs is not fair. The program is so specialized that it ONLY works for this particular GRAPHICS mode.

**************************************

* MICHAEL SANDAU *

* PERFORMANCE ATARI 800 *

* TASK:FILL A *

* GRAPHICS 15 SCREEN *

* *

* FILE: SPD.SRC *

**************************************

OLOOP EQU $B0 ; OUTER LOOP SHADOW

SDLISTL EQU $230 ; DL PTR LOW

SDLISTH EQU $231

DL1LEN EQU 32 ; LENGTH OF 1. DL

DL2LEN EQU 192 ; LENGTH OF 2. DL

ZPDLL EQU $B1 ; DL PTR IN ZERO PAGE

ZPDLH EQU $B2

ZPSPL EQU $B3 ; SCR PTR FOR COUNT

ZPSPH EQU $B4

PIXCNT EQU $B5 ; COUNTER FOR PIX/BYTE

SCRMEML EQU $00 ; SCREEN MEMORY GRAPHICS

SCRMEMH EQU $80

SDLISTC EQU $AA00 ;COPY OF DL

*

* AdresS OF CODE

*

ORG $A800

*

* PERFORMANCE TEST

*

JSR SAVEDL * SAVE DISPLAY LIST

JSR SETDL * SET DL TO GRAPHICS 15

JSR PERF * do perf test

JSR PERF

JSR RESTDL * restore DL

RTS

*

* **** END ******

*

PERF LDA #SCRMEML ; CREATE A PTR

STA ZPSPL ; TO LOOP THRU

LDA #SCRMEMH ; SCREEN MEMORY

STA ZPSPH

* OUTER LOOP 1: X REGISTER

LDX #0

LOOPXS NOP

* INNER LOOP 2: Y REGISTER

LDY #0

* ** SELECTING PIXELS *****

* ** AND STORE THEM IN SCREEN MEM

LOOPYS LDA #$40

STA (ZPSPL),Y ; 1 PIXEL

LDA #$50

STA (ZPSPL),Y

LDA #$54

STA (ZPSPL),Y

LDA #$55

STA (ZPSPL),Y ; 4 PIXELS

* STA $D40A ; wait wsync

INY

CPY #39 ; NEXT BYTE

BNE LOOPYS

* END INNER LOOP 2

* ADD 40 TO ZERO PAGE SCREEN POINTER

CLC

LDA ZPSPL

ADC #39 ;ADD 40 COLUMNS

STA ZPSPL

LDA ZPSPH

ADC #0 ;ADD CARRY IF NEEDED

STA ZPSPH

* CONTINUE OUTER LOOP

INX

CPX #191

BNE LOOPXS

* END OUTER LOOP 1

* END PERFORMANCE

RTS

***** SAVES THE ORIGINAL DISPLAY

***** LIST

SAVEDL LDY 0

LDA #$01

STA $C1 ; TRACE

LDA SDLISTL

STA ZPDLL

LDA SDLISTH

STA ZPDLH

LOOPDLS LDA (ZPDLL),Y

STA SDLISTC,Y

INY

CPY #DL1LEN

BNE LOOPDLS

RTS

SETDL LDY #6

LDA #$E ; antic mode

LPDL1S STA (ZPDLL),Y

INY

CPY #198 ; 192 TIMES

BNE LPDL1S

LDA #$41 ; SET END OF DL

STA (ZPDLL),Y

INY

LDA ZPDLL

STA (ZPDLL),Y

LDA ZPDLH

STA (ZPDLL),Y

LDY #3 ; LMS POS

LDA #$4E ; ANTIC MODE + LMS

STA (ZPDLL),Y

LDA #SCRMEML ;SET SCREEN MEMORY

INY ;SCREENMEM LOW

STA (ZPDLL),Y

LDA #SCRMEMH

INY

STA (ZPDLL),Y

RTS

*** RESTORES THE OPRIGINAL DISPAY LIST

RESTDL LDY #0

LDL1R LDA SDLISTC,Y

STA (ZPDLL),Y

INY

CPY DL1LEN

BNE LDL1R

RTS

******* END **********************

Geister · August 7, 2018

The Action! language is very closely designed around the particulars of the 6502 specifically, so the object code it makes is often tighter, faster, and more efficient than other high level languages that compile for the 6502. On the other hand, Action does not even natively support real (floating point) numbers. There is a library to add the capability, but it is a bit unwieldy.

A lot of software really doesn't require real number support though, and unwieldy or not, if you do need it you can "bolt it on." In all, Action is a great language for 6502 based machines and is the one I always hope to see get the most attention and effort in enhancing. A cross (or native) compiler that could output object code directly to disk would be wicked for example. There is... anyway I'm digressing. For a lot of tasks, you will be hard pressed to beat Action! for speed of high level language.

Can anyone elaborate on the floating point library for Action? Where can it be found and is there any documentation for using it?

+slx · August 8, 2018

Can anyone elaborate on the floating point library for Action? Where can it be found and is there any documentation for using it?

There's one including docs here and probably more on the same website. It's also included in GoodByteXL's Action! documentation that can be found one level up on the same website.

bugbear · August 8, 2018

Can anyone elaborate on the floating point library for Action? Where can it be found and is there any documentation for using it?

If your purpose is speed (which is what Action is for) you should probably think long and hard before using floating point.

Unless you're writing numerical analysis software for the 6502, in which case - good luck with that!

BugBear

Geister · August 10, 2018

SIX, thanks for the reply, i'll definitely check it out.

bugbear, I'm also interested in bench-marking the various languages available on the 8-bit Atari, but I feel to get an over-all bench mark you have to test more than just the fastest features. I feel that a good graphical routine that uses floating point gives you a more realistic bench mark than just finding primes with integer math...plus, it's slightly less boring having something to look at.

My benchmark of choice is that old chestnut the 3-D hat (Sine) program. It really stresses the system, and any improvement over the box stock basic or floating point routines sticks out like a sore thumb.

And thanks in advance for any speed up tips for the Hat program itself, but that's not really what I'm looking for. I just want to as directly as possible translate the original program code into each programming language using floating point so that I'm comparing apples to apples.

Altirra Basic and floating point routines are pretty quick and point to the biggest speed up for the Atari. Replace your OS ROM and Basic ROM with the Altirra versons and you almost tripled the speed of your Atari's Basic performance.

Of course you can get the same results by running Turbo Basic XL

Edited August 10, 2018 by Geister

JamesD · August 10, 2018

The Fedora hat 3D plot is a good measure of the speed of parts of the math library, but that doesn't always translate to other programs.

bugbear · August 10, 2018

Benchmark design is so hard that entire books have been published about it...

BugBear

What is a good benchmark for comparing the speed of languages?

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Recently Browsing 0 members