New benchmark timings up

[Maury Markowitz]

I don't know how I did not come across this earlier; in the September 1982 Insight Atari, Bill posted a short benchmark to compare variations of BASIC on the Atari - along with Applesoft as a comparison. Here are the results (at the bottom of the page).

 

The bench is fascinating - it's clearly written specifically to make AB look good. Ultimately all it does is loop 10,000 times doing a simple addition in the middle of the loop. Typically one would use a FOR/NEXT for this sort of task, but in this case it uses a branch. This plays to AB in two ways, one is that FOR performance in AB is (comparatively) very poor, and the other is that backward branches do not take advantage of that little tweak in MS BASIC's so both are searching from the top in both cases.

The modified versions are designed to illustrate some edge cases, but again, they are carefully selected to play into AB's strengths. Note for instance that version B uses ONE more decimal place. This triggers MS's double-precision mode and slows it down, but is still within AB's single-precision format limit. If you add one more decimal MS continues to work (albeit slowly) while AB now fails to give the proper result.

 

The next two variations in the original article add long variable names that differ only in the last letter, which forces MS to do string searches across the entire variable name. This is not quite so much a cheat, but one might consider that most MS's don't support these at all, so this is really testing "Atari MS BASIC" vs AB, not AB vs the MS hoi polloi.

 

In spite of any of that, there's still some really interesting bits one can find in these tests. For instance, note the difference in time between the a and c variations using ABrevC. The c version simply replaces a single numeric constant with a variable containing the same value. So the slight difference in time, .5 seconds, appears to be the 10,000 executions of the quarter page of assembler needed to look up the value from the variable table. This is a very useful bit of data!

 

What's even more interesting is the comparisons with other BASICs on the platform. One would not expect, for instance, BASIC XL to be much faster than AB without using FAST, but as you can see, it runs these tests roughly twice as fast across the board! And that's *without* a new math library or FAST. Note that this test has only a single branch and it's in a tiny program, but in spite of that, FAST mode still shaves a whopping 5.5 seconds off the times. The same is true in the XE versions, where adding FAST also results in just over 5 seconds improvement.  But in that case, we gain even more insight from the fact that replacing the math library speeds things by almost 11 seconds.

 

Note that in this bench, XL and XE manage to outperform Turbo by some reasonable amount. I attribute this to the fact that the entire program fits within the first "group" of line numbers and thus gains nothing from their caching concept. It is surprising that XE's math lib outperforms it in this case, as I've yet to see an example of that in the past - but of course there's no multiplication in this example so Turbo's real improvements aren't being kicked in.

Edited January 2, 2022 by Maury Markowitz