What really makes TI EX-BASIC slow?

[+hloberg]

I ran some test on my Ti-99/4a v. Atari 800xl (emulator).

I used TI Ex-BASIC and Atari OSS BASIC XL and found that Atari BASIC XL was more than twice as fast as TI Ex-BASIC overall in the tests. Atari Turbo-BASIC was even faster.

(for you unfamiliar with the Atari world OSS BASIC XL would be about the closest thing to TI's Ex-BASIC. Turbo-BASIC came later.)

With the TI having a faster processor, a (mostly) 16-bit processor and dedicated memory for video (so no need for wait states with shared memory) I can only theorize that the bottle neck is the double interpreting with GPL. Or is it just lousy programming?

Any thoughts?

Oh, before I get flamed, I'm a big TI-enthusiast with a question not an Atari guy trying to pick a fight.

Edited January 30, 2012 by hloberg

[Tursi]

It's the GPL interpreter and the wait states -- your comment that there is no need for wait states is sadly not true. Wait states happen on almost every memory access in the TI because the 16-bit CPU is interfaced to the world (including the internal peripherals) through an 8-bit bus.

 

Although we've almost certainly covered this elsewhere in the forum, basically:

 

-The bare console (without expansion) has only 256 bytes of CPU RAM, so everything to do with your BASIC/XB program is stored in video RAM

-Video RAM is only accessible through a memory-mapped port, one byte at a time. Wait states happen for every access AND the interpreter follows TI's recommendation to put a dummy instruction between every access.

-The GPL interpreter itself is fairly slow, as it works in a number of different modes and seems to perform some redundant operations at times - it doesn't seem to be optimized for speed at any rate.

-GROM memories do take some blame, as they are about 30 times slower than ROM memory. But I've done a lot of testing and I'd say that only about one memory access in 30 actually goes to GROM on average, so it doesn't hit as bad as you'd think.

 

Speed is helped some by:

-Memory expansion lets XB store program and data in CPU RAM instead of VDP RAM, which is much faster to access

-The GPL interpreter itself runs in 16-bit ROM with no wait states

-Extended BASIC has some functions re-written in assembly

[matthew180]

There are also little bits of "undesirable" code scattered around that contribute to the slowness. For example, since the keyboard is not debounced in hardware, a software debounce is performated in the form of a delay loop in the console ROM keyboard routine. This delay will affect any software that uses those routines (which includes BASIC and XB.)

[RXB]

I think another huge problem is the VDP is where so much is stored.

Example: Variables, Constants for caclulations, and buffers.

GPL is not really to blames as when a GPL command like MOVE V@SIZE,V@FROM,V@TO

Varibable SIZE is in VDP

Variable FROM is in VDP

Variable TO is in VDP

 

The above command branches to a GPL command that is written in Assembly so like Tursi said the GPL is accessed 30 times less then the VDP.

This command above has Assembly access VDP thousands of times more often then GPL so really VDP is the reason the TI Basic and XB are slow.

 

If VDP was in main RAM you would THEN SEE GPL is the problem, but let us be honest it is clear that VDP is what is slowing everything down.

Edited January 30, 2012 by RXB

[+hloberg]

Having read what has just been posted (correct me if i'm wrong) moving the tables out of VDP sounds like one thing that would speed things up a lot for XBASIC. That would cut down on a lot of unnecessary access to the VDP. Since only updates to the VDP memory would be sent, that would be a lot less data being sent through the 8byte pipe.

That would also open up all the video modes to being usable to XBASIC (something TI should done from the start) since there will be no chance of conflicting with XBASIC tables.

Sound about right?

[Tursi]

If VDP was in main RAM you would THEN SEE GPL is the problem, but let us be honest it is clear that VDP is what is slowing everything down.

 

I'm unconvinced that it is the MAIN bottleneck, but you have a point in that doesn't help. My unproven and potentially controversial opinion is that better than 80% of the blame lies in the GPL interpreter itself. I think a new interpreter written for speed (and yes, including our modern lack of fear of VDP overrun ), and dropping some of the unused compatibility code, could get twice the performance of the current one.

 

I'm not saying whoever wrote the original GPL interpreter was sloppy. I just think that there is a lot of unrealized functionality in there (as there is in much of the console) that we could strip away. (Although, I do think it has its sloppy parts, but all code does.)

 

If it was easier to replace the console ROMs, it would make a great project as a potential upgrade.

[Tursi]

Having read what has just been posted (correct me if i'm wrong) moving the tables out of VDP sounds like one thing that would speed things up a lot for XBASIC. That would cut down on a lot of unnecessary access to the VDP. Since only updates to the VDP memory would be sent, that would be a lot less data being sent through the 8byte pipe.

That would also open up all the video modes to being usable to XBASIC (something TI should done from the start) since there will be no chance of conflicting with XBASIC tables.

Sound about right?

 

It would speed it up a bit. You can see how much it helps by just running Extended BASIC on a machine with 32k versus a machine that does not have 32k. But it's not a very large jump. The main problem is that it's written to be /able/ to.

 

Can someone remind me -- did we verify that Extended BASIC has its own GPL interpreter in its ROM?

 

As for the video modes... with the exception of bitmap I don't see why they didn't give us access to them all at the start. Text and Multicolor would have played fine with BASIC. My assumption is that someone didn't want to deal with display-size independence.

[+hloberg]

eliminate GPL, put tables off VDP.

sounds easy enough.

Seriously, It sounds the only way to gain any real speed would be to do a rewrite an XBASIC from scratch with mostly all assembler and proper use of VDP (no tables). (see my next post. looks like someone already did that).

Edited January 31, 2012 by hloberg

[+hloberg]

Possible answer to part of my question.

I have MYARC XBASIC II on my PC99. ran some of the speed test on the MY XBII and it is at least twice as fast as standard XB. One of the things they did was move the tables off the VDP onto main RAM. of course they probably did other things too to speed it up, maybe re-wrote some of the code as assembler.

Also introduced integer variables.

 

might be worth my while to dis-assemble it one day.

Edited January 31, 2012 by hloberg

[RXB]

This is exactly what my Basic/XB to GPL Compiler will do. All the variables are in RAM and Strings and Contants. The program resides in GPL with full normal XB Assembly support.

( Dare anyone to fully support XB Assembly support like it normally works but in the compiler)

 

How could GPL be the problem when access to VDP is happening 10 times as much as access to GPL. If you hate GPL just admit it instead of making crap up.

 

You guys have access to GPL, make a program that writes 1k to VDP 1000 times and 1k to RAM 1000 times and compare the times.

 

Then do the same thing in Assembly, you will be very suprised at the results you get.

 

Heck you can see the difference if you just do it in XB or Basic.

[apersson850]

Since I have a computer that has all memory expansion as 16 bit RAM, I can tell you that the speed difference between Extended BASIC using that, compared to using the standard 8-bit memory expansion, isn't noticeable.

You can measure it, if you measure carefully, but we're talking differences like 59 seconds instead of 60. In spite of six clock cycles for an ordinary RAM expansion access and two with my design.

[matthew180]

How could GPL be the problem when access to VDP is happening 10 times as much as access to GPL.

 

How? Like this:

 

Fact: GPL is an interpreted language, and thererfore will always be slower than a assembly or a compiled language. This has *NOTHING* to do with what kind of system resources the programs are accessing. VDP latency time, RAM latency time, etc. are *the same* for any language.

 

Fact: BASIC and XB are interpreted languages written in GPL, which itself is interpreted, and therefore adds to the languages being slower than those written in a non-interpreted language like assembly.

 

GPL execution is slow compared to a compiled program (assembly in our case since there isn't much else viable right now), XB is slower, and BASIC is at the bottom. All the arguing in the world won't change those facts.

 

If you hate GPL just admit it instead of making crap up.

 

First, quote from this thread where anyone said they hated GPL. No one said that. Why is it that any time we discuss GPL performance that you get on a high-horse about it? You love GPL, that's great, but that does not change the fact that it is a slow language compared to assembly. I like assembly, but I don't get all bent out of shape when someone says it is hard or sucks. That's their opinion, but it does not change the fact that assembly is faster than GPL or BASIC.

 

Second, no one made any thing up. Quote me something "made up" from this thread.

 

You guys have access to GPL, make a program that writes 1k to VDP 1000 times and 1k to RAM 1000 times and compare the times.

 

You have access to assembly, you write the programs. 1K of data 1000 times... That's 1MB of data. Why would I ever write 1MB to VDP RAM?

 

Then do the same thing in Assembly, you will be very suprised at the results you get.

 

Not really. Your suggested program is trying to isolate the VDP access time, which is fixed by the hardware no matter the language you are using. Your suggested program is simply measuring the loop overheard of GPL vs. assembly, which no matter what you want it to be, will always be slower than assembly.

[matthew180]

Seriously, It sounds the only way to gain any real speed would be to do a rewrite an XBASIC from scratch with mostly all assembler and proper use of VDP (no tables).

 

Maybe, but probably not. Even if you wrote a 100% assembly version of some language (BASIC in this case), the 99/4A is still a seriously crippled design. Based on the system specs, the 99/4A should have smoked all the home computers of its time, instead it manages to be one of the slowest. Basically you just have to love it for what it is. Everyone here has an affinity for the machine for their own reason, and that's the binding factor of this group.

 

We all have dreams of what we would like the machine to be, or have had, or what we could make with technology today, but none of that changes what the 99/4A was and is.

 

It is fun to dream about and discuss new hardware / software (although that seems to bother some people endlessly), and every once in a while something comes along and we get just a little more enjoyment from our classic computer of choice. However, I don't think the 99/4A will ever get a truely fast, 100% assembly written, BASIC dialect. That's just my opinion though, and it should not stop anyone from dreaming or trying. :-)

[+hloberg]

Actually Myarc XBII DOES address most of the problems. It's speed is up there with the best of the 8bit machines (I have run test) and has access to the full features of the 9918. So it was/is possible. Only thing is you need at least 128k to do it.

Was it GPL, VDP or bad programming? Only the guys at Myarc would know what they did to speed it up.

So you are right, out of the box the TI99 is a very crippled machine (at least BASIC wise).

 

When I was in college I first had a TI99 but later moved on to an Atari 800XL. Yes, the Atari was a superior machine in many respects and the Atari community has gotten some amazing things out of the 8-bit line.

 

But coming back to the TI99 as hobby I'm seeing that the TI had/has a lot of potential that was never properly exploited.

The machine has the potential to compare favorable with the other machines of it's time with a little work.

 

As for your challenge about a faster BASIC, I think Myarc already took you up on that a proved it could be done.

I would like to see if that Myarc code could be redesigned to work in any machine with 128k or more?

Maybe when I have time I might take that as a project (like I ever will).

[Stuart]

eliminate GPL, put tables off VDP.

sounds easy enough.

Seriously, It sounds the only way to gain any real speed would be to do a rewrite an XBASIC from scratch with mostly all assembler and proper use of VDP (no tables). (see my next post. looks like someone already did that).

 

There's also the port of the 9995-based Powertran Cortex BASIC I done for the TI - <http://www.avjd51.dsl.pipex.com/ti/ti.htm#programs>. Pure assembly, access to text and bitmaps modes on the VDP. Unfortunately due to the size of the code, user programs are limited to around 5K or so which makes it of little practical use. It's significantly faster than TI BASIC and Extended BASIC. You might want to try it against the Atari BASIC and see how it shapes up.

 

Stuart.

[RXB]

How could GPL be the problem when access to VDP is happening 10 times as much as access to GPL.

 

How? Like this:

 

Fact: GPL is an interpreted language, and thererfore will always be slower than a assembly or a compiled language. This has *NOTHING* to do with what kind of system resources the programs are accessing. VDP latency time, RAM latency time, etc. are *the same* for any language.

 

Fact: BASIC and XB are interpreted languages written in GPL, which itself is interpreted, and therefore adds to the languages being slower than those written in a non-interpreted language like assembly.

 

GPL execution is slow compared to a compiled program (assembly in our case since there isn't much else viable right now), XB is slower, and BASIC is at the bottom. All the arguing in the world won't change those facts.

 

If you hate GPL just admit it instead of making crap up.

 

First, quote from this thread where anyone said they hated GPL. No one said that. Why is it that any time we discuss GPL performance that you get on a high-horse about it? You love GPL, that's great, but that does not change the fact that it is a slow language compared to assembly. I like assembly, but I don't get all bent out of shape when someone says it is hard or sucks. That's their opinion, but it does not change the fact that assembly is faster than GPL or BASIC.

 

Second, no one made any thing up. Quote me something "made up" from this thread.

 

You guys have access to GPL, make a program that writes 1k to VDP 1000 times and 1k to RAM 1000 times and compare the times.

 

You have access to assembly, you write the programs. 1K of data 1000 times... That's 1MB of data. Why would I ever write 1MB to VDP RAM?

 

Then do the same thing in Assembly, you will be very suprised at the results you get.

 

Not really. Your suggested program is trying to isolate the VDP access time, which is fixed by the hardware no matter the language you are using. Your suggested program is simply measuring the loop overheard of GPL vs. assembly, which no matter what you want it to be, will always be slower than assembly.

 

OMG you write to VDP 1000 times, you have 16K. How would you ever assume that I ment you could write 1000K of VDP? You are kidding right?

 

Maybe it is the fact that I know more about GPL than anyone else as I can write in it with no problem at all. Not to mention that XB Source is mostly what I look at all the time.

 

You guys are just guessing most of the time about GPL, it is why I see maybe this or maybe that said so much. I am not saying GPL is fast or better or anything.

 

I am pointing out that the flipn bottleneck is VDP not GPL, look when you access VDP it is 1 BYTE AT A TIME AND YOU NEED A WAIT STATE EACH TIME.

 

Secondly the TI accesses the VDP thousands of times more often the GPL, this is just a fact. XB has ROMs to try and make up for this huge slow down. Disassemble one and see.

 

Look at the RXB source code it is directly mostly TI XB source that I have patched. It has routines to take the place of GPL MOVE command and is not faster at all, VDP is the problem.

 

At least acknowledge that VDP is mostly what the system is doing in any language and it is the problem. Assembly, GPL, Pascal, Forth, or any other language.

[Tursi]

Rich, you seem to be taking it personally. There's less guessing involved in GPL discussion than you think.

 

I've sat there and measured the performance of the GPL interpreter, and I've spent hours tracing through it. I learned to program GPL at the byte level. You know GPL from the programmer's point of view. No offense intended but I believe that I know more about how GPL /works/ than you do. You know more about how to /use/ GPL than I do.

 

My opinion is that you are wrong that VDP delays are the /main/ cause of BASIC's performance. I back up my opinion with all the experimentation and research I've done with the machine. I have sat there with my logic analyzer and watched the system operate. I have replaced the GROMs with a microcontroller and traced the system operation. I have spent HOURS tracing through the GPL interpreter. The main cause of BASIC's performance, in my opinion, is the large amount of work that the GPL interpreter does to decode every instruction it reads.

 

If you time "MOVE 1000 FROM CPU@>A000 to VDP@>1000" in GPL versus a tight copy loop in assembly, GPL unfortunately loses by a large margin. The actual VDP time in both cases is equivalent. But the MOVE instruction sets up the GPL address for every byte, because it has to assume that the VDP interrupt may have occurred. It also has to abstract both the source and destination addresses, since it doesn't enforce anything particular. This is the GPL MOVE loop, starting at >065E in the ROM (just the loop, not the setup or instruction decode):

 

065E
B *6 Execute source (jumps to one of the source functions below)  

Source ROM or RAM:
MOVB *1+,11 Fetch
B *7 Execute destination (jumps to one of the destination functions)

Source VDP RAM:
MOVB @>83E3,*15 Write address
MOVB 1,*15
INC 1
MOVB @>FBFE(15),11 Fetch data
B *7 Execute destination

Source GROM:
MOVB 1,@>0402(13) Write GROM address
MOVB @>83E3,@>0402(13)
INC 1
MOVB *13,11 Fetch data
B *7 Execute destination

Destination RAM:
MOVB 11,*2+ Write
JMP >06CA Go on

Destination GROM:
MOVB 2,@>0402(13) Write GROM address
MOVB @>83E5,@>0402(13)
INC 2 Next address
MOVB 11,@>0400(13) Write into GRAM
JMP >06CA Go on

Destination VDP register:
CB @>83E5,14 R2 Lbyte >01?
JNE >06AC
COC @>0012,14 Version?
JNE >06A8
ORI 11,>8000 Set 16k bit
06A8 
MOVB 11,@>83D4 Register value 1
06AC 
MOVB 11,*15 Write
ORI 2,>0080 VDP register
MOVB @>83E5,*15 Write from Lbyte R2
INC 2 Next register
JMP >06CA Go on

Destination VDP RAM:
MOVB @>83E5,*15 Write address VDP
ORI 2,>4000 Writing
MOVB 2,*15
INC 2 Next address
MOVB 11,@>FFFE(15) Write data

06CA 
DEC 8 End ?
JGT >065E No, go on
B @>083E Return GPL interpreter, set condition bit and GROM address from substack

 

The above code executes for every single byte of the MOVE instruction. And that's one of the faster GPL instructions, because after it's decoded, it's running entirely in 16-bit assembly. But look at the case for copying from CPU (say a cartridge) to VDP memory. There are 10 instructions per byte. Three of those incur the VDP waitstate (the same wait state would be incurred if the CPU access was in 8-bit space, which for a cartridge it is). But instructions have varying durations.

 

In terms of cycles, we have this in 16-bit ROM, so no wait states on code execution. Registers are in scratchpad. So we're running at top speed. I get 174 cycles per byte for this loop. Of that, 4 cycles is wait state for reading from the 8-bit cartridge ROM (2%), and 24 cycles are wait state for talking to the VDP, thanks to read-before-write (13%). The wait state overhead is high, but GPL has to set the address every loop, so it's doing 3 times as many writes as assembly has to. And even 13% is still the minority of the time.

 

Assembly doesn't have to be flexible. So it can be MUCH faster.

 

MOVB *R0+,*R15   * write from memory to VDP, autoincrement source
DEC R2
JNE LP

 

Let's assume normal case - this code running in 8-bit memory, not scratchpad. Even so, I get 70 cycles per byte. 12 cycles are wait state for reading instructions from 8-bit memory (17%), 4 cycles are wait state for reading CPU (5%) and 8 cycles are wait state for writing VDP (11%). The code is almost 2.5x faster than the GPL loop, though, and that's running from 8-bit memory with wait states!

 

The VDP is the SAME SPEED as the 32k memory expansion, and as every expansion card in the PEB. The difference being it's not random access. Accessing the VDP memory sequentially takes the same amount of time as accessing 8-bit RAM or ROM sequentially. In fact, if you are doing a copy loop to or from RAM or ROM, VDP can actually be FASTER, because you can drop one of the post-increments in your loop, giving the 9900 less work to do.

 

But until it has been PROVEN otherwise, by someone actually correcting the issue and demonstrating it, it's still just theory and opinion.

 

[Willsy]

I would also add to Tursi's post, that, in addition to the extra work that GPL has to do in things like MOVE loops (explained by Tursi, above) there is the added overhead of the GPL interpreter actually *interpreting* the byte code. GPL is a virtual machine. More accurately a virtual processor. Therefore it is reading byte codes, determining which format op-code the instruction is, masking the source/destination bits of the instruction etc etc all in software (machine code) BEFORE it gets to execute the damn instruction!

 

Don't get me wrong; it's a neat as hell - but it adds to the overhead.

 

If it's any consolation Rich, I think GPL is super cool. The 4A is probably ten times more sophisticated than any other home computer ever built, with the exception of the Amiga. It's probably about equal in terms of sophistication as the Atari ST.

[marc.hull]

I ran some test on my Ti-99/4a v. Atari 800xl (emulator).

I used TI Ex-BASIC and Atari OSS BASIC XL and found that Atari BASIC XL was more than twice as fast as TI Ex-BASIC overall in the tests. Atari Turbo-BASIC was even faster.

(for you unfamiliar with the Atari world OSS BASIC XL would be about the closest thing to TI's Ex-BASIC. Turbo-BASIC came later.)

With the TI having a faster processor, a (mostly) 16-bit processor and dedicated memory for video (so no need for wait states with shared memory) I can only theorize that the bottle neck is the double interpreting with GPL. Or is it just lousy programming?

Any thoughts?

Oh, before I get flamed, I'm a big TI-enthusiast with a question not an Atari guy trying to pick a fight.

 

It is because of two things...

 

#1) Double interpretation of the language.

#2) VDP memory being the ONLY storage option for a console. BASIC NEVER gets away from this ball and chain.

 

It's a little from column A and a little from column B. It's unfortunate that it happened this way but it gives the cycle counters something to do....

Edited January 31, 2012 by marc.hull

[+hloberg]

Wherefore GPL? One of those eternal questions that may never be answered.

<Been looking through the forum and haven't found these answers.>

Now for some dumb (and maybe obvious) questions. Is GPL processed by the VDP and not the CPU? Is GRAM and GROM memory accessed by the VDP only? How can you program GPL from an emulator and what tools you use?

 

Suggestion Rich, maybe a tutorial on GPL would help people appreciate it's power. As you point out in a previous post it can use a lot less coding than assembler and is almost as fast.

Edited January 31, 2012 by hloberg

[+hloberg]

Just read Willsy comments. The GPL processor is virtual? i was assuming it was more like the graphics language used in high end VDPs of today. It sounds more like JAVA.

[+hloberg]

eliminate GPL, put tables off VDP.

sounds easy enough.

Seriously, It sounds the only way to gain any real speed would be to do a rewrite an XBASIC from scratch with mostly all assembler and proper use of VDP (no tables). (see my next post. looks like someone already did that).

 

There's also the port of the 9995-based Powertran Cortex BASIC I done for the TI - <http://www.avjd51.ds...ti.htm#programs>. Pure assembly, access to text and bitmaps modes on the VDP. Unfortunately due to the size of the code, user programs are limited to around 5K or so which makes it of little practical use. It's significantly faster than TI BASIC and Extended BASIC. You might want to try it against the Atari BASIC and see how it shapes up.

 

Stuart.

 

Thanks, actually tried it. Tad faster but Myarc XBII still is faster. It's a shame he hasn't included the source code might be able to do something with it.

[+Lee Stewart]

Wherefore GPL? One of those eternal questions that may never be answered.

<Been looking through the forum and haven't found these answers.>

Now for some dumb (and maybe obvious) questions. Is GPL processed by the VDP and not the CPU? Is GRAM and GROM memory accessed by the VDP only? How can you program GPL from an emulator and what tools you use?

 

Suggestion Rich, maybe a tutorial on GPL would help people appreciate it's power. As you point out in a previous post it can use a lot less coding than assembler and is almost as fast.

 

Uh, oh! You're gonna catch it for that. You probably have not checked out the "GPL HOW 2 SERIES" on this forum in the TI-99/4A Development Resurces at http://www.atariage....ment-resources/, have you?

 

...lee

[marc.hull]

Wherefore GPL? One of those eternal questions that may never be answered.

<Been looking through the forum and haven't found these answers.>

Now for some dumb (and maybe obvious) questions. Is GPL processed by the VDP and not the CPU? Is GRAM and GROM memory accessed by the VDP only? How can you program GPL from an emulator and what tools you use?

 

Suggestion Rich, maybe a tutorial on GPL would help people appreciate it's power. As you point out in a previous post it can use a lot less coding than assembler and is almost as fast.

 

Uh, oh! You're gonna catch it for that. You probably have not checked out the "GPL HOW 2 SERIES" on this forum in the TI-99/4A Development Resurces at http://www.atariage....ment-resources/, have you?

 

...lee

 

Ya know if you want to take a jab at Rich (applies to everyone) then have the balls to come out like a man and take a jab at him. Shove your smileys and your sarcasm and just say what's on your mind. If you can't do that then STFU and sit back down in your corner. This is gutless BS.

[jchase1970]

This forum keeps taking little dives towards the yahoo group mentality. This is a programming forum for people to learn and experience the Ti99 in. Using BASIC, XB, GPL, FORTH, ASSEMBLY, or anything else. We need no, "MY WAY IS THE BEST WAY" or "YOU WAY IS TERRIBLE" flame wars on here.

 

We want a friendly open enviroment that the TI has NEVER had to expand the quantity and quality of the programs available for the TI. A place where beginners and experienced can come and share and learn and teach the ins and outs of the TI. It's a very unique machine that has quarks that make it so much fun to learn about and try to over come.