Extended BASIC Compiler

[sometimes99er]

Harry, - how about a dialect of HCHAR and VCHAR with 3rd parameter being a string ? Perhaps then not allowing the optional 4th repeat parameter - if it makes implementation that much easier.

[+Vorticon]

If you’re asking how big an XB program it will take, then that should be any size or about 24K. See post #16.

I don't think you can have the full 24K available because per that post the compiler run-time routines and the variables are stored in high memory alongside the program itself. Harry, how much space do the run-time routines occupy in high memory typically? From there, and knowing the number and type of variables a particular program has, one can estimate the maximum size that could be squeezed in high memory.

Edited December 4, 2012 by Vorticon

[senior_falcon]

Even if your XB program uses the full 24K it should be OK with the compiler with room to spare. An example:

10 CALL HCHAR(1,1,42,768) used 34 bytes in XB.

It uses 10 bytes in the compiled code plus another 6 for space reserved for the numeric constants.

L10 DATA HCHAR,NC1,NC1,NC2,NC3 and later:

NC1 DATA 1

NC2 DATA 42

NC3 DATA 768

So that's about half the size of the XB program, and that degree of space saving is pretty typical. The one caveat is that strings are the same size whether compiled or not, so a 24K text adventure might not fit.

[senior_falcon]

OK, now I have some real numbers for you. As they stand now, the runtime routines use 5404 bytes of code, leaving 19084 bytes free for your program. If the 2:1 reduction in size is typical (and I think it is), then a 24K program would be 12K plus 5.4K long-a total of a bit over 17K long, .

 

sometimes99er asked: "how about a dialect of HCHAR and VCHAR with 3rd parameter being a string ? Perhaps then not allowing the optional 4th repeat parameter - if it makes implementation that much easier."

How about CALL LINK("HDISP",ROW,COL,A$) or CALL LINK("VDISP",ROW,COL,B$) I can see how these might be useful, especially the ability to print vertically. Opinions? Comments?

Edited December 5, 2012 by senior_falcon

[Willsy]

How about CALL LINK("HDISP",ROW,COL,A$) or CALL LINK("VDISP",ROW,COL,B$) I can see how these might be useful, especially the ability to print vertically. Opinions? Comments?

 

I would suggest CALL LINK("PRINT@",ROW,COL,STRING$,DIR)

 

Where dir is an integer that represents the direction:

 

1 = from left to right

-1= move to the left after each character

32= move down one row after each character

-32=move up after each character

33=move to the right and down after each character

31=move to the left and down after each character

-31=move up and to the right after each character

-33=move up and to the left after each character

 

You get the idea. Basically, DIR is added to the VDP screen address after each character write. It's very versatile - the one routine can do all the above and more. For example, want a one character gap between each character of the string? Use a DIR of 2:

 

CALL LINK(1,1,"HELLO",2)

 

Would give H E L L O

 

I had this in my Mini-Memory TI BASIC extension (called The Game Writers Toolkit) back in 1991.

 

Here's a dis-assembly of the code from all those years ago (original source long lost!):

 

#################	  
# PRINT ROUTINE #	  
#################	  
   CLR  R0	     
   CLR  R3	     
   LI   R1,>0001   
   LI   R2,>7AAC   
   BLWP @NUMREF    
   BLWP @XMLLNK    
   DATA >1200	  
AG	 MOV  @FAC,R5    
   LI   R4,>0020   
   MPY  R4,R5	  
   INC  R1	     
   BLWP @NUMREF    
   BLWP @XMLLNK    
   DATA >1200	  
AH	 MOV  @FAC,R5    
   A    R5,R6	  
   INC  R1	     
   BLWP @NUMREF    
   BLWP @XMLLNK    
   DATA >1200	  
AI	 MOV  @FAC,R7    
   INC  R1	     
   LI   R5,AJ	  
   MOVB R5,@>7AAC  
   BLWP @STRREF    
   MOVB *R2+,R3    
   SWPB R3	     
   MOV  R6,R0	  
AK	 MOVB *R2+,R1    
   AI   R1,>6000   
   BLWP @VSBW	  
   A    R7,R0	  
   DEC  R3	     
   CI   R3,>0000   
   JNE  AK	     
   CLR  @GPLST     
   B    *R11	   

 

Funny to look at that code after all these years: Using a MPY to multiply by 32 instead of a shift! No factoring of the parameter fetching. Oh well. I was young, and it was written originally using the line by line assembler from code written in a tiny notebook!

[sometimes99er]

Yep, of course something that also works before compilation is better.

[senior_falcon]

Yep, it has to work before compilation, otherwise you can't test it in XB. That's the whole idea behind this compiler-to be able to test the program easily in XB and do all the error trapping there, then compile for the speed gain. These extras will be loaded to XB with CALL INIT, then CALL LOAD("DSK1.CXTRAS"). Then they can be accessed from XB with CALL LINK. The equivalent routines will be in the compiler and it will execute the code the same way.

[unhuman]

So - Harry - back when I was moving Bloxo-TI-z to your compiler - I found a bug. It had something to do with keypress / release not being detected. So the key stays down.

 

Can't find the post, but probably easier to find my source to see if I documented it.

 

-Howie

[senior_falcon]

There was a bug in my keyboard scanning routine. The key that was pressed (or -1 if no key was pressed) is placed by KSCAN in >8375. I retrieved that byte and put it in a register, then shifted it into the least significant byte. This gives the right value for the key except for when no key was pressed. In that case, instead of -1 it gives a value of 255. That's been fixed so it reports the right value. Doh!! I know better, I just don't always do better!

 

I put together an additional display routine combining suggestions from Willsy and Sometimes99er. CALL LINK("DISPLY",ROW,COL,A$) will print A$ on the screen in 32 columns starting at ROW and COL. Including an optional 4th parameter lets you tell the routine to print vertically, diagonally or backward (per Willsy's suggestion). An optional 5th parameter lets you print the string more than once.

[+Vorticon]

So will the compiler essentially run any XB program without modifications or are there limitations to note?

[sometimes99er]

There are several limitations. Take a look at the old DOCS. No floating point. Numbers are integers between -32768 and 32767. To have the XB program behave more or exactly like the compiled version, you can use INT here and there. RND has to be used like INT(RND*3) to generate numbers 0, 1 and 2. Delay loops may have to be adjusted. If I remember correctly the IF THEN only take linenumbers. Arrays can't be nested. A handful of functions not supported like DEF and SIN. No file processing.

 

The previous compiler was more like a TI Basic compiler. With COLOR, CHAR and multiple-statement lines more like XB. The new compiler will then have sprites, DISPLAY and ACCEPT (I guess). Other XB enhancements like subprograms and error handling will probably not be available (right away).

 

[unhuman]

Hey - this got me thinking.... How about a new delay command that sleeps for x ms. Would making timing games *MUCH* easier and far less arbitrary.

 

Of course, it would break regular XB.

 

Thanks - Howie

 

There are several limitations. Take a look at the old DOCS. No floating point. Numbers are integers between -32768 and 32767. To have the XB program behave more or exactly like the compiled version, you can use INT here and there. RND has to be used like INT(RND*3) to generate numbers 0, 1 and 2. Delay loops may have to be adjusted. If I remember correctly the IF THEN only take linenumbers. Arrays can't be nested. A handful of functions not supported like DEF and SIN. No file processing.

 

The previous compiler was more like a TI Basic compiler. With COLOR, CHAR and multiple-statement lines more like XB. The new compiler will then have sprites, DISPLAY and ACCEPT (I guess). Other XB enhancements like subprograms and error handling will probably not be available (right away).

 

[senior_falcon]

It's reasonable to want a compiler that can handle every XB program ever written and run it flawlessly. Such a compiler is theoretically possible, but you have to understand that the speed increase would be way, way less. In my (very limited) tests with floating point numbers there was about a 3x increase in speed using the compiler as compared to 20 to 40x for integer math. Every number would have to be treated as a floating point number if you want to be assured that an existing XB program would compile and run the same way. The choice is really quite simple: do you want speed or do you want floating point numbers with full compatibility. You only get to choose one.

[+Vorticon]

Speaking from my purist side, I'd rather go with full compatibility. A 3x performance boost would still constitute a HUGE improvement over current state for XB, while allowing a lot more creativity in programming.

In a perfect world, we would have two versions of the compiler: an integer version and a floating point version with full compatibility. But that's easy for me to say since I'm not the one doing the work .

Regardless, even the integer compiler with the currently implemented features will meet the needs of >90% of XB programmers, and we owe you a big thank you for reviving this project.

[senior_falcon]

Someone, I think it was Myarc, made an XB interpreter that ran completely in machine language with no GPL. I think it was claimed to be 2-3x faster which I can easily believe. That might be one option. Full compatibility ain't gonna happen on my watch. If someone wanted to write runtime routines for floating point, I would consider modifying the compiled code to allow that. But there would still not be any error trapping, named subprograms, IF THEN ELSE the way XB uses it. So you'd still wind up having to modify most programs for compatibility. What would be more interesting to me is to adapt the compiler to work with THE MISSING LINK.

[+retroclouds]

Harry can you explain us a bit about the internal workings of your compiler ? Are you running an interpreter when the compiled program is running? Also how are you allocating variables? How long did it take to write the compiler back in the old days? Besides for the program code, how much RAM are you using?

 

Ok, now for something crazy... Could you imagine having the compiled program running from the cartridge space? I mean sure still need RAM for storing variables. But it would be mighty cool having compiled programs running as a cartridge. Thinking about kaboom! and the other activision game Rocky was/is working on.

 

Myself I made an assembly language runtime library with a bunch of routines I used in Pitfall and Tutankham but doing a compiler is something I really want to try someday so am interested in learning how you did all this.

 

 

 

[Willsy]

I think I saw some of the source once. IIRC it's threaded code, the same as Forth. However, instead of a data stack to carry parameters, parameters are stored in-line, and each word (e.g. HCHAR) is responsible for getting its own parameters.

 

I'm sure our Harry will spill the beans. In fact, it would be cool if the source could be released when finished. We could all learn something!

[senior_falcon]

I am tracking down a glitch in the PRINT routine. DISPLAY works perfectly, but PRINT doesn't. It used to, so I obviously made an "improvement". Don't have a lot of time to respond to Willsy and Retroclouds, but here's a quick description. More will come once this thing is finished and I have more time.

Here's a BASIC program:

5 A$="TESTING"

10 FOR I=1 TO LEN(A$)::CALL HCHAR(10,I,ASC(SEG$(A$,I,1)))::NEXT I

20 GOTO 20

 

This compiles to:

DEF RUN,CON

RUNEA B @RUNEA5

FRSTLN

L5

DATA LET,SV1,SC1

L10

DATA LEN,SV1,NT1

FOR1

DATA FOR,NV1,NC1,NT1,ONE,0,0

DATA SEGS,SV1,NV1,NC1,ST1

DATA ASC,ST1,NT1

DATA HCHAR,NC2,NV1,NT1

DATA NEXT,FOR1+2

L20

DATA GOTO,L20

LASTLN DATA STOP

OPTBAS DATA 0

NC0

ZERO DATA 0

ONE DATA 1

PI DATA 3

RND DATA 0

NC1 DATA 1

NC2 DATA 10

NV0

NV1 DATA 0 I

NT1 DATA 0

SC0

SC1 DATA SC1+2

BYTE 7,84,69,83,84,73,78,71

EVEN

SV0

SV1 DATA 0 A$

ST1 DATA 0

SA0

NA0

FRSTDT

LASTDT

EVEN

COPY "DSK3.RUNTIME.TXT"

END

 

Here's the key lines of code:

LI R13,FRSTLN start program from 1st line

(then some other lines)

 

RTN LIMI 2 the main loop, all subs come back to here

LIMI 0

MOV *R13+,R12

B *R12 r12 has code. i.e. GOTO, SPRITE, etc.

 

Starting at the first line of code the compiler fetches the address of the next routine and puts it into R12, then branches to that address. Then it's up to the routine to retrieve the strings and numbers and do something with them, when finished with them it returns to RTN with R12 pointed to the pointer to the next instruction.

 

That's not a very detailed description, but I will get into it more in the weeks to come.

[Willsy]

Yep. It's a direct-threaded interpreter. Cool.

[senior_falcon]

"Direct-threaded interpreter." My vocabulary just got a bit larger! OK, here's a question about terminology. I have been calling the support routines the "runtime routines". Is that correct? Would they be described better as the "interpreter"? Or something else?

[Willsy]

Runtime routines sounds okay to me.

 

The interpreter is the small loop that you have that fetches the address of the next routine to execute and branches to it.

 

Mark

[RXB]

The best approach for speed is to stay with integer unless the number needs to be Floating Point.

The compiler should be able to pull this off as an example:

100 A=10 ! Integer number

110 B=1E1 ! Floating Point number

120 C=82.7 ! Floating Point number

130 D=72 ! Integer

140 E=A*B+C/D ! Floating point as any number that is Floating Point now becomes Floating Point

150 F=D/A ! Integer as no Floating Point numbers are used

 

How I would approach the problem is how the numbers are stored determine if Floating Point is required or not, thus the Compiler handles the mess.

[Tursi]

One thing you could try, in theory, is tagging the variables for floating point.

 

For instance, in Applesoft, "A" was a floating point variable, but "A%" was integer. Offhand I can't remember all the punctuation marks that are legal in TI BASIC variable names, but a similar method could be used.

 

Since the general intent of the compiler is to use integer variables, I'd use the tag to indicate floating point. Possibly adding a switch early on (in a REM statement?) to optionally make the default floating point instead. this way it still works in XB. For example:

 

100 REM BY DEFAULT, VARIABLES ARE INTEGER UNLESS TAGGED
110 A=5 :: REM A STANDARD INTEGER VARIABLE
120 B_=2.2 :: REM A FLOATING POINT VARIABLE, TAGGED BY UNDERSCORE

 

100 REM COMP_OPTION: FLOATS
110 REM ABOVE COMMENT TELLS THE COMPILER TO USE FLOATS
120 A=5 :: REM NOW THIS WILL BE COMPILED AS FLOATING POINT
130 B_=2.2 :: REM SO WILL THIS

 

A suggestion for your consideration.

[RXB]

TI Extended Basic already tags all integers and Floating point numbers in any format MERGE or Program.

 

 

* EQU >C6 spare token

STRINZ EQU >C7 QUOTED STRING

UNQSTZ EQU >C8 UNQUOTED STRING

NUMZ EQU >C8 ALSO NUMERICAL STRING

NUMCOZ EQU >C8 ALSO UNQUOTED STRING

LNZ EQU >C9 LINE NUMBER CONSTANT

* EQU >CA spare token

 

This is from the GPL Source code of Extended Basic.

The first byte of code after a >C8 is the length byte, so if the string contains a E (Exponent) or . (period) then it is a Floating point number.

If not then it is a Integer only. This was how I was writing my XB compiler. (Any number over 32768 or -32767 is Floating Point)

A DATA statement using strings is where it gets real complicated and why I thought of using the unused tokens >CA and >C6 in RXB.

Edited December 11, 2012 by RXB

[unhuman]

Personally, I'm after integer math. Not to say I'd never use floating point, but...

 

I certainly prefer Tursi's method. I wouldn't want to promote anything unwantingly to floating point. I would do a lot of math in a larger space and then divide down to integers - to give greater precision and then cut it down for actual usage.