Jump to content
IGNORED

StrangeCart


speccery

Recommended Posts

I've been very busy in the last month or so with work, no progress with any of my TI related projects. But now I have a long weekend, with some time to work on the strangecart and I also plan to work on the grommy2 firmware.

 

I've refactored the strangecart firmware somewhat, this has been a background project, I started it already a couple of months ago. It's been slow going progress due to lack of time. I've reached a point where the code is somewhat easier to work with.

 

Anyway, I've now been able to get some progress done with the Basic interpreter. These are not only visible internally to the code but also providing additional functionality. One of the open points in the strangecart Basic was the lack of proper handling of INPUT statements. That's now in much better shape, supporting proper handling of input prompts and input to multiple variables. I still need to improve the line editing functionality, it's now super simple and does not support any editing ;)  but I will change that. The strangecart firmware already has a line editor which is used when communicating with it using the serial port (over USB). That line editor also supports command history features. As part of the refactoring I ported that code from my previous C-language version to C++. I will need to see if I use that as the basis for the input line editor, or if I create another version. 

 

Having a line editor will also be necessary to add in the future tokeniser (i.e. conversion of input text to Basic lines). I want to do that since I want the strangecart be able to also offer interactive Basic mode. Once I have that, I want to add support for 40 column text mode and also support 80 column mode with F18A. My intention still is to resist going wild with extensions - tempting as those are, but rather make the Basic programming and debugging experience friendlier.

 

The bottom line is that there is progress, and hopefully quite soon a new version of the firmware. There aren't many strangecarts out yet, but if you have feature requests please let me know!

Edited by speccery
  • Like 6
Link to comment
Share on other sites

1 hour ago, TheBF said:

Did you ever find a use for Mecrisp or was that a side project?

I did test it a little. I tested it on the grommy2 board but I didn’t try to get the actual GROM functionality running it. I think it is interesting and very useful for brining boards up, but I am so much more productive with C++ that at least for now I am keeping it on the side. I am still considering making it available on the StrangeCart, but if I do it it would rely on the existing code for communication with the TI, so I would need to integrate it.

  • Like 2
Link to comment
Share on other sites

I released new firmware for the StrangeCart, please see description at the Github wiki page.

The new firmware is available here.

The updates are relatively minor on the outside, but internally more significant. The big one is a more comprehensive INPUT handler with a line editor with command history memory. One thing I forgot to mention in the Wiki is that the line editor currently works a little differently from the one we're used to with the TI: FCTN+1 works like backspace (delete to the left) rather than delete. Insert mode is on all the time.

Link to comment
Share on other sites

  • 1 month later...

I have continued to work on the StrangeCart BASIC over the past several weeks. I haven't recently had the time work on the hobby for extended amounts of time in one go, with these constraints working on software seems to be somewhat easier than hardware (I have several things to do there such as new picocart PCB layout). 

 

Anyway I realised that what I was missing was the thing BASIC is known for - immediacy i.e. command mode. Up til now what you could have been doing with the StrangeCart is to load an existing TI BASIC program and save it to the memory of the StrangeCart, and use its CALL RUN to have run the BASIC program very fast. In other words, BASIC program editing would be done with regular TI BASIC; access to StrangeCart has been a bit indirect, as save to the flash memory of the board has the side effect of making the program available for the StrangeCart BASIC interpreter.

 

Earlier, the firmware has thus treated the BASIC program as a ROM: even though my interpreter can run the BASIC program, it could not modify it. I am now in the process of adding a tokeniser and building blocks for editing the loaded program. The revised BASIC has already now a command interpreter, which can be entered and it runs entirely on the StrangeCart. Within the interpreter there are bunch of standard BASIC features already working:

  • RUN command enables running the program (nothing new)
  • LIST command enables listing the program (supports now ranges of lines)
  • OLD enables loading a BASIC program from the SCD1 drive
  • BYE exits the command interpreter

Those commands were already there in various forms, but now:

  • Entering NEW clears the BASIC program and variables (yes, progress comes in small steps)
  • Entering a line number alone deletes the line from memory
  • Entering a line number and code after it tokenises the code and inserts the code line into the right place of the program.

These sound trivial additions, but the size of the BASIC and other parts of the firmware expanded by almost 1000 lines of C++ to support these features.

 

Tokenising TI BASIC lines takes a bunch of code, and the memory management of altering BASIC programs in memory is also a bit tedious: I am maintaining in the StrangeCart's memory the BASIC program in normal TI BASIC save format: there is a 8 byte header, followed by the line number table (backwards as in the VDP memory) followed by the program code (also backwards, highest numbered line of code in the lowest memory address). The only change I implemented is that while in VDP memory the BASIC program size is restricted to 16K (in practice less) and is loaded near the top of the 16K space, now after loading a I modify the 16-bit line pointers so that they are zero based relative to the memory address where the BASIC program is stored. This is reversible - it simply means subtracting a fixed memory offset from every line number table address entry. That can be added back, and if subsequently saved the program can be loaded back to normal TI BASIC (I haven't tested it yet but I am sure I can make it work since the format is the same). But the plus side is that the tokenised program size can extend up to 64K. The compatibility at the moment with the use of 16-bit line offsets means that 64K is the limit for tokenised code - data areas are on top of that.

 

When fully completed and the onboard command mode is finalised, these changes mean that I can support different video modes with ease. My intention is to do this so that when one enters the StrangeCart BASIC command interpreter, the 16k VDP memory contents are copied to the StrangeCart's memory for backup, and then I will be free to mess around with VDP memory to support any video mode - including F18A 80 column mode - for TI BASIC programs. When returning from the StrangeCart BASIC back to TI BASIC I intend to restore the state of the VDP.

 

Even with these limited features I have had a lot of fun playing with the command interpreter while working on the code. The command mode makes it self contained. In fact I am testing the interpreter without even plugging the board into the TI, since the command interpreter also supports operation over serial line.

 

Edited by speccery
  • Like 8
  • Thanks 1
Link to comment
Share on other sites

Unfortunately I have like zero time to work on TI projects lately. In my Stevie editor I have TI-Basic integration working, so getting something like that combined with the pico cart would be real fun. My dream is to have a full-screen basic editor, where I can jump between multiple “sessions”. 

I have it working for the most part, if you look around in the forum you can find my Stevie development thread with some videos. 

Jump into TI Basic, load or type a program, jump back to Stevie and the TI basic program gets detokinzed and shown in the editor, including file name it was loaded from. Hooked into the ISR to keep track of what is happening while in TI Basic.

 

Didn’t have time to work on the tokenizing yet, but that will come.

 

Having the possibility to run TI-Basic at hyperspeed would be magic.

I really think that TI Basic is underestimated, because of XB and its enhancements there.

For me jumping into TI Basic and back out, without losing the program in VDP, quitting to the TI title screen, back into the editor, and “resuming’ the TI Basic session by restoring VDP contents what an eye opener. This is all done with SAMS, so it was just a question of switching pages.

 

Anyway, looking forward how your picocart further develops. 

  • Like 2
Link to comment
Share on other sites

Note that TI also had the option of saving TI BASIC programs into the GROM3-7 space and executing it. A lot of early programs from TI used this trick, as did most of the Scott Foresman cartridges. Decoding the method and using that option would let you save programs of up to 40K into that space and actually run them as a cartridge on a standard, unexpanded console. IIRC, the method is somewhat described in some of the technical specifications.

  • Like 4
Link to comment
Share on other sites

59 minutes ago, Ksarul said:

Note that TI also had the option of saving TI BASIC programs into the GROM3-7 space and executing it. A lot of early programs from TI used this trick, as did most of the Scott Foresman cartridges. Decoding the method and using that option would let you save programs of up to 40K into that space and actually run them as a cartridge on a standard, unexpanded console. IIRC, the method is somewhat described in some of the technical specifications.

Thanks @Ksarul I remember this being mentioned by someone (you?) in some call or at least in the forums. Is there an example cartridge somewhere I could take a look at? I would be interested in taking a look. Would be a great match for this project - and a nice way to compare performance as the StrangeCart can both emulate GROMs allowing in cartridge mode and-  if I can extract the BASIC program - I could run it using it's BASIC interpreter.

  • Like 2
Link to comment
Share on other sites

4 hours ago, HOME AUTOMATION said:

Thanks @HOME AUTOMATION that's great. Looking forward to studying this with a bit more time, but I checked this quickly by loading it in js99er.net, launching it and looking up a few things in TI Intern book. When launched, the line number table pointers at @>8330 and @>8332 are set to C03C and D24B respectively. Looking briefly at those addresses in the file (hexdump -C file | less) and noting the >6000 offset as this GROM stuff is loaded at address >6000 in GROM space, there are valid looking line number table entries in there. The highest numbered lines there are 26140, 26040, 26030, with respective line pointers to >8830, >c038, >8163. These address clearly are not in VDP space but would be valid GROM pointers. I looked at the tokens on those lines as well, last line was a RETURN which makes sense, the 2nd to last an assignment which also makes sense.

  • Like 2
Link to comment
Share on other sites

9 hours ago, Ksarul said:

Note that TI also had the option of saving TI BASIC programs into the GROM3-7 space and executing it. A lot of early programs from TI used this trick, as did most of the Scott Foresman cartridges. Decoding the method and using that option would let you save programs of up to 40K into that space and actually run them as a cartridge on a standard, unexpanded console. IIRC, the method is somewhat described in some of the technical specifications.

@Ksarul, if memory serves, I believe that Craig Miller had a utility on the 2nd GramKracker utility disk that would move a basic program into those GRAM spaces?

  • Like 4
Link to comment
Share on other sites

16 hours ago, HOME AUTOMATION said:

Perhaps a bit OT, but I had considered the idea ...that by utilizing GRAM, and a program running from ROM pages, a CALL, could be added to BASIC, that could somehow crunch the BASIC program into GRAM/allow for longer BASIC programs.:ponder:

This is a very nice idea, and something the StrangeCart could "easily" do. I will keep that in mind. It would be cool to have both the ability to execute code from GROM cartridges as well as save programs to GROM cartridges. This should mean that GROM spaces 3-7 could be used for BASIC, and if the whole 8k GROM size was used for each GROM slot, actually the TI BASIC programs could be larger than TI Extended BASIC programs... 40K of BASIC code and the entire VDP memory for variables. @pixelpedant could be interesting for your amazing TI BASIC projects - of course then we are no longer talking about something running on the bare console, but at least it would be still compatible with the console BASIC and a GROM only cartridge.

Edited by speccery
  • Like 1
  • Thanks 1
Link to comment
Share on other sites

11 hours ago, speccery said:

This is a very nice idea, and something the StrangeCart could "easily" do. I will keep that in mind. It would be cool to have both the ability to execute code from GROM cartridges as well as save programs to GROM cartridges. This should mean that GROM spaces 3-7 could be used for BASIC, and if the whole 8k GROM size was used for each GROM slot, actually the TI BASIC programs could be larger than TI Extended BASIC programs... 40K of BASIC code and the entire VDP memory for variables. @pixelpedant could be interesting for your amazing TI BASIC projects - of course then we are no longer talking about something running on the bare console, but at least it would be still compatible with the console BASIC and a GROM only cartridge.

Why TI Basic and not GPL? 

I mean if you focused on GPL you could run most carts faster while TI Basic is so limited, no sprites and without Assembly impossible to do anything very impressive.

And if focus was on GPL instead of TI Basic you could run from bare console only.

Using FinalGROM and RXB I proved you can get Assembly to work on bare console only.

 

Tursi originally told me way back that you were working on speeding up GPL for all carts, so I was disappointed it was only TI Basic.

Your efforts are very impressive for TI Basic, but again what could a 40K TI Basic program do compared to XB?

No sprites, no Accept AT, no Display AT, no ON ERROR, CALL SAY, SUB, SIZE and many other built-in commands in XB.

 

Maybe your next project could be GPL?

  • Like 1
Link to comment
Share on other sites

On 6/6/2023 at 8:48 PM, RXB said:

Why TI Basic and not GPL? 

I mean if you focused on GPL you could run most carts faster while TI Basic is so limited, no sprites and without Assembly impossible to do anything very impressive.

Thanks for the comments, good questions. This reply became rather long...

 

From a motivational perspective, I have never written a language interpreter before, so I found it interesting to do an interpreter which would be compatible with TI BASIC. Well I have written Forth implementations, but they are not BASIC so perhaps they don't count. Writing a GPL interpreter would be far simpler than this Basic project, but the challenge would be how to integrate it with the rest of the TI software.

 

You know, I really have to restrain myself from going crazy with Basic extensions. So far I have only implemented one extra command called VARS -  it will show all currently declared variables, their dimensions and values for scalars - so primarily useful for debugging both the interpreter and BASIC programs. I have also declared two new tokens for functions: ADDR to find the address of a variable and FRE to return the amount of free memory in various areas; but I haven't yet implemented these two. Anyway with the basis I have now in place it should not be hard to incorporate Extended Basic functionality, but I will probably still first work on TI Basic to try out some things I am interested in: improved debugging support and programming support in general, token formats, JIT compilation to intermediate format and perhaps compilation from TI Basic to TMS9900 machine code directly on the StrangeCart. I am also thinking if it would make sense to have a built in TMS9900 assembler available, similar to how the BBC Basic works for the 6502 and Z80. This would make it possible to mix Basic and assembly in the same source file.

 

I have allowed myself to spend some time to make a few simple optimisations, which resulted in 20% speed increase in Noel's BASIC program (I don't remember his handle here). The benchmark which has in total 10000 iterations and the inner loop has one assignment now runs in about 0.166 seconds on the StrangeCart. According to the chart I have this takes 77 seconds on the TI-99/4A, I don't remember anymore if this is with extended Basic or regular Basic. Anyway the interpreted Basic performance with floating point variables is pretty much on par with the speed of machine code instructions on the TMS9900. The current interpreted performance is limited by the token format TI used, which is really stupid from a performance point of view. A lot of time is spent just finding out how many characters each variable name is. I may replace this with my own tokens internally used by the Basic, so that variable references would be fixed length and directly lead to the symbol table entries. If I do that I still want to be able to load and save normal TI Basic programs.

 

Coming back to the question on GPL vs TI Basic, a GPL interpreter running on the StrangeCart would certainly be massively faster than what the ROM implements, but it would be a different kind of project, the challenge would be how to integrate that processing with the TI-99/4A software infrastructure so that it's compatible with existing GPL code. I have studied speeding up the GPL interpretation in my icy99 FPGA project. In that project I added a couple of new instructions to my TMS9900 compatible CPU core to handle GPL address mode decoding in hardware. That was interesting but added a lot of complexity to the CPU core; it did condense many TMS9900 instructions into one and immediately boosted GPL performance.

 

A final point is that I am able to compile and run my Basic interpreter on multiple platforms: I develop and debug it as a macOS command line application, but I have also used it on a Raspberry Pi, and I also ported it to STM32G431 microcontroller in addition to the StrangeCart. But the usefulness on other platforms that the StrangeCart in a TI-99/4A is limited, since most existing I/O and UI operations (like CALL HCHAR) only make sense in a TI. But adding some extensions would make it possible to use this Basic in microcontroller projects.

  • Like 6
  • Thanks 1
Link to comment
Share on other sites

A good portion of XB3 is Assembly in the ROMs so I have been disassembling XB3 as there is no source code.

So far I have 90% of ROM 1 done and 30% of ROM 2 done.

With the GPL and ROMs source I can make XB way faster as more Assembly replaces GPL and is 100% backwards compatible.

If GPL could be speeded up it would kill my task entirely as useless.

  • Like 2
Link to comment
Share on other sites

1 hour ago, RXB said:

If GPL could be speeded up it would kill my task entirely as useless.

At least I don't see it that way. When we think about speeding up GPL in the broad sense, it appears there would be three ways to go:

  1. Convert GPL to TMS9900 machine code, which is what you have been doing. The beauty of this is that all you would need in addition to the bare computer is a ROM/GROM cartridge (I guess ROM only if everything was converted from GPL to assembly). Also, this approach is "era correct" since you're using the original CPU etc. aside from potentially using higher density memory chips RXB could have existed back in the day with the same good performance.
  2. Using a coprocessor, which is what the StrangeCart effectively is. From a software perspective that replaces the problem of execution speed with a software integration issue as I wrote above. Definitely not an "era correct" approach since these chips - if they existed back in the day - would have been in the supercomputer territory in 1979. 
  3. Enhance the GPL execution speed without using a coprocessor. One "simple" way of doing it would be to replace the system ROMs with a higher performance implementation of the GPL interpreter - of course not in practice simple since system ROM replacement is not a plug-and-play job but requires a pretty involved soldering job; and the development of such an optimised GPL interpreter which almost certainly would need much more ROM memory than the 8K we have.

As a variation of 1 and 3, have you considered creating a compiler from GPL to TMS9900 machine code? I have been sometimes thinking if this would be doable. I wonder if something like that has already been created, perhaps even back in the day?

  • Like 1
Link to comment
Share on other sites

3 hours ago, RXB said:

A good portion of XB3 is Assembly in the ROMs so I have been disassembling XB3 as there is no source code.

So far I have 90% of ROM 1 done and 30% of ROM 2 done.

With the GPL and ROMs source I can make XB way faster as more Assembly replaces GPL and is 100% backwards compatible.

If GPL could be speeded up it would kill my task entirely as useless.

What is XB3 ? Is that the myarc enhanced version?

Link to comment
Share on other sites

5 hours ago, speccery said:

At least I don't see it that way. When we think about speeding up GPL in the broad sense, it appears there would be three ways to go:

  1. Convert GPL to TMS9900 machine code, which is what you have been doing. The beauty of this is that all you would need in addition to the bare computer is a ROM/GROM cartridge (I guess ROM only if everything was converted from GPL to assembly). Also, this approach is "era correct" since you're using the original CPU etc. aside from potentially using higher density memory chips RXB could have existed back in the day with the same good performance.
  2. Using a coprocessor, which is what the StrangeCart effectively is. From a software perspective that replaces the problem of execution speed with a software integration issue as I wrote above. Definitely not an "era correct" approach since these chips - if they existed back in the day - would have been in the supercomputer territory in 1979. 
  3. Enhance the GPL execution speed without using a coprocessor. One "simple" way of doing it would be to replace the system ROMs with a higher performance implementation of the GPL interpreter - of course not in practice simple since system ROM replacement is not a plug-and-play job but requires a pretty involved soldering job; and the development of such an optimised GPL interpreter which almost certainly would need much more ROM memory than the 8K we have.

As a variation of 1 and 3, have you considered creating a compiler from GPL to TMS9900 machine code? I have been sometimes thinking if this would be doable. I wonder if something like that has already been created, perhaps even back in the day?

Thanks though all three have major issues.

1. Conversion of GPL to TSM9900 machine code is a memory hog that takes twice the memory and you forget cartridge ROM is slower then 16 bit ROM OS memory.

2. Using a coprocessor for GPL would be way faster and could use even faster RAM then the 16 bit ROM OS memory and bypass GROM one.

3. I do not think there is much you could do to optimize GPL without screwing up how it works.

 

I personally hate compilers unless they work like Forth where you get both Interpreter and compiler so you have best of both worlds.

Interpreter's are slower but are flexible, while compilers add extra steps to just make the programs work.

Interpreter's are like digging a hole in forest with a shovel, compilers are like having to get the backhoe drive it around the trees, it is faster but way more extra steps.

Link to comment
Share on other sites

4 hours ago, retroclouds said:

What is XB3 ? Is that the myarc enhanced version?

Sorry no XB3 was from ASGARD written by Winfred Winkler that has 3 cartridge ROMs that had some backward compatibility issues.

95% of XB worked fine with it and it was faster than XB version 110.

XB3 had a whole bunch of math routines updated and he added many new commands.

  • Like 1
Link to comment
Share on other sites

2 hours ago, RXB said:

Thanks though all three have major issues.

1. Conversion of GPL to TSM9900 machine code is a memory hog that takes twice the memory and you forget cartridge ROM is slower then 16 bit ROM OS memory.

2. Using a coprocessor for GPL would be way faster and could use even faster RAM then the 16 bit ROM OS memory and bypass GROM one.

3. I do not think there is much you could do to optimize GPL without screwing up how it works.

 

I personally hate compilers unless they work like Forth where you get both Interpreter and compiler so you have best of both worlds.

Interpreter's are slower but are flexible, while compilers add extra steps to just make the programs work.

Interpreter's are like digging a hole in forest with a shovel, compilers are like having to get the backhoe drive it around the trees, it is faster but way more extra steps.

Thanks for the comments.

1. I am very much aware that cartridge ROM is slower than 16-bit ROM OS memory. But the conversion to machine code results in such a performance boost, that the 8-bit accesses won't be a problem performance wise. Very true that it results in a big increase of memory footprint.

3. I think the GPL interpreter can be be optimised, but like you say, it would be very easy to screw it up. I have first hand experience of this: when I built the FPGA version of the TMS9900 which had extra instructions to speed up GPL execution, I did modify the OS ROM to support the new instructions. I did get it to work, but I only touched on a very small section of the GPL interpreter.

 

I very much like compilers and they do an amazing job at compiling very expressive modern languages into very efficient machine code. I have in the past written a lot of heavily optimised code professionally over many decades. I oftentimes had to resort to assembler programming to get the performance (or actually cost effectiveness) where it needed to be. I would often develop the algorithm in C or C++ using the great debuggers available for them, then compile the code to assembler to get the boilerplate in place, and finally hand optimise key sections of the code. Hand written code was for a long time the only way to get good access to SIMD instruction sets in modern processors. I wrote code for some TI DSPs, but mostly for Intel MMX, SSE, SSE2, SSE3 and SSSE3 instruction sets. The performance gains were phenomenal, certainly the difference between a proof of concept and actually commercially viable product. I found it very interesting to see how compilers generated code and what the compiled code looked like, it is also a very useful skill to have when debugging crash dumps, where you often just have a binary blob and the address of the instruction which blew things up.

  • Like 3
Link to comment
Share on other sites

2 hours ago, RXB said:

I personally hate compilers unless they work like Forth where you get both Interpreter and compiler so you have best of both worlds.

Interpreter's are slower but are flexible, while compilers add extra steps to just make the programs work.

Interpreter's are like digging a hole in forest with a shovel, compilers are like having to get the backhoe drive it around the trees, it is faster but way more extra steps.

Your words do not reflect reality and the metaphor is rather lame.
The extra steps to compile are so quick and easy to execute that the least gifted user can access. The very few user actions during the process a near always the same that could be done closed eyes, and even could be easily automatized .
Compiled, programs run smoother and very quickly, permitting to create more elaborated programs, things that were impossible before.
Compilation is acclaimed by most of the current users, because it bring the most important feature that missed them during old days and they want. To realize it one just counts the number of programs made and converted with XB256 compiled  and the very few (not to say infinitesimal), made with RXB  that doesn't offer the feature.

In my point of you, you can add all the subprograms you want in RXB, maybe it could be interesting or practical for some usage, but it will always runs too slow and will continue to be less attractive.

I understand your position that intends to defend your product and the purpose you want for it, it's normal and I respect it. But please, be objective.

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...