Stevie Development Thread

[GDMike]

We, you, us. Need a good bank switching routine. Even if it's a 2K move or whatever, something that takes our input address, a buffer size and a way to switch in/out of it by altering a variable..lol

[+retroclouds]

Here's a new demo. It's Stevie loading an assembly source file from github.

 

 

[+Vorticon]

Very cool. Would it be possible to save a file back to the Github?

[+retroclouds]

2 hours ago, Vorticon said:

Very cool. Would it be possible to save a file back to the Github?

 

Short answer; no.

 

Due to the complexity involved I don’t see it happening. Unless using the tipi for doing all heavy lifting.

Long answer included below.

 

https://atariage.com/forums/topic/308551-dx10-home-computer-software-development-system/?do=findComment&comment=4596872

[+Vorticon]

7 hours ago, retroclouds said:

 

Short answer; no.

 

Due to the complexity involved I don’t see it happening. Unless using the tipi for doing all heavy lifting.

Long answer included below.

 

https://atariage.com/forums/topic/308551-dx10-home-computer-software-development-system/?do=findComment&comment=4596872

I was indeed thinking of the tipi doing all the work, but unfortunately I don't have a good understanding of the underlying work involved in such a task. It would have been so awesome though because then the documents created with STEVI could be backed to the Github with no fear of local hardware crashes and loss of data...

[Asmusr]

4 hours ago, Vorticon said:

I was indeed thinking of the tipi doing all the work, but unfortunately I don't have a good understanding of the underlying work involved in such a task. It would have been so awesome though because then the documents created with STEVI could be backed to the Github with no fear of local hardware crashes and loss of data...

Connecting the TI via the TIPI to cloud storage, e.g. Google Drive as I have done in JS99er.net, would probably be quite simple, but connecting to git via the TIPI would require a much more advance API than a DSR to be useful. You would have to be able to resolve merge conflicts, for instance.

[+retroclouds]

5 hours ago, Vorticon said:

I was indeed thinking of the tipi doing all the work, but unfortunately I don't have a good understanding of the underlying work involved in such a task. It would have been so awesome though because then the documents created with STEVI could be backed to the Github with no fear of local hardware crashes and loss of data...

If you are thinking more about backup and local data loss of files created and edited on the TI-99/4a, then there surely are other solutions to the problem that do not require git or github.
I've put it on my "mental" to-do list for Stevie. Could imagine having a Stevie plugin that controls a (still to implement) service on the tipi

[+Vorticon]

20 minutes ago, Asmusr said:

Connecting the TI via the TIPI to cloud storage, e.g. Google Drive as I have done in JS99er.net, would probably be quite simple, but connecting to git via the TIPI would require a much more advance API than a DSR to be useful. You would have to be able to resolve merge conflicts, for instance.

How does one access Google Drive with the TI and TIPI? That would work just as well.

[Asmusr]

8 minutes ago, Vorticon said:

How does one access Google Drive with the TI and TIPI? That would work just as well.

In JS99er I'm using the Google Drive HTTP API, and the TIPI software on the RPi could possibly do that as well, but perhaps there is an out-of-the-box solution where you just map a directory on the RPi to your Google Drive? Anyway, it's probably better to discuss that in a TIPI thread.

[+FarmerPotato]

42 minutes ago, Asmusr said:

Connecting the TI via the TIPI to cloud storage, e.g. Google Drive as I have done in JS99er.net, would probably be quite simple, but connecting to git via the TIPI would require a much more advance API than a DSR to be useful. You would have to be able to resolve merge conflicts, for instance.

I guess it would be nice to have a function on the TI end to tell the Pi to 'git pull' or 'git add, git commit'. In case of any merge conflicts, abort and you'd have to login to the PI anyway. That's still useful.

 

I guess Stevie can open the txt as DF128? no DV80 involved? so native format CRLF and all? Then write back to a TIPI disk as native format?

 

For other editors,  If the file conversion was managed,  I would be happy with a step where the file lived on the PI volume, and I used cmd-line login to the PI to use git.

 

I could imagine a git wrapper on the PI that kept both TXT and DV80 versions. It converts the whole TXT directory over to DV80 TIFILES after a pull. It converts the whole DV80 TIFILES directory to the TXT directory before a commit/push.

 

How I use gitlab now:

 

I currently edit on a PC, assemble with xas99, 'make' into a dsk image with xdm, test in Classic99, commit to Gitlab on the PC command line.

 

So, no DV80 files in that flow. However, with some TIPI in the mix, I could usefully switch back and forth from PC to 4A for any debug/edit/commit cycle.

 

At the very least, I commit the dsk image to gitlab, so it could be pulled onto the TIPI from the cmd line.

(but right now  I am still loading it onto a CF card with Ti99dir, then using 4A+CF7+. I fried my TIPI with 12V.)

 

Not asking anybody else to write software! If I get my TIPI fixed, I'll try to write that git wrapper (I already wrote a nice DV80 extractor in C)

 

[+9640News]

1 hour ago, FarmerPotato said:

Not asking anybody else to write software! If I get my TIPI fixed, I'll try to write that git wrapper (I already wrote a nice DV80 extractor in C)

 

I just went through an exercise of using VBA Excel to write DIS/VAR 80 files out to the PI so the TI/Geneve could read it with the TIPI.  On the TI side of things, the DSR does the heavy lifting of writing the files out to the file.  On the PC side of things, one has to write out the 128 byte TIFILES header with the appropriate sector length/record counts.  And, then, you must have the records stay within each subsequent 256 byte block with each record having a line length byte.  Finally, the final record must be terminated properly.

 

Anyways, just saying there is some code involved.  I have VBA macro code in my FileCreator spreadsheet for the AfterHours BBS source that showed my logic.  Basically, each sector was a 4 line x 40 character max input so that for each file record/description, it took one sector.  For 3000 files, it took 3000 sectors.

 

It is likely you already know what I mentioned, but if not, at least my VBA Excel logic is in that spreadsheet if it would help you if you decide to write something turns something back into a DIS/VAR 80 file.  I'm sure my VBA code could have been improved, but it works for my test case...…...

 

Beery

 

[+retroclouds]

I’ve started working on saving DV80 and well, what can I say.... I didn’t like it. Too much hassle swapping memory.

 

Due to the fact that I’m using my spectra2 library for doing the heavy lifting, I’m making extensive use of scratchpad memory. The library is used for memory copy, keyboard reading, all things VDP, task handling, etc.

Original purpose of the library was to run cartridge based games on an unexpanded Ti-99/4a console.

 

Using scratchpad by itself is not an issue, as long as you keep away from DSR or anything else related to the TI-99/4a itself. The solution I currently have in place when reading files, is that I just page out scratchpad memory, page-in scratchpad memory as required for dsrlnk and DSR file-access.

 

No need to say, you’re pushing plenty of memory. Doing a rough calculation, I’m about between 1K and 1,5K for each record read. It shows the power of the 9900 CPU and storage devices like HRD4000B and TIPI that I’m still getting good performance out of it.

 

But honestly pushing between 3 and 4,5 MB when reading a DV80 file with 3000 records is something I’m not fond of. Obviously you can state that most files are A LOT smaller and you don’t even notice it. But it matters to me.

 

With that in mind it’s back to the drawing board. The SP2 library is using equates allover the place. 

So it should be not too big of an issue to move the SP2 work memory from scratchpad to low memory expansion.

This introduces additional wait states, but I don’t think I’ll ever notice that working with the editor. 

If it does I will have to do the required CPU cycle counting.

 

With SP2 reworked, I can then throw out the scratchpad memory swapping when reading and writing DV80 files.

Expecting a decent performance boost out of that.

 

Other than that, I’m also enhancing my keyboard polling routine, so that it also detects individual “non-standard” keys like alpha lock down, left shift, left fctn, left ctrl, etc.

 

So in a nutshell, it will take some more time before I release the first Stevie beta, but it will be worth it (at least to me).

 

On a sidenote, there will be a future version of spectra2 that will be “compatible” with Client Pulleys C. 
I will also making necessary interfaces so that the spectra2 functions can be called from C.
What I want to do at a later time is integrate the C compiler into the editor.

 

Do am aware that most are using gcc for cross-compiling now.

Then again, having an integrated C environment on the TI-99/4a itself, is something I really like to have. 

 

But first things first... ?

[Tursi]

I've done the "swap in scratchpad for DSRs" before too.. but, the scratchpad is only 256 bytes. Why do you need to move 1.5k /per record/? Even if you did all of scratchpad, that's 512 bytes, plus 80 bytes max for the D/V80 record?

 

That said, certainly nothing wrong with your plan to move stuff over to regular RAM. Scratchpad really only offers a large benefit for frequently accessed data - if you only hit it now and then you'd be hard pressed to see the difference. It's only 4 cycles slower to read and 8 cycles slower to write - still faster than a single 9900 instruction.

 

[+retroclouds]

9 hours ago, Tursi said:

I've done the "swap in scratchpad for DSRs" before too.. but, the scratchpad is only 256 bytes. Why do you need to move 1.5k /per record/? Even if you did all of scratchpad, that's 512 bytes, plus 80 bytes max for the D/V80 record?

 

That said, certainly nothing wrong with your plan to move stuff over to regular RAM. Scratchpad really only offers a large benefit for frequently accessed data - if you only hit it now and then you'd be hard pressed to see the difference. It's only 4 cycles slower to read and 8 cycles slower to write - still faster than a single 9900 instruction.

 

 

The code I had basically did the following (assuming DSR scratchpad is already loaded):

1. READ DV80 RECORD
2. Backup "DSR" scratchpad: 256 bytes
3. Restore "SP2" scratchpad: 256 bytes
4. Do stuff with record....
5. Backup "SP2" scratchpad: 256 bytes
6. Restore "DSR" scratchpad: 256 bytes
7. GOTO 1 (*)
    

(*) That gives me 1K. For sake of simplicity I'm not counting VDP read/write for record and some other stuff I had in there.

 

Anyway, turned out it paid of not to use any fixed memory locations in my source code. Just had to change the required equates and it worked ok. 
I'm now only using scratchpad for the workspace and 8 bytes of "machinecode" that runs there for copying stuff to VDP and memory.

A while back I checked what scratchpad memory addresses dsrlnk is using. Think I'm safe, but have to check with multiple devices to see if everything is fine.

 

Tested it with the HRD4000B and TIPI and it runs fine. 

 

Speed increase is roughly 40%.  Loading the invaders source code (96KB, 3063 lines of code) from HRD4000B now takes roughly 36 seconds compared to the 60 seconds it took before.

What can I say, I'm a  happy camper for now ?

 

This makes me wonder how far it can be pushed. At least using ROS' direct read to CPU memory could still give a boost. That's something I'd like to try.
I'm also still using normal file I/O, which I heard has quite some overhead in it for record handling. Doing sector I/O and reconstructing the record myself could give another boost.

Not sure if I would go down that road though. I'd rather have good compatibility instead of shaving of another few seconds. Then again....

[Tursi]

Ah, great to hear!

 

Only tip I'd have on the old process -- you don't need to save the DSR scratchpad between DSR calls. Just backup the scratchpad once at program start, and restore it when you need to call DSRLNK (or any console services), but you don't need to back it up again afterwards. In fairness you don't even need the entire scratchpad, there's lots the console doesn't use, but it's quick enough. That would save you one copy at least.

 

[+arcadeshopper]

On 7/29/2020 at 7:28 PM, Vorticon said:

Very cool. Would it be possible to save a file back to the Github?

not that way..URI/pi.http is readonly

 

[+retroclouds]

Small update on Stevie; I finally have the save functionality in place. So I guess I can officially call it an editor now. ?

Works great on tipi and HRD4000B. But currently not so great on the standard TI Disk Controller.

I get an immediate lockup when loading/saving from that controller. Guess I still have an old bug lurking around (thought I fixed that a long time ago, but there you have it, it's there again).

Presume it has to do with scratchpad pollution and the TI Standard disk controller being more picky, so will take some time debugging this.

[Tursi]

Could be more than scratchpad. Most of the TI Controller bugs that hit Classic99-sourced software were VDP memory hits. Classic99 has checks for all the cases I know how to test for - you could try it there and watch the debug log.

 

[+retroclouds]

10 hours ago, Tursi said:

Could be more than scratchpad. Most of the TI Controller bugs that hit Classic99-sourced software were VDP memory hits. Classic99 has checks for all the cases I know how to test for - you could try it there and watch the debug log.

 

oh yes, I’ve been thinking about that. First have to bring back 24x80 mode though.

The thing is that the file dialogs are on rows 26-30 so I’m kinda blind while loading/saving files.

But bringing back 24x80 mode is well worth the effort if it allows me to run classic99 debugger.

[Tursi]

Bah, surely you have the keys memorized enough to be able to type blind. If not, you can always watch VDP memory in the debugger instead of looking at the screen.

 

[+retroclouds]

11 hours ago, Tursi said:

Bah, surely you have the keys memorized enough to be able to type blind. If not, you can always watch VDP memory in the debugger instead of looking at the screen.

 

hehe, I seriously thought about that for a moment ?

[+retroclouds]

On 8/19/2020 at 8:35 PM, retroclouds said:

oh yes, I’ve been thinking about that. First have to bring back 24x80 mode though.

The thing is that the file dialogs are on rows 26-30 so I’m kinda blind while loading/saving files.

But bringing back 24x80 mode is well worth the effort if it allows me to run classic99 debugger.

 

I've been able to nail down the problem in the Disk Controller DSR. Don't fully understand it yet, but the story goes like this:

 

1. DSR takes over

2. DSR calls preparation routine

3. In the preparation routine, the DSR looks for a VDP file buffer that was created by itself by comparing CRU stored in VDP file buffer. (On startup of the TI-99/4a  with call files equivalent done by DSR power-up routine). 

4. The preparation routine calls a VDP read routine that should bring back the CRU in the VDP file buffer. The problem is that it always brings back >00 instead of the CRU MSB >11
5. As there is no match, it jumps to the next buffer in (3) and is in an infinite loop

 

So I need to find out why the VDP byte read routine in the DSR brings back >00 instead of the CRU >11. That will have to wait for another day.

 

 

Here's what I did to get there:

1. I used mame and got a version of Stevie running there (no F18a support) so I'm a little bit blind, but works ok. Mame is setup with SAMS and TI Disk Controller

2. I downloaded the commented disassembly of the TI Disk Controller DSR from Thierry's Site and used that to step through the DSR and set breakpoints.

 

By the way, I really dig the mame debugger:

• Multiple windows possible each with memory, disassembly, debugger, ...
• Multiple breakpoints possible, trace to disk, etc.
• Control the debugger via LUA scripts (sorta, not sure how well that works though)

 

Don't think I would ever be able to find the bug without stepping through the DSR to actually see what is going on. 
It works fine in all other emulators as there is no Disk Controller DSR there, so cannot reproduce.

 

I thought there was an experimental feature in classic99 where you can enable the TI Disk Controller ROM, but could not find it anymore (checked multiple classic99 revisions).

[HOME AUTOMATION]

I think it's always there at >1100? just need to enable CRU bit 0.

 

 

To get it to use your files... maybe you need to load thusly(Dunno)...

 

 

[+retroclouds]

10 hours ago, HOME AUTOMATION said:

I think it's always there at >1100? just need to enable CRU bit 0.

 

 

To get it to use your files... maybe you need to load thusly(Dunno)...

 

 

Thanks for pointing out where to select the TI Disk Controller! I knew I had seen it somewhere in classic99 but could not find it.

 

The problem is not the CRU card itself, it’s there.

Somehow the VDP byte read routine in the DSR code gets messed up and returns >00. 

Yesteday evening I didn’t have time anymore to step through that part as well, so that’s something I like to try this evening.

 

 

 

[+retroclouds]

ok, I solved this stupid, stupid bug that took me too much time ?

 

The problem was that R15 did not contain >8C02 at the time the TI-Disk Controller DSR was entered.  >8C02 is the VDP port address.

Due to the fact that R15 was >0000 at the time the TI Disk Controller DSR was entered, obviously no byte was ever read from VDP memory and the DSR could not check if it owned the VDP file buffer itself and went into a loop between >4744 - and >4754

 

Might be a check to add to classic99; verify that R15 contains value >8C02 when the TI Disk Controller is entered 

 

 

Edited August 24, 2020 by retroclouds