Help with saving in IntyBasic

[+atari2600land]

My program won't save any more. I'm using the following to read and write:

write_row: PROCEDURE
    FOR c = 0 TO 15
    if c=0 then #row(0)=levelnumber
    if c=1 then #row(1)=lemonnumber
    if c=2 then #row(2)=health
    NEXT c    
    
    FLASH WRITE FLASH.FIRST,VARPTR #row(0)
    return
    END
    
read_row: PROCEDURE
    FLASH READ FLASH.FIRST,VARPTR #row(0)

    for c= 0 to 15

    next c
    
    return
    END

I'm doing this before every level:

    gosub write_row
    gosub read_row

I put the --jlp command in when compiling and playing in jzintv but it still won't remember my progress. What am I doing wrong?

[+atari2600land]

A little better: when I enter level 3 it displays what I saved in level 2.

[Zendocon]

I had written definitions to sort of automate the Flash save and load functions.  If you're still having trouble, I'll dig them up.

[+atari2600land]

Problem solved. You must ERASE what you have before you can rewrite to it.

[+atari2600land]

Problem NOT solved. This is stupid. I hate living. When I get to a new level it gets rid of #row(1) and #row(2).

[+atari2600land]

All I want to do is store three variables into the jlp flash memory. One is called levelnumber, one is called lemonnumber, and one is called health. Why does it have to be so HARD?! Can someone please tell me how to do this? The flash sample program is not understandable.

[Kiwi]

6 hours ago, atari2600land said:

read_row: PROCEDURE
    FLASH READ FLASH.FIRST,VARPTR #row(0)

    for c= 0 to 15
    if c=0 then levelnumber=#row(0)
    if c=1 then lemonnumber=#row(1)
    if c=2 then health=#row(2)
    next c
    
    return
    END

 

Should read_row:PROCEDURE have the value loaded from flash to variable?

[DZ-Jay]

I haven't used IntyBASIC for FLASH storage, but from reading the documentation, there are a few things you have to consider.

• You need to include FLASH INIT somewhere at the start of your program.
• The array used must have at least 96 entries.  This means you need to have "DIM #row(96)" somewhere at the top as well.
• You must always erase a sector prior to writing it, so you should use FLASH ERASE prior to FLASH WRITE.

 

@atari2600land, what does "c" represent in your program?  In the example program they use it to iterate through the #row array to print it to the screen; but your code does not seem to be doing that.

 

I do not understand what you are trying to accomplish with the loop:  it goes from 0 to 16, but only uses the first three to assign the values to the array for storage.  Couldn't you just do something like:

 

' Somewhere at the top of your program ...
	FLASH INIT
	Dim #row(96)

'
' Your program goes here ...
'

' ... time to write to flash:
GOSUB erase_sector
GOSUB write_row
' ...


'
' More program goes here ...
'


' ... time to read from flash:
GOSUB read_row
' ...

write_row: PROCEDURE
    #row(0)=levelnumber
    #row(1)=lemonnumber
    #row(2)=health
    
    FLASH WRITE FLASH.FIRST,VARPTR #row(0)
    return
    END
    
read_row: PROCEDURE
    FLASH READ FLASH.FIRST,VARPTR #row(0)

    levelnumber=#row(0)
    lemonnumber=#row(1)
    health=#row(2)

    return
    END

erase_sector: PROCEDURE
    FLASH ERASE FLASH.FIRST
    return
    END

 

[+nanochess]

DZ-Jay is right with the structure of the code. Furthermore I would do the variable saving code like this:

 

    DIM #row(96)
    FLASH INIT
    
    ' ... other code ...

write_row: PROCEDURE
    #row(0)=levelnumber
    #row(1)=lemonnumber
    #row(2)=health
    FLASH ERASE FLASH.FIRST
    FLASH WRITE FLASH.FIRST,VARPTR #row(0)
    END
    
read_row: PROCEDURE
    FLASH READ FLASH.FIRST,VARPTR #row(0)

    levelnumber = #row(0)
    lemonnumber = #row(1)
    health = #row(2) 
    END

Edited February 22, 2021 by nanochess

[DZ-Jay]

1 hour ago, nanochess said:

DZ-Jay is right with the structure of the code. Furthermore I would do the variable saving code like this:

 

    DIM #row(96)
    FLASH INIT
    
    ' ... other code ...

write_row: PROCEDURE
    #row(0)=levelnumber
    #row(1)=lemonnumber
    #row(2)=health
    FLASH ERASE FLASH.FIRST
    FLASH WRITE FLASH.FIRST,VARPTR #row(0)
    END
    
read_row: PROCEDURE
    FLASH READ FLASH.FIRST,VARPTR #row(0)

    levelnumber = #row(0)
    lemonnumber = #row(1)
    health = #row(2) 
    END

@nanochess,

 

I though of that, but didn't really know why it was split to being with.  Is there any reason why you would ever call FLASH WRITE without first calling FLASH ERASE?  If not, why not build it into the call to FLASH WRITE?

[+nanochess]

2 minutes ago, DZ-Jay said:

@nanochess,

 

I though of that, but didn't really know why it was split to being with.  Is there any reason why you would ever call FLASH WRITE without first calling FLASH ERASE?  If not, why not build it into the call to FLASH WRITE?

 

This is because my ahead thinking: FLASH ERASE erases eight rows, what if you want to write fast and write each row in sequence until filling the eight rows? Better to leave the choice for the tech-savvy programmer.

 

 

[DZ-Jay]

7 minutes ago, nanochess said:

 

This is because my ahead thinking: FLASH ERASE erases eight rows, what if you want to write fast and write each row in sequence until filling the eight rows? Better to leave the choice for the tech-savvy programmer.

 

 

 

Ah, so writing doesn't occur per sector, I see.  I thought you had to read/write/erase in blocks of 8 rows.  Re-reading the documentation I see that erase is per sector, but read and write is per row.

 

Implied in all that is that a "FLASH row" can hold an array of 96 words, is this true?

 

 

 

 

[+nanochess]

1 minute ago, DZ-Jay said:

 

Ah, so writing doesn't occur per sector, I see.  I thought you had to read/write/erase in blocks of 8 rows.  Re-reading the documentation I see that erase is per sector, but read and write is per row.

 

Implied in all that is that a "FLASH row" can hold an array of 96 words, is this true?

 

That's right

 

[DZ-Jay]

1 minute ago, nanochess said:

That's right

 

 

Gotcha.  I think the confusion comes from the fact that the array used to interface between JLP RAM and NVRAM is called "#row," which at a glance implies that each element is a row itself.

 

Thus, when one sees "#row(0)" and "#row(1)", they look like two separate "FLASH rows," when in fact they are all part of the same actual row, because each row is a record of 96 words.

 

     -dZ.

[+atari2600land]

edit: It's not nanochess's code, it's something I'm doing wrong. It goes back to the title screen after doing the level intro and I don't know why.

[+atari2600land]

Health is being displayed as "I5" when it should be "95".

[+atari2600land]

How odd, it seems to be working correctly now. I wonder if that save I had before just got corrupted sitting on my computer for some reason. Thanks, all.

[DZ-Jay]

I just caught up with this ... OK, so ... does it work now?  Are you good?

 

    -dZ.

[+atari2600land]

I'm good for now.

[intvnut]

On 2/22/2021 at 10:00 AM, DZ-Jay said:

Ah, so writing doesn't occur per sector, I see.  I thought you had to read/write/erase in blocks of 8 rows.  Re-reading the documentation I see that erase is per sector, but read and write is per row.

 

Implied in all that is that a "FLASH row" can hold an array of 96 words, is this true?

 

Yes.  This is exactly the same JLP flash savegame feature we used on Christmas Carol.  I don't know if you have any of the notes or code from that era, from when we were going to integrate a high-score table, etc.

 

Flash consists sectors with 8 rows of 96 words/row.  Thus, each sector has 768 words.  You can read/write on row granularity, but you can only erase groups of 8 rows. These quantities are all dictated by the hardware.  IntyBASIC provides a raw interface to the read/write/erase primitives that I provide in JLP, and my JLP primitives are thin wrappers on the hardware.

 

On 2/22/2021 at 9:41 AM, DZ-Jay said:

I though of that, but didn't really know why it was split to being with.  Is there any reason why you would ever call FLASH WRITE without first calling FLASH ERASE?  If not, why not build it into the call to FLASH WRITE?

One big reason, as already explained, is that erase will erase 8 rows, while write only writes 1.  But why not just expose an API that deals with 768 word chunks instead, and have everyone erase/write on that granularity?

 

Here's why:  The flash wears out.  The flash cells on JLP are rated for a minimum of 10,000 erase/write cycles.  I usually recommend to folks to use a collection of rows, and spend one word of the save data to store a "generation counter."  I recommend spreading your writes over at least 32 rows.  That takes you from 10,000 writes to 320,000 writes at least.  There's an additional advantage:  If you lose power or suffer a glitch between the erase and write, you only lose your most recent update, rather than losing everything.

 

So far, I haven't gotten a lot of traction there, it feels like. To be fair, 10,000 erase/write cycles seems like a lot.  And, if you only write to flash when the values change, it's unlikely that you'll see 10,000 changes.  My not-unfounded fear is that most folks will just blindly say "save values" even if they haven't changed.  Depending on what you're saving and when (e.g. saving "progress flags" when you reach certain places in a game), that could end up being a lot of extra "saves" that just burn flash lifetime without changing what's stored.

 

I have produced some reference code examples (and I think I even did one or two in IntyBASIC at some point), but it doesn't seem to stick.  Honestly, I think to make it stick, IntyBASIC needs to provide a higher level API that just handles this, period.  I suppose it could be something that lives in IntyBASIC's contrib/ directory, if the IntyBASIC docs also point to it.

 

It's possible I'm oversensitive to this.  *shrug*

 

Just know that LTO Flash! aggressively wear levels its flash, and its flash has a larger E/W endurance rating than JLP (100,000 per cell, IIRC).  I keep a minimum pool of 40 sectors to wear level over when the device is "full"; otherwise, I wear level over all free sectors.*  And if you run a game that uses JLP Savegame functionality on LTO Flash!, it actually gets translated to LTO Flash!'s filesystem and gets transparently wear-leveled in LTO Flash!'s filesystem.  So, you won't break LTO Flash! unless you put in some dedicated effort.  ("We're defending against Murphy, not Machiavelli.")   Manual wear leveling for JLP only matters on single-game JLP boards.

 

___________

*In case anyone is curious, I measure LTO Flash!'s remaining lifetime assuming it only supports ~2,000,000 erase/write cycles.  But, you might notice I wear level over a minimum of 40 sectors, and 40 × 100,000 = 4,000,000, so there's already a built in 2:1 safety factor.  And, if your device is less than full, I wear level across all the free space.  So, LTO Flash!'s remaining flash life indicator is conservative by at least a factor of 2, and probably much larger for most folks.  And, I'm willing to bet nobody's below 80% flash life yet.  I'd be surprised if there's anyone below 90%, actually.

[DZ-Jay]

4 hours ago, intvnut said:

 

Yes.  This is exactly the same JLP flash savegame feature we used on Christmas Carol.  I don't know if you have any of the notes or code from that era, from when we were going to integrate a high-score table, etc.

 

Yup, got that, thanks.  It's been a while. 

 

Quote

Flash consists sectors with 8 rows of 96 words/row.  Thus, each sector has 768 words.  You can read/write on row granularity, but you can only erase groups of 8 rows. These quantities are all dictated by the hardware.  IntyBASIC provides a raw interface to the read/write/erase primitives that I provide in JLP, and my JLP primitives are thin wrappers on the hardware.

 

I noticed that, and reading the "savegame" document that comes with jzInt, I understand how it works now.  Thanks

 

Quote

One big reason, as already explained, is that erase will erase 8 rows, while write only writes 1.  But why not just expose an API that deals with 768 word chunks instead, and have everyone erase/write on that granularity?

 

Here's why:  The flash wears out.  The flash cells on JLP are rated for a minimum of 10,000 erase/write cycles.  I usually recommend to folks to use a collection of rows, and spend one word of the save data to store a "generation counter."  I recommend spreading your writes over at least 32 rows.  That takes you from 10,000 writes to 320,000 writes at least.  There's an additional advantage:  If you lose power or suffer a glitch between the erase and write, you only lose your most recent update, rather than losing everything.

 

So far, I haven't gotten a lot of traction there, it feels like. To be fair, 10,000 erase/write cycles seems like a lot.  And, if you only write to flash when the values change, it's unlikely that you'll see 10,000 changes.  My not-unfounded fear is that most folks will just blindly say "save values" even if they haven't changed.  Depending on what you're saving and when (e.g. saving "progress flags" when you reach certain places in a game), that could end up being a lot of extra "saves" that just burn flash lifetime without changing what's stored.

 

But that's kinda happening already.  Most of the examples I've seen include an erase/write combo at the end of the game to store state, irrespective of changes.  Plus none of this complexity is even hinted at in the IntyBASIC manual nor in most of the guidance that is given around here (your specific posts on the subject excepted).

Quote

I have produced some reference code examples (and I think I even did one or two in IntyBASIC at some point), but it doesn't seem to stick.  Honestly, I think to make it stick, IntyBASIC needs to provide a higher level API that just handles this, period.  I suppose it could be something that lives in IntyBASIC's contrib/ directory, if the IntyBASIC docs also point to it.

 

I definitely agree with all that.  Unfortunately the example I found in the IntyBASIC distro gives you a low-level example of how to use the FLASH-related commands as one-off routines, and the manual doesn't even hint at the complexity of managing generations or the downsides and potential failures.

 

Then it seems that IntyBASIC programmers are told via the examples and in the forum to just call erase before writing without fully understanding the implications of this, and at that point it's just copy+pasting boilerplate.  *shrug*

 

Could you point me to one of those examples you said you prepared for IntyBASIC?  I could include them in the next SDK and help promote a more robust usage pattern.

 

Quote

It's possible I'm oversensitive to this.  *shrug*

 

Maybe, but I think it's important to consider the implications of using the technology. 

 

    -dZ.

Edited March 13, 2021 by DZ-Jay

[Brian's Man Cave]

Hey!

 

This has been very helpful in figuring out how to save my game in progress.

 

One issue I am having... When I test the game save, it works. I hit reset and when I hit load save game, it works fine. However, when I exit JZINTV, then run the game again... the save is now gone.

 

Am I missing a step somewhere?

 

[DZ-Jay]

2 minutes ago, Brian's Man Cave said:

Hey!

 

This has been very helpful in figuring out how to save my game in progress.

 

One issue I am having... When I test the game save, it works. I hit reset and when I hit load save game, it works fine. However, when I exit JZINTV, then run the game again... the save is now gone.

 

Am I missing a step somewhere?

 

 

You need to add "--jlp-savegame=<filename>" to the command-line of the emulator, where "<filename>" is the name of a file in which to save/load the state.

 

    -dZ.

[Brian's Man Cave]

3 minutes ago, DZ-Jay said:

 

You need to add "--jlp-savegame=<filename>" to the command-line of the emulator, where "<filename>" is the name of a file in which to save/load the state.

 

    -dZ.

Here is what I use for my command.

 

intybasic --title "Venture 2" --jlp vent.bas vent.asm

 

where would I put that part?

[DZ-Jay]

By the way, I plan on adapting @intvnut's reference code for IntyBASIC to provide a library for saving game data.  The solutions he recommend are not that trivial, but they are robust.

 

    dZ.