The perfect dumper?

[Thomas Jentzsch]

Inspired by Atari's compatibility list for the 2600+, I wondered how reliable auto detection for a dumper can become.

 

Stella has some quite good auto detection (within ~3600 property entries, I count less than 30 entries which overwrite the auto detection, mostly Xin1 carts), but it is based on the whole ROM. Stella uses the file size and then looks for certain byte pattern. The dumper only sees 4K initially. So it cannot work like that. Instead it has to trigger hotspots and check the result.

 

The dumper code for the R77 is known. It only detects 4K, F8 (8K) and F6 (16K). For that it triggers four hotspots ($1ff6..$1ff9), calculates the checksum of the dumped 3.5K (it avoids the first and the last 256 bytes) and if it is different, it knows it has detected a hotspot. Each hotspot pair ($fff6/$fff7 and $fff8/f9) clearly defines a bankswitching scheme (F6/F8). The spoiler shows the R77 code with my own comments.

Spoiler

MAPPER_CHECK:
; detects bankswitching scheme
            MOV     R0,#0X40        ; pointer to checksum results
            MOV     STAR_ADDRL,#0XF6; first hotspot 1ff6
            MOV     STAR_ADDRH,#0X1F
            MOV     R5,#4           ; 4 hotspots to check
XXREAD_EEPROM_LOOP1:
            MOV     R6,#56          ; first and last 256 bytes skipped
            MOV     CHKSUMH,#0X00
            MOV     CHKSUML,#0X00
            ; trigger hotspot
            MOV     ADDRL,STAR_ADDRL
            MOV     ADDRH,STAR_ADDRH
            CALL    ADDR_SET_SUB
            CALL    DELAY_10us
            CALL    DELAY_10us
            ; start reading from 0x0100
            MOV     ADDRL,#0X00
            MOV     ADDRH,#0X01
            CALL    ADDR_SET_SUB
            CALL    DELAY_10us
            CALL    DELAY_10us
            ; start calculating checksums for this bank
XXREAD_EEPROM_LOOP2:
            MOV     R7,#64          ; read in chunks of 64 bytes
XXREAD_EEPROM_LOOP:
            CALL    ADDR_SET_SUB    ; set address to read from
            CALL    DELAY_5us       ; wait 5µs
            MOV     DPTR,#rGPIOC_DAT; read byte
            MOVX    A,@DPTR
            ; update checksum
            ADD     A,CHKSUML
            MOV     CHKSUML,A
            JNC     XXCHKSUM_NOT_OVER
            INC     CHKSUMH
XXCHKSUM_NOT_OVER:
            ; increment address by 1
            INC     ADDRL
            MOV     A,ADDRL
            JNZ     XXADD_OVER_256
            INC     ADDRH
XXADD_OVER_256:
            DJNZ    R7,XXREAD_EEPROM_LOOP
            DJNZ    R6,XXREAD_EEPROM_LOOP2

            MOV     A,CHKSUML       ; store low byte into
            MOV     @R0,A           ;  0x40, 0x42, 0x44, 0x46
            INC     R0
            MOV     A,CHKSUMH       ; store high byte into
            MOV     @R0,A           ;  0x41, 0x43, 0x45, 0x47
            INC     R0

            MOV     R6,#56          ; (again) first and last 256 bytes skipped
            INC     STAR_ADDRL      ; next hotspot (1ff6, 1ff7, 1ff8, 1ff9)
            DJNZ    R5,XXREAD_EEPROM_LOOP1 ; loop hotspots

; compare checksums
            MOV     A,0X40
            XRL     A,0X42          ; 0x40 XOR 0x42 (lo 1ff6 vs 1ff7)
            JZ      F6L_EQU         ; if equal, try high
            JMP     MAPPER_F6       ; else F6 (16K)
F6L_EQU:
            MOV     A,0X41
            XRL     A,0X43          ; 0x41 XOR 0x43 (hi 1ff6 vs 1ff7)
            JZ      F6H_EQU         ; if equal, try F8
MAPPER_F6:                          ; else F6 (16K)
            MOV     A,#0X02         ; 2 = 16K
            JMP     MAPPER_OK

F6H_EQU:
            MOV     A,0X44
            XRL     A,0X46          ; 0x44 XOR 0x46 (lo 1ff8 vs 1ff9)
            JZ      F8L_EQU         ; if equal, try high
            JMP     MAPPER_F8       ; else F8 (8K)
F8L_EQU:
            MOV     A,0X45
            XRL     A,0X47          ; 0x45 XOR 0x47 (hi 1ff8 vs 1ff9)
            JZ      F8H_EQU         ; if equal, 4K
MAPPER_F8:                          ; else F8 (8K)
            MOV     A,#0X01         ; 1 = 8K
            JMP     MAPPER_OK
F8H_EQU:
            MOV     A,#0X00         ; 0 = 4K (or undetected)
MAPPER_OK:
            MOV     CART_MAPPER,A
            RET

This could be extended to F4 (32K) easily. And I suppose it could be done for all kind of unique hotspot addresses. But how slow will this become? Brute force will most likely not work, this has to be done methodical (e.g. a following a decision tree). And how will it detect e.g. Activision's FE? Will it handle Xin1 carts work? How about carts with a menu, where the user selects the game?

 

And Pitfall II and the ARM carts will most likely never work via a simple dumper.

 

What are your thoughts and ideas here?

[Albert]

I'm kind of surprised it doesn't already have code to detect F4 games.  There are quite a few 32K homebrew games, that would certainly help with compatibility for cart-dumper type systems.  I'm sure there are many schemes that cannot easily be detected, and of course more advanced carts like the ones you mentioned are going to be more problematic unless you can perhaps read a portion of the cart and then compare that against a small portion of the cart stored in Stella's database, but that becomes more problematic legally.

 

 ..Al

[Thomas Jentzsch]

5 minutes ago, Albert said:

I'm kind of surprised it doesn't already have code to detect F4 games.  There are quite a few 32K homebrew games, that would certainly help with compatibility for cart-dumper type systems.

Maybe the compatibility list will improve over time. After all the console is only for pre-order now. 🤞

5 minutes ago, Albert said:

I'm sure there are many schemes that cannot easily be detected, and of course more advanced carts like the ones you mentioned are going to be more problematic unless you can perhaps read a portion of the cart and then compare that against a small portion of the cart stored in Stella's database, but that becomes more problematic legally.

I think a known hash of that portion would work as well.

Edited August 28, 2023 by Thomas Jentzsch

[Albert]

8 minutes ago, Thomas Jentzsch said:

Maybe the compatibility list will improve over time. After all the console is only for pre-order now. 🤞

Well, it's not shipping until November, I wonder how much time they'd have to update Stella before then...

9 minutes ago, Thomas Jentzsch said:

I think a known hash of that portion would work as well.

That's even better.

 

 ..Al

[Thomas Jentzsch]

7 minutes ago, Albert said:

Well, it's not shipping until November, I wonder how much time they'd have to update Stella before then...

Not Stella, the dumper.  

[Albert]

1 minute ago, Thomas Jentzsch said:

Not Stella, the dumper.  

Well, I guess there are two pieces, the dumper AND Stella would need to be updated.

 

 ..Al

[Thomas Jentzsch]

15 minutes ago, Albert said:

Well, I guess there are two pieces, the dumper AND Stella would need to be updated.

Why Stella? As of now if knows all bankswitching schemes. 

[Albert]

I don't know how the dumper on the 2600+ works, but surely there needs to be some way the dumper can communicate to Stella that it only has a partial dump of the game.  Either that, or you are suggesting the dumper takes a partial dump, then compares it to its own internal list of partial dumps, which are associated with the full md5 checksum that would then be passed to Stella, or something to that effect.

 

 ..Al

[Thomas Jentzsch]

1 minute ago, Albert said:

I don't know how the dumper on the 2600+ works, but surely there needs to be some way the dumper can communicate to Stella that it only has a partial dump of the game.  Either that, or you are suggesting the dumper takes a partial dump, then compares it to its own internal list of partial dumps, which are associated with the full md5 checksum that would then be passed to Stella, or something to that effect.

Ah, I misunderstood you then. Yes, in such a case, Stella would have to be updated.

I suppose the user would have to provide the ROM (somehow) due to legal reasons. 

[+Bruce-Robert Pocock]

• Save-to-cart won't work with a "dumper" either
• Why a dumper at all? Wouldn't it make much more sense to just write a CartPhysical class for Stella that directly manipulates the address bus / reads the data bus tied to the cartridge in real time? Is there a performance constraint there that I'm not seeing?

[Thomas Jentzsch]

13 minutes ago, Bruce-Robert Pocock said:

• Why a dumper at all? Wouldn't it make much more sense to just write a CartPhysical class for Stella that directly manipulates the address bus / reads the data bus tied to the cartridge in real time? Is there a performance constraint there that I'm not seeing?

That would require cycle accurate emulation. Which is what an FPGA can do. An emulator can do too, but only by using massive CPU power.

[Dionoid]

Assuming the dumper code works, I’m wondering how Stella would be able to differentiate between NTSC and PAL60 rom dumps, as Stella won’t be able to extract information from the rom’s file name.

[Thomas Jentzsch]

2 minutes ago, Dionoid said:

Assuming the dumper code works, I’m wondering how Stella would be able to differentiate between NTSC and PAL60 rom dumps, as Stella won’t be able to extract information from the rom’s file name.

Usually Stella can't. There is a setting which tries to do that based on the color codes used, but it is only an experiment and pretty unreliable.

So Atari would have to provide property files for these games.

[Dionoid]

12 hours ago, Thomas Jentzsch said:

For that it triggers four hotspots ($1ff6..$1ff9), calculates the checksum of the dumped 3.5K (it avoids the first and the last 256 bytes) and if it is different, it knows it has detected a hotspot.

Hi Thomas, do you have an idea why the fist and last 256 bytes are ignored for the checksum? I get that the hotspot addresses need to be ignored, but why the first 256 bytes? Is this done to support carts with extra RAM (e.g. SARA superchips)?

[Thomas Jentzsch]

6 hours ago, Dionoid said:

Is this done to support carts with extra RAM (e.g. SARA superchips)?

I am pretty sure that's the reason. 

 

BTW: Later the dumper has to read the whole 4K for each bank. There it must not accidentally switch banks. The R77 has some simple logic here. If the read address is >= $FFF0, it simply switches banks before each byte is read. 

 

SC RAM is not detected, this is done later by Stella.

Edited August 29, 2023 by Thomas Jentzsch

[Blinky]

19 hours ago, Thomas Jentzsch said:

            CALL    ADDR_SET_SUB
            CALL    DELAY_10us
            CALL    DELAY_10us

 

19 hours ago, Thomas Jentzsch said:

            CALL    ADDR_SET_SUB    ; set address to read from
            CALL    DELAY_5us       ; wait 5µs

Odd they use such long delays. The 6502/7 CPU has access times < 1 us so 1 us should suffice.

 

19 hours ago, Thomas Jentzsch said:

Pitfall II and the ARM carts will most likely never work via a simple dumper

Those will only work when the cart is accessed in real time by the emulator.

 

As for a good dumper. I expect it to have pull up resistors on the databus so when there is no cart inserted all data will read as FF Hex (in case of pull down resistors the value would be a stable 0 ) scanning for this can also be used as a method to detect if a cart is present or not.

 

My idea would be to read addresses 1FFF down to  1FF4 and AND then together with FF if the result is still FF then there is no cart (these locations are never all FF on a valid ROM because of the vectors) . Reading these addresses in descending order ensures that bank 0 of F4/F6/F8 carts is selected after the 1st scan and the vectors of bank 0 are guaranteed to be read on successive scans.

As for detecting rom differences I'd start by checking the last page as bankswitch code is usually located there. If no differences are found the next page below can be tested. I'm not sure on how far down this can be repeated. Need to look at the non atari carts bankswitch schemes that have RAM.

 

 

 

 

 

 

 

 

 

[alex_79]

19 hours ago, Dionoid said:

Assuming the dumper code works, I’m wondering how Stella would be able to differentiate between NTSC and PAL60 rom dumps, as Stella won’t be able to extract information from the rom’s file name.

If there's some way to access Stella's UI, you can set the TV format after starting the game, and Stella will save and remember that if the rom dumped always has the same checksum (which might not be always the case if the cart has RAM). Lots of ifs.

[Dionoid]

29 minutes ago, alex_79 said:

If there's some way to access Stella's UI, you can set the TV format after starting the game, and Stella will save and remember that if the rom dumped always has the same checksum (which might not be always the case if the cart has RAM). Lots of ifs.

My guess is that the Atari 2600+ will just push the rom-dump to Stella without the option to access Stella's settings. So when creating a PAL50 version of a game, the developer just has to make sure it outputs 312 scanlines each frame, so Stella detects it as a PAL rom.

Edited August 29, 2023 by Dionoid

[alex_79]

Quote

My guess is that the Atari 2600+ will just push the rom-dump to Stella without option to access Stella's options.

Yes, but it is upgradeable, so maybe that limitation will be soon circumvented...

There's a "misterious" switch on the back, (to select aspect ratio between 4:3/16:9? or between 2600/7800 emulation?), and if it's software controlled maybe it might be repurposed to bring up a menu.

 

Another way could be to just add properties for new games when updating Stella through the usb port. (there's still the issue of games with RAM that could potentially have different checksums)

 

Edited August 29, 2023 by alex_79

[xeex]

13 minutes ago, alex_79 said:

There's a "misterious" switch on the back, (to select aspect ratio between 4:3/16:9? or between 2600/7800 emulation?), and if it's software controlled maybe it might be repurposed to bring up a menu.

 

I believe it's a 16:9 / 4:3 switch. I copied this image from their announcement trailer, flipped it and zoomed a little:

 

[+MrZarniwoop]

14 minutes ago, alex_79 said:

There's a "misterious" switch on the back, (to select aspect ratio between 4:3/16:9?

I took a screen grab from the video, zoomed it in and flipped it. Yup.

 

[xeex]

4 minutes ago, MrZarniwoop said:

I took a screen grab from the video, zoomed it in and flipped it. Yup.

Hey, great minds think alike. Your image came out a bit clearer.

[Blinky]

2 hours ago, alex_79 said:

might not be always the case if the cart has RAM

If the dumper detects ram it should dump the ram part of a rom image as all FFs. The hash will then be the same.

[Thomas Jentzsch]

On 8/29/2023 at 6:53 PM, Blinky said:

If the dumper detects ram it should dump the ram part of a rom image as all FFs. The hash will then be the same.

The R77 dumper dumps the RAM as it reads it. That works well, since it is identical for all banks.

Edited September 2, 2023 by Thomas Jentzsch

[Thomas Jentzsch]

3 hours ago, MrZarniwoop said:

I took a screen grab from the video, zoomed it in and flipped it. Yup.

 

I wonder why they decided for a switch here (R77 has a button here, but then it also has buttons for the switches). Stella usually expects an event for switching between stretched and non-stretched (actually this is only a 16:9 switch for a 16:9 display), so a button would have matched better.