Emulator test suite?

[webdeck]

I've been learning a lot about various edge cases from threads here, and I am wondering if anyone has already compiled a good set of tests to put an emulator through.  Right now I've just been trying different carts, but that's not a very methodical approach.

[+wavemotion]

For me, the acid test has been the MEGADEMO which hits so many edge cases - VDP timing, modes, sprite handling, etc. 

 

Additional VDP stress testers are Pole Position (timing is tight - sometimes the game will glitch/freeze while the starting banner scrolls across the screen) and Popeye where the sea-hag throws bottles and will leave visible graphical artifacts on screen if you don't have the VDP prefetch just right.  Skyway needs the 5th sprite stuff to work properly. @Asmusr has a demo of 48 sprites on a screen (birds going left-to-right-to-left) that is really worth trying but I forget the name.

 

For SAMS, I use the AMSTEST4 and @jedimatt42's Expanded Memory Test - as well as Dungeons of Asgard (needs 128K SAMS) and Realms of Antiquity (well worth the $20 and despite saying 1MB SAMS, it works fine on 512K SAMS).

 

For bank switching, the Dragon's Lair Demo at 8MB is solid - though really anything that hits the banks works (Donkey Kong is a simple C/D game with just one bank but it hits HARD in terms of numerous switches).

 

Some other isolated cases...

 

Slymoids is one of the few carts utilizing the CPU IDLE handling - my emulator froze on that one for some time.

 

Eric in Monsterland requires that you scan and latch sprite collisions more than once per frame.

 

Miner 2049er requires that the 5S sprite number act like a counter when not latched (or collision detection won’t work). 

 

Any of the MBX games (I tend to use Bigfoot because, well, it's awesome) for the 1K of MBX RAM and odd banking.

 

Lasso will hang if you don't handle the Speech module correctly (at least you need to make it think it exists).

 

Congo Bongo and Buck Rogers in original C/D form require RAM mirrors in >8000 area to work (Congo Bongo won't crash until the 2nd stage).

 

Borzork and Mission Destruct both hit the sound chip with occasional 16-bit writes that caused me some trouble until recently (audio squealing).

 

I tend to prefer the original dumps of the carts rather than anything that has been doctored to run on the FinalGROM. Nothing against the FG - stunning bit of tech... but sometimes in the conversion things get moved/patched and original troublesome spots might not show.
 

I'm sure there are more... I've now forgotten more than I remember on all the troublesome carts.

 

Dave

[PeteE]

Here's my torture test, updated yesterday to catch the post-increment cases recently discussed in the assembly thread:

 

 

cputest.rpk cputestc.bin

[+wavemotion]

I totally forgot the CPU Test!

You're killing me, @PeteE!

 

I was clean on your CPU test until this version.

 

 

To save me some reading... what is the right order? I guess it's the order I'm not doing it...

 

 

Edit: found this comment I left for myself in the code:

 

 

I think we've answered the question about what this will cause

[PeteE]

47 minutes ago, wavemotion said:

To save me some reading... what is the right order? I guess it's the order I'm not doing it...

 

This is the right order: https://forums.atariage.com/topic/162941-assembly-on-the-994a/?do=findComment&comment=5415033

 

 

[+wavemotion]

Thanks @PeteE

 

 

I'm standing down from DEFCON-1

[+Torrax]

Check further down in this thread on the X instruction emulation problems, and Tursi's fix for Classic99.

 

 

 

[webdeck]

5 hours ago, PeteE said:

Here's my torture test, updated yesterday to catch the post-increment cases recently discussed in the assembly thread:

 

 

cputest.rpk 2.47 kB · 6 downloads cputestc.bin 6.81 kB · 6 downloads

Thank you so much for this!  It found some stupid status flag mistakes for ABS, S, and SRA in my emulator.  I managed to get the MOV *R0+,*R0+ correct.

[webdeck]

6 hours ago, wavemotion said:

For me, the acid test has been the MEGADEMO which hits so many edge cases - VDP timing, modes, sprite handling, etc. 

 

Additional VDP stress testers are Pole Position (timing is tight - sometimes the game will glitch/freeze while the starting banner scrolls across the screen) and Popeye where the sea-hag throws bottles and will leave visible graphical artifacts on screen if you don't have the VDP prefetch just right.  Skyway needs the 5th sprite stuff to work properly. @Asmusr has a demo of 48 sprites on a screen (birds going left-to-right-to-left) that is really worth trying but I forget the name.

 

For SAMS, I use the AMSTEST4 and @jedimatt42's Expanded Memory Test - as well as Dungeons of Asgard (needs 128K SAMS) and Realms of Antiquity (well worth the $20 and despite saying 1MB SAMS, it works fine on 512K SAMS).

 

For bank switching, the Dragon's Lair Demo at 8MB is solid - though really anything that hits the banks works (Donkey Kong is a simple C/D game with just one bank but it hits HARD in terms of numerous switches).

 

Some other isolated cases...

 

Slymoids is one of the few carts utilizing the CPU IDLE handling - my emulator froze on that one for some time.

 

Eric in Monsterland requires that you scan for collisions more than once per frame.

 

Any of the MBX games (I tend to use Bigfoot because, well, it's awesome) for the 1K of MBX RAM and odd banking.

 

Lasso will hang if you don't handle the Speech module correctly (at least you need to make it think it exists).

 

Congo Bongo and Buck Rogers in original C/D form require RAM mirrors in >8000 area to work (Congo Bongo won't crash until the 2nd stage).

 

Borzork and Mission Destruct both hit the sound chip with occasional 16-bit writes that caused me some trouble until recently (audio squealing).

 

I tend to prefer the original dumps of the carts rather than anything that has been doctored to run on the FinalGROM. Nothing against the FG - stunning bit of tech... but sometimes in the conversion things get moved/patched and original troublesome spots might not show.
 

I'm sure there are more... I've now forgotten more than I remember on all the troublesome carts.

 

Dave

Thanks - I'll need to try all of these out.  I already saw the Popeye issue and now have it fixed thanks to your pointer as to what was causing it.  I'm doing the >8000 mirroring and partial address line decoding, so those should be good.  I haven't gotten around to implementing sound, speech, or SAMS yet.

 

I'm curious about what you mean by "Eric in Monsterland requires that you scan for collisions more than once per frame."  My rendering approach is that I have two back buffers - one for the screen bitmap, and one for the sprite bitmap.  Both bitmaps start out as all transparent pixels.  I render the screen buffer based on the graphics mode and tables.  Then I do a pass on the sprite attributes to mark which sprites exist on which scan lines.  Then I render the sprites by scan line from top to bottom onto the sprite bitmap, starting with the 4th lowest sprite number on that line to the highest number - whenever I see a write to a pixel where there isn't transparency, I set COINC, and the first time I see 5 sprites on a scan line I set the 5th sprite flag and set the 5th sprite number.  If I don't see 5 sprites on any lines in the frame, I set a random 5th sprite number, since I saw the thread on the Miner 2049er bug that looks at the wrong bit for COINC.  Then I blit the screen bitmap onto the background color, and blit the sprite bitmap on top of it.  To cut down on work, I only re-render those buffers if there have been VDP writes to any registers or to any address ranges that matter for whatever screen mode I'm in based on the table ranges.  So what does scanning for collisions more than once per frame mean?

 

I've got what I assume is some sort of timing issue with Parsec - the enemy ships are flying down too low when they first appear and the ones that come from behind are wrapping all the way around off the right and back to the left again when they come out.  I also see the explosion that appears when I hit a ship to be shifted to the left of where the ship was by a good amount.

 

[+wavemotion]

1 hour ago, webdeck said:

I'm curious about what you mean by "Eric in Monsterland requires that you scan for collisions more than once per frame.

It sounds like you check for sprite collisions on each scanline - you'll be fine. To gain speed on my emulator for the DS handheld, I was only scanning for collisions of sprites at the end of the line. 99% of all games only scan for collisions at the end of a frame - true also for Coleco and thousands of MSX games. But Eric in Monsterland and the Megademo poll the collision bits during the frame - and specifically EiM will fail to detect collisions most of the time if you don't check and latch more frequently.

 

I'd love to switch to just scanning for collisions as the sprites are laid down - but it might be just a bit too much precious CPU time in my world.

 

No clues on Parsec... didn't run into that in my wayward travels.

[webdeck]

7 minutes ago, wavemotion said:

It sounds like you check for sprite collisions on each scanline - you'll be fine. To gain speed on my emulator for the DS handheld, I was only scanning for collisions of sprites at the end of the line. 99% of all games only scan for collisions at the end of a frame - true also for Coleco and thousands of MSX games. But Eric in Monsterland and the Megademo poll the collision bits during the frame - and specifically EiM will fail to detect collisions most of the time if you don't check more frequently.

 

I'd love to switch to just scanning for collisions as the sprites are laid down - but it might be just a bit too much precious CPU time in my world.

 

No clues on Parsec... didn't run into that in my wayward travels.

I will probably have issues now that I think about it.  I have a separate thread that is doing the rendering every 1/60th of a second, and that thread renders at full speed, so I may need to slow it down to whatever rate the VDP actually renders scan lines at.

[+khanivore]

On 2/22/2024 at 6:04 PM, PeteE said:

Here's my torture test

Is this good?  I'm guessing not ...

[Asmusr]

17 hours ago, webdeck said:

I will probably have issues now that I think about it.  I have a separate thread that is doing the rendering every 1/60th of a second, and that thread renders at full speed, so I may need to slow it down to whatever rate the VDP actually renders scan lines at.

What (most) other emulators do, is to draw one scan line, then run the CPU for the corresponding number of clock cycles, then draw the next scan line, and so on. But if you could get the threads to synchronize, I guess that would be a more realistic, and maybe faster, emulation.

Edited February 23 by Asmusr

[+wavemotion]

1 hour ago, khanivore said:

Is this good?  I'm guessing not ...

I had some of that very early on in my emulation testing with Pete's CPU test.

He once explained what the columns were... this is what I have in my scratch-pad notes that must have come from Pete himself:

For the first 24 failures, a line is printed containing: the instruction name, the first and second input, the result, the expected result, then the status flags, and finally the expected status flags. The first status flag byte is the result of the instruction after only EQ is set, and the second byte is the result after LGT AGT C OV OP are set. Instructions ending in I have the inputs swapped. Shift instructions ending in zero use the R0 register for the shift amount, otherwise are shifted by 1.

It's failing on Add - your results look good but your status flags were not as-expected. if he's going alphabetically, you've got a hill to climb

 

[+khanivore]

Looks like the parity bit - Yeah I never bothered implementing that one

[PeteE]

1 hour ago, khanivore said:

Looks like the parity bit - Yeah I never bothered implementing that one

FYI the tester uses an AB instruction to set the flags to a known value before testing each instruction twice:

 

       LI R2,>A000
       AB R2,R2   ; set status LGT AGT C OV P
and
       CLR R2
       AB R2,R2   ; set status EQ

 

After each instruction, the status flags are compared to the expected flags.  By using two sets of flags, we can determine if the instruction only modifies the flags that are intended.

 

Hopefully, once you get the parity check added to the byte-variant instructions, most tests should start passing.

[webdeck]

7 hours ago, Asmusr said:

What (most) other emulators do, is to draw one scan line, then run the CPU for the corresponding number of clock cycles, then draw the next scan line, and so on. But if you could get the threads to synchronize, I guess that would be a more realistic, and maybe faster, emulation.

Could someone point me to what the proper timing is to use for the VDP drawing?  I've been looking for it, but I can't find it.

[Tursi]

TMS9918 (vdp).pdf

 

Page 3.8 has the timing in pixel clocks and lines, which is what the VDP uses internally. You can find the other timing nearby. Note the "approximately". Real hardware is allowed to be off by 0.1% based on the input clock (page 5-4).

 

The concept of synchronizing VDP to CPU per scanline is probably the best approach for a current emulator, trading off performance for accuracy. The CPU can't really synchronize to the VDP too much tighter than that anyway. Every few scanlines seems to generally work for the DS emulator too, which is fair - it takes the CPU more than a scanline to react to a scanline detection anyway.

 

 

 

[webdeck]

On 2/22/2024 at 4:04 PM, webdeck said:

I've got what I assume is some sort of timing issue with Parsec - the enemy ships are flying down too low when they first appear and the ones that come from behind are wrapping all the way around off the right and back to the left again when they come out.  I also see the explosion that appears when I hit a ship to be shifted to the left of where the ship was by a good amount.

 

Ok, the explosion appearing too far to the left was a silly early clock bit mistake in sprite rendering (was looking at the wrong bit.)  As for the Parsec timing issues, it seems to be related to instruction timing - I need to check my math on instruction times because it seems I'm running a bit too fast.

[webdeck]

Yep, instruction timing.  I was trying to be super-accurate by sleeping after every instruction, but the sleep resolution isn't granular enough for that, and despite my trying to compensate, I was running a bit too fast.  Now I'm accumulating the sleep amount after each instruction and only sleeping after it gets large enough to warrant a sleep call (still fine tuning what that number is), and that seems to be working better.

[+khanivore]

Any wake up on a multi user OS is not going to happen at a precise time unfortunately unless you are running on an isolated CPU core.  What I did was create a recurring timerfd and do a blocking read on the socket to synchronise.  At least that averages out at the right rate.  I didn't find any VDP timing issues but when it came to cassette load and save I had to jump through a few hoops.

[webdeck]

On 2/23/2024 at 6:48 PM, Tursi said:

TMS9918 (vdp).pdf 5.61 MB · 3 downloads

 

Page 3.8 has the timing in pixel clocks and lines, which is what the VDP uses internally. You can find the other timing nearby. Note the "approximately". Real hardware is allowed to be off by 0.1% based on the input clock (page 5-4).

 

The concept of synchronizing VDP to CPU per scanline is probably the best approach for a current emulator, trading off performance for accuracy. The CPU can't really synchronize to the VDP too much tighter than that anyway. Every few scanlines seems to generally work for the DS emulator too, which is fair - it takes the CPU more than a scanline to react to a scanline detection anyway.

 

 

 

Thanks - I missed this manual - I was using a different TMS9918A reference.