Coleco Emulation - What do these games have in common?

[Ikrananka]

8 hours ago, llabnip said:

Freewheeling Games released the FW Diag rom some time back:  http://fwgames.ca/roms_new/ 

Thank you ?

[artrag]

Try to have a look at openmsx as reference. It is open source and its vdp implementation is almost perfect,  it supports colecovision and colecovision + SGM

 

[NIAD]

Picked up a DSi and been having a blast playing CV games thru ColecoDS. THANKS!!!

 

From reading thru this thread, I know your knee deep in trying to solve some of the compatibility issues with a small amount of games, but wanted to ask if you have plans to:

 

- support the 3rd and 4th firebuttons of the SAC. Besides the obvious sports games, there are a couple others that can be configured thru in-game selections to use them… Stat Trek and Spy Hunter come to mind.

 

- support the SAC Speed Roller via the touch screen for the sports games, say via a slid bar with sensitivity settings that could also be configured as Left and Right controller movement to be used with games like Kaboom and Turbo SCE.

 

- overlay pic support… think I saw mention of this being a possibility in a future release.

 

- possibility to increase the size of the keypad display on the touch screen and make the menu options list smaller. I know I could always just assign keypad buttons to the X, Y, Left and Right buttons as an alternative input means, but a slightly larger keypad display could prove beneficial especially for a game like Mouse Trap.

 

 

[+wavemotion]

1 hour ago, NIAD said:

- support the 3rd and 4th firebuttons of the SAC. Besides the obvious sports games, there are a couple others that can be configured thru in-game selections to use them… Stat Trek and Spy Hunter come to mind.

 

- support the SAC Speed Roller via the touch screen for the sports games, say via a slid bar with sensitivity settings that could also be configured as Left and Right controller movement to be used with games like Kaboom and Turbo SCE.

 

- overlay pic support… think I saw mention of this being a possibility in a future release.

 

- possibility to increase the size of the keypad display on the touch screen and make the menu options list smaller. I know I could always just assign keypad buttons to the X, Y, Left and Right buttons as an alternative input means, but a slightly larger keypad display could prove beneficial especially for a game like Mouse Trap.

 

 

You’re a couple of revs behind

 

Keypad is bigger. Overlays are in. 3rd and 4th buttons are supported. The full SAC is coming soon…  I’m releasing almost daily until most of what I want gets in! Enjoy. 
 

 

 

Edited December 15, 2021 by llabnip

[NIAD]

4 hours ago, llabnip said:

You’re a couple of revs behind

I blame it on having to wait for the seller to get in one of the colors that I liked. I'm also not used to having to check daily for emulator updates... lesson learned. Thanks again.

4 hours ago, llabnip said:

Keypad is bigger. Overlays are in. 3rd and 4th buttons are supported. The full SAC is coming soon…  I’m releasing almost daily until most of what I want gets in! Enjoy.

See people, ask nicely and you get what you want! ?

 

I'll be checking your GitHub daily as as far as "until most of what I want gets in", what about what we want??? ?

 

[zyzzle]

This emulator project looks really wonderful, but the trouble seems to be how to find a reasonably-priced DSi XL to run it on. NAID mentions having to wait to get a good color, but how much did he actually pay? Ebay looks scummy and scuzzy, with most of those scalpers either selling beat-up, broken, or Japanese editions of the DSi XL. What is a reasonable price? I would think $50 complete. But, I'm seeing nowhere near that unless one wants a scratched, crappy version without an AC adaptor.

 

And a big, huge THANK YOU to llabnip for resurrecting this emulator from the dead and making so many wonderful new features available. I honestly know that individuals with your dedication, coding ability, and willingness to please (!) are about as rare as 25K gold! The DSi does seem the *perfect* system, as it is pixel-perfect for Coleco games! I wish I could give you pointers on the "missing 12" games, and it is indeed a conundrum, as you said it probably relates to a timing problem and not a Z80 emulator core problem. I wish you the best in brainstorming sessions, luck, and serendipity in getting those last games working. (I feel their coding is testing the physical limits somehow, as 4 or 5 are from the same company / programmer).

[+wavemotion]

Definitely keep looking... I got most of my units on eBay. You should be able to get a reasonably good DSi LL (Japanese Version - you won't care as you'll be using custom firmware anyway to render stuff in English) for about US$60 shipped.  I got a couple of mine for under $50 but most were in the $60 range.  A normal sized DSi should be in the US$40 range but I'd recommend the XL/LL as the better all-around handheld for emulation of classic 8-bit systems. The screen refresh on the XL/LL is a little slower and so it "holds on" (for lack of a better technical term) to the pixels a little longer semi-simulating the phosphor effect.  This can be seen in games with very small bullets (e.g. Space Fury or Galaxian) where on a normal DSi they can be very hard to see as they travel fast enough that the faster screen refresh makes them hard to see.  I added a "blend mode" to ColecoDS to combat this - it effectively OR-blends 2 frames to give that illusion of slow phosphor decay - it comes at the cost of some CPU/performance which, fortunately, the DSi can spare (but just barely!). The DSi XL/LL renders these images without the blend mode quite nicely.  

 

Some of my other DS emulators don't have this blend mode (either because there just isn't enough CPU bandwidth or because of the way the screens are being rendered) - so the DSi XL/LL ends up being the real winner here.

[NIAD]

What llabnip said (^^^^) re acquiring a DSi XL or LL. I ended up with a Japanese LL model as I was not concerned with using the DSi for anything other than emulation especially since one can play all the DS games (or most, haven’t played those enough yet) from rom images on a modded system. In the very small chance that I end up wanting to use the DSi LL in normally, should be easy enough to figure out the Japanese text menus… but as was mentioned the Homebrew Launcher text is configured in English.

 

I had a modded PSP in the past and could have modded my son’s PSVita since he hardly uses it anymore, but when I started looking into a portable emulation system, the DSi ended up being the best option for what I wanted with the added push coming from ColecoDS being actively developed with most of all the modern features like SGM and MegaCart PCB support… if F18a support could be added in the future, that would just be the icing on the cake, but so few games truly utilize it besides the added benefit of no sprite flicker which can be done in emulation anyway.

 

Checkout Facebook Marketplace and you should be able to find someone local where you can actually see and test the unit. An OEM power adapter would be a nice include but thankfully  it’s a standard mini-USB port so you probably have what’s needed already if not included in a purchase.

 

While CV emulation is the most important feature for me, I’m really interested in trying out some of the computer emulators that use/display the systems keyboard in the Touch Screen.

[+wavemotion]

44 minutes ago, NIAD said:

...the added benefit of no sprite flicker which can be done in emulation anyway.

Right - I do set the max sprites per line to 32 by default - so you shouldn't see any flicker due to 4+ sprites on a line (you may still see flicker due to how the game was programmed). 

 

Just released 4.3 which fixes a few CPU timing issues (someone noticed the enemy cars not spawning in Spy Hunter in the last few builds... oops!).  Still working on the core games mentioned in this thread as time permits. 

[NIAD]

2 hours ago, llabnip said:

Right - I do set the max sprites per line to 32 by default - so you shouldn't see any flicker due to 4+ sprites on a line (you may still see flicker due to how the game was programmed). 

Have seen that in F18a videos of some games like Antarctic Adventure when the penguin falls in an opening in the ice and his entire body is still visible instead of the bottom half being concealed since he’s in a hole. Thankfully with the CollectorVision Phoenix, the F18a option can be turned on/off as needed.

2 hours ago, llabnip said:

Just released 4.3 which fixes a few CPU timing issues (someone noticed the enemy cars not spawning in Spy Hunter in the last few builds... oops!).  Still working on the core games mentioned in this thread as time permits. 

Just when I thought I had caught up! ?

[+wavemotion]

Just now, NIAD said:

Have seen that in F18a videos of some games like Antarctic Adventure when the penguin falls in an opening in the ice and his entire body is still visible instead of the bottom half being concealed since he’s in a hole. Thankfully with the CollectorVision Phoenix, the F18a option can be turned on/off as needed.

Interesting - perhaps I should make it configurable where the sprite limit is!  That's easy enough.

 

Quote

Just when I thought I had caught up! ?

 

[Ikrananka]

5 minutes ago, llabnip said:

Interesting - perhaps I should make it configurable where the sprite limit is!  That's easy enough. 

Highly recommended to do that.  There are a few games (at least Antarctic Adventure and QBIQS) where the normal sprite limit of 4 is used to enact a form of sprite masking, so being able to switch between a sprite limit of 4 and 32 is important.  I actually have a physical switch on my F18A modded CV to do just that.  Some info on this in the thread below.

 

 

[+wavemotion]

I didn't realize there were games that took advantage of the 4 sprite max limit!  Neat.

 

I just checked in a build with configurable support (default is 32... but can change to 4). I also default that parameter to 4 specifically for Antarctic Adventure (plus prototype) and Qbiqs.

 

And, as with any change to the VDP or CPU, I ran the "problem twelve" through again to see if there were any improvements. Nothing changed on that front.

[NIAD]

40 minutes ago, llabnip said:

I didn't realize there were games that took advantage of the 4 sprite max limit!  Neat.

 

I just checked in a build with configurable support (default is 32... but can change to 4). I also default that parameter to 4 specifically for Antarctic Adventure (plus prototype) and Qbiqs.

Ran across another example of this in CPK: Adv in the Park - CRC 6AF19E75 or that first one could be a G. There are numerous versions of CPK Adv out there but this one is the verified good dump from Ikrananka’s ROM Dump project. The issue arises on the second level… after you made it thru the park once and are on the next run.

 

At the scene with the 5 puddles/little pools of water that the red fish jump out of, the fish are actually visable as if laying on the ground.

 

Make it to run 3 through the park and at the camp fire scene, the fire ball does the same thing instead of shooting out of the fire.

 

 

[Ikrananka]

That's cool to know NIAD.  I must document all this somewhere, perhaps in the collectors list......

[NIAD]

6 hours ago, Ikrananka said:

That's cool to know NIAD.  I must document all this somewhere, perhaps in the collectors list......

While I think the F18a VGA upgrade is/was a great upgrade, I just could never pull the trigger on having it installed in one of my ADAMs out of concerns for compatibility issues especially since there is no way that I could expect Matthew and others to test all the ADAM software (along with CV games) and be as familiar with them as I am. Also, the composite video output that I get from my two systems is very crisp and I am more than happy with it. Different strokes for different folks and I made someone very happy when I sold my unused F18a to them seeing as they are no longer available.

 

Did you test CPK Adventures on your F18a modded system?

 

[Ikrananka]

9 hours ago, NIAD said:

Did you test CPK Adventures on your F18a modded system?

Not recently, but I'll do a dedicated test over the weekend.  I'll try it with the sprite limit set to both 4 and 32.  It works great with Antarctic Adventure, i.e. when the sprite limit is set to 4 the sprite masking works as it should.

 

I've used the system a lot over the years and am extremely happy with it.  It met my expectations and needs which is all that matters for anyone who either uses stock systems and/or modded systems.  As you say, different strokes.  That said, I was recently beta testing a game that had extremely tight VDP routine timing and it caused occasional graphical issues on my F18A system that didn't occur on a stock system.  Most, but not all, of the problems were eliminated with a rewrite of the VDP routines.  I can't say that I can recall issues with any other games.

 

Anyway, I'm now scouring eBay and Kijiji for a DSi XL/LL thanks to this thread ?

 

 

[+wavemotion]

Just a brief update... I've added a sort of mini-Z80/VDP debugger into the emulator to see if it helps identify where the problems are.  Unfortunately this effort didn't lead to much as I'm just not good enough with the various chips being emulated as I'd like - however, there were a few things that jumped out. 

 

[Note: The emulated Z80 registers used by the DrZ80 core are 32-bits wide to facilitate mapping onto modern processor cores which is how it achieves relatively blazing speeds. A decade or two ago, many emulators were switching to the DrZ80 core because it could run games on modest hardware at full speed... but it was known that it was not a perfect core. As hardware improved, more accurate (but slower) Z80 cores could be used instead. However, as I'm developing for a 67MHz ARM core handheld (roughly the equivalent of a mid-range 486PC), I'm trying to not bring in a new Z80 core that will render the 300 working games so slow as to be unplayable... so I continue down this path of trying to improve the compatibility of the core I've got. It's the devil you know ....  ]

 

First - the few games that crash some random # of seconds into the game... in every case the Z80 stack pointer (SP) went ended up as an out-of-range value and was fetching garbage and the program just jumped off into Z80-space. 

 

Second - Super Pac-Man is hammering the enable/disable of VDP interrupts (via bit 0x20 of VDP[1])  during initialization - it appears several hundred times per second. And I think the Z80 core I'm using will only generate an IRQ at the end of a given scanline (i.e. it doesn't check between instructions if the pending IRQ is set).  I see all kinds of garbage on screen - and the VMODE is often nonsensical (often one of the undocumented "strange" modes... like 1+2+3 where it just shows 40 vertical bars of colors). Once it gets through the init sequence - the game will play fine except there is garbage on screen.   The 'Under4MHz' ports appear to be doing similar things except those tend to result in a crash.  There are other working games that disable and enable via that VDP[1] bit 0x20 (much less frequently, however) and those seem to be running fine.

 

Third - Deep Dungeon Adventures appears to be running in a sort of loop executing the same set of code from bank 0, 1 and 6... and will respond to a press of  '5' on controller 2 to run the Easter-egg game but nothing ever shows on screen. The # of IRQs is incrementing very slowly - a couple of times per second. I can't tell if it's resetting or if it's just running really slowly...   

 

Fourth - Uridium appears to lock-up about 1 second after enabling the SGM memory (base of 0x0000 so it's disabled the BIOS) - the image below is the state of the world when the program halts. You can see there was never a single NMI IRQ generated - interrupts were never enabled via VDP[1].

 

Most of the games that are working fine will generally leave the VDP interrupts enabled and the IRQ counter just ticks along as you would expect (60 counts per second corresponding to the NTSC version of the TMS9918 chip). 

 

I can't help but feel the problems here are all related to how accurate the VDP-to-CPU interrupt is being emulated.

 

Edited December 20, 2021 by llabnip

[NIAD]

We all appreciate the hard work and dedication you are putting forth on ColecoDS! Wish I had the technical chops to help you figure out some of these remaining games issues.

 

I will be testing out the games that you asked about with the latest build and report back.

 

Also, check www.adamarchive.org later tonight for the emulator source files that could help with that other thing that you are considering adding support for. I have a few more things to sort thru before I upload them.

[Tursi]

On 12/20/2021 at 6:29 AM, llabnip said:

I can't help but feel the problems here are all related to how accurate the VDP-to-CPU interrupt is being emulated.

Nice research!

 

It might be worth checking this edge case: VDP end of frame has already happened, so the bit is set in the status register, but interrupts in the VDP were disabled when it was set. Later, interrupts are enabled -- this should immediately trigger an interrupt to the CPU.

 

The output of the interrupt line is the interrupt enable bit in Reg1 AND'ed with the Frame bit in Status. So if the frame happens while interrupts are enabled and the emulator only checks whether to trigger an interrupt at that point (rather than treating the output as a continuous operation that must be continuously checked), you can miss the interrupt. Missing interrupts in some games will lock them up.

 

[+nanochess]

I've ported Deep Dungeon Adventure to Colecovision, and I've wrote the CoolCV emulator.

 

1. It is terribly important the Z80 T-state counting (or clock cycles). Each instruction should take the right cycles (from the Z80 datasheet), plus one extra for each byte read (internal Colecovision wait time). (check the column Z80+M1 in http://map.grauw.nl/resources/z80instr.php )

2. The VDP interrupt is generated after displaying row 192 of video (more or less once each 59569 clock cycles)

2.1 The VDP generates 262 lines (each line takes 228 clock cycles)

3. The interrupt disable bit in the VDP register 1 disables directly the VDP interrupt. So even if reached row 192, it doesn't appear on the output line, but stays set internally.

4. The VDP interrupt is connected to the NMI input of the Z80, and the NMI input pin of the Z80 works by recognizing the fall from one to zero. (not level zero like the INT input pin, on a side-note because of this the Z80 requires INT to be zero for at least 32 cycles)

5. The VDP interrupt is only reset by reading the VDP status register (this works even if the interrupt disable bit is set)

 

This means the following subtle interactions can happen:

 

1. If the VDP generated an interrupt, BUT THE INTERRUPT DISABLE BIT OF VDP IS 0, the NMI pin is 1. But if the interrupt bit in VDP register 1 is set to 1, the NMI immediately kicks in because the change from 1 to 0.

2. Many games depend on an interrupt not happening in a determined time frame. (because of this is important the T-state counting and the interrupt being generated after line 192).

 

I hope this information is useful for you.

 

I don't remember right now if Deep Dungeon used the IXH,IXL,IYH,IYL special instructions, I think it could have been because of the music player, but the code is on a machine that I don't have available at this time.

 

[+wavemotion]

Thanks Óscar - as always, you're knowledge is helpful!

 

I spent a few hours grafting in a new Z80 core for the ARM processor I'm using.  It's the Z80 core used in fMSX as the one in CoolCV couldn't be located

 

And, lo-and-behold, it does work better.  All of the "problem twelve" now work... though Sudoku went back on the problem list with new graphical glitches.

 

However, the speed will be the death of me. As expected - the higher compatibility has come with some speed challenges on the ancient handhelds. 

 

The original DS hardware (67 MHz ARM) can't keep up on any game - though the faster CPU DSi/XL/LL (133 MHz ARM) does handle more than half of the library at full speed... Mega Cart bank switched games and games utilizing all 6 sound channels (SN+AY for Super Module Games) tend to slow down considerably. 

 

Most of the games I'm having problems with do run full-speed with the new core on the DSi so I'm probably going with the following strategy:

 

Support two independent Z80 emulated cores... One is "High Compatibility but Slow" and the other is "Lower Compatibility but Fast" - and for the older hardware, I can keep using the DrZ80 core for blazing speed and 95% compatibility and the slower more accurate core can be used for the games that need them.  I'll let the user override the defaults and switch cores as they like.

 

I continue to be appreciative of all the well-meaning support here!

[Tursi]

I wonder, what if you hacked a debug channel into one of the DS emulators, and dumped a Z80 CPU trace from each core, and try to see where they diverge? It's a lot of output, but a good diff tool should be able to help.

 

I've spent most of this week doing just that with my own emulator, hehe

 

[+wavemotion]

Good idea, @Tursi - something to think about. 

 

Right now I've got both Z80 cores working in the same build - I default to the DrZ80 "fast but not perfectly accurate" except for 18 games (everything by Under4MHz - there were more than I thought, plus those mentioned here). The user can switch cores when loading a game (and save the core so-chosen on a per-game basis as a default).  

 

All games are now running smoothly - though the new Z80 core will be too slow on the older DS-LITE / DS-PHAT handhelds (DSi and above required - and really the DSi LL/XL is the preferred handheld for virtually unlimited 8-bit emulation). 

Deep Dungeon Adventures has periods of brief slowdown - mostly on the "flame" title-screen... but it's very playable and seems to hold 60FPS during actual gameplay. Super Space Acer is struggling to keep up even with the DSi on the new core but it's close. And now I have two avenues to attack improvements in the future:  speed up the more accurate Z80 core (I'm good at this) and/or improve the accuracy of the drZ80 core (I've proven to be less good at this  )

 

 

Edited December 22, 2021 by llabnip

[Tursi]

8 hours ago, llabnip said:

Super Space Acer is struggling to keep up even with the DSi on the new core but it's close. And now I have two avenues to attack improvements in the future:  speed up the more accurate Z80 core (I'm good at this) and/or improve the accuracy of the drZ80 core (I've proven to be less good at this  )

That's surprising to me... but I guess I don't know what's going on under the hood. However, SSA hits the VDP really hard, especially during the bosses. I wonder if optimizing the VDP draw code might help? Earlier versions of Classic99 had performance hits on heavy VDP software caused by the emulator triggering the screen draw more than once per frame in some cases.. since it was a full frame renderer rather than a scanline based one, there was no point doing so, it just wasted CPU time.