Jump to content
IGNORED

Coleco Emulation - What do these games have in common?


Recommended Posts

For the past month I've been working on updating ColecoDS - a Colecovision Emulator for the DS/DSi. It's based on the ColEM VDP (tms9918a) core (which I realize has not been perfect) with a DrZ80 emulated CPU core.

 

Of the 300 games I've played, 95% run really well. But there are some standouts that won't yet run. I've done quite a bit of debugging but am perplexed enough that I'm going to try reaching out for help here in case someone has any insight as to what these games might be doing ... differently. Maybe there is a pattern - and even the smallest clue can go a long way to finding a needle in the proverbial haystack!

 

The first two games on the non-compatibility list both come from Trilobyte:  Deep Dungeon Adventures and Uridium.  Both give just a blank screen - though both are "running" Z80 code at least enough to enable the MC and SGM algorithms. Both are 128k megacart games utilizing the SGM and yes, many other 128k (and 256k, 512k) MC games with and without SGM and AY sound are running fine on the emulator. Deep Dungeon Adventures seems to be running and bankswitching but constantly resetting - though interestingly enough if I press the magic '5' button on both controllers I can get into the hidden "Easter egg" game and play that fine. 

 

The next game is Super Pac-Man.   This one has major graphical glitches - I'm fairly sure I've got the "undocumented" VDP modes working (many other games were graphically glitchy without that) but maybe it's doing something even more unusual. The game will play (and sound) fine aside from the major graphical artifacts on screen.

 

The next game is Super Space Acer which runs for about 10 seconds and then crashes. The crash is not always at the same point or time... but it never runs more than about 10 seconds. 

 

The last set of games is pretty much everything ported by Under4MHz. For some reason those just crash even before they get going and/or show complete garbage on screen. A few crash hard enough to lock up my Z80 emulation core.

 

Thoughts of any kind?  I'm half suspecting my old DrZ80 core which is all in assembly (for speed on these older handhelds) but my skills in ARM Assembly are mediocre at best. But maybe it's something more simple than that.

 

Again, even a small clue can go a long way.  I don't mind giving up - but not without a reasonable fight :)

 

 

1636392804217.png

Edited by llabnip
  • Like 4
2 minutes ago, alekmaul said:

I think the problem can be the video chip emulation with interrupt trigger. Perhaps it is not good in ColecoDS.

You're probably right, Alek. I've gone through the latest ColEM 5.6 TMS9918A code as well as MAME and some of the updates to other projects (like PicoDrive) and the original core that was used in ColecoDS was, indeed, quite buggy. That accounted for some 30+ games not running right. I've fixed / patched most of that and most of those games started to work fine but these others are stubborn and won't run. But you're probably right - I've probably only "improved" it but not gotten it quite accurate enough.  

Unfortunately nothing I can do as far as helping with the issues you are trying to fix with certain games, but just wanted to say Thanks for all your hard work on ColecoDS. I’ve previously used a PSP for on the go CV gaming but the DS lineup, especially the DSi XL, looks like the ideal portable system for the CV with it’s dual screens.

 

Hopefully one day you might consider adding ADAM emulation, time permitting.

  • Like 1

Interrupt trigger is definitely the source of most issues on the ColecoVision.

 

There's nothing too special about Super Space Acer, it doesn't try any significant tricks -- but there /are/ some heavy VDP copy loops that might have issues if the interrupt period is too fast (I'd have to double check the code). It's also a WIP, incomplete project and I'd make sure I was running the latest released version from the thread here on AtariAge, and not from any of the download sites or ROM packs who offer it against my will. ;)

 

I'd be curious what the crash looks like for that reason - is it graphical corruption or does the Z80 actually jump off to bad code?

  • Like 1

Thanks for the response, Tursi! I have the Nov-2016 ROM of the game - sourced here at Atariage.  

When it freezes, the PC runs for another few seconds after the screen stops updating. Sometimes the screen has garbage after that... sometimes it just freezes solid. 

I've attached a screenshot of a freeze... the PC is stuck at that value (the delta between the ARM9 base and the PC is the Z80 PC address it's stuck on). 

 

image.thumb.png.18b45e0fb9aa34bb89e879b634e96b07.png

 

And to be clear - the obvious answer to "what do these games have in common" is "me - and my buggy emulator!" :) but it will be improved given time...

 

I'm definitely still coming up to speed on how the VDP interrupt stuff works - but my general understanding is that the VDP can generate an interrupt to the CPU (on NMI which seems rather silly but whatevs!) and the CPU clears that interrupt by reading the VDP status register. So I'm unsure how it could be longer or shorter than in other emulators - but I'll admit this stuff is still a bit of a mystery as I come up to speed!

 

Edit:  the freeze happens in other places ... it seems rather random what address the PC finally halts at. The game does run for several seconds where the screen is either frozen or has garbage showing before the Z80 code finally comes to a stop. 

Edited by llabnip
  • Haha 1

( I just retested and played through the whole thing here on my emulator. I'm currently using a Z80 core by Manuel Sainz de Baranda y Goñi - https://github.com/redcode/Z80 )

 

That's the right release. I'll check later if I still have the map file for that release, figuring out what function it's in might offer a clue.

 

Porting a Z80 test suite might be a good idea if you have doubts about the CPU core, looks like there are a few on Github.

 

Software crashes usually can't be triggered from VDP access (yes from interrupt since it's an NMI here), but if you suspect that make sure your VRAM access correctly - check the emulation of the prefetch. That's critical to making ill-advised tricks like setting a write address but reading (or vice versa) work correctly.

 

Some ColecoVision software seems to be hyper sensitive to the timing of the NMI, so check that timing. As for how it can be different - you need to make sure the CPU gets enough cycles per frame... you should be able to do some math to work out what that is! ;)

 

Also double check the interrupt in the VDP is correctly masked - it should not fire when it's disabled in VDP Register 1, but the status byte must still reflect end of frame is active. If the interrupt bit in VDP register 1 is enabled and the status bit is already set, the NMI should be triggered immediately. And of course clear the bit when reading the status register. (I guess also make sure the interrupt is edge triggered at the Z80 side, not level triggered.)

 

To tie that more directly to the implementation - the NMI line to the Z80 should be active when VDP Register 1 bit 0x20 is set AND the status register bit 0x80 is set, and the NMI itself on the Z80 needs to be edge triggered (ie: an NMI occurs only on transition from inactive to active). Handling it any other way will cause bugs.

 

Also, make sure the status register frame flag (the 0x80) is set after the last visible scanline is rendered, if you're doing scanline emulation anyway. Otherwise you should be able to get away with setting it once per frame.

 

With the success rate you have, I suspect that the only things that are even remotely likely wrong might be the prefetch emulation or gotchas like not triggering the NMI when a disabled VDP Reg 1 bit is made enabled and the frame flag is already set. But it never hurts to sanity check all the parts - bugs hide under assumptions. ;)

  • Like 1
  • Thanks 1

... IF the map I have is correct for that build, then the PC is in the function __div16 (0x8F9A - looks like your PC is at 0x8F9E). Seems like a strange place to hang...

 

When I breakpoint on that address, it does indeed run for a random number of seconds before it hits it, so it's not a function called every frame. This is what the code looks like:

 

  D739 cd82  call    8f82h               (17)  ; main code calling __divsint
       8f  

  8F82 f1    pop     af                  (10)  ; __divsint - seems to mostly be a wrapper (useless stack manip?)
  8F83 e1    pop     hl                  (10)
  8F84 d1    pop     de                  (10)
  8F85 d5    push    de                  (11)
  8F86 e5    push    hl                  (11)
  8F87 f5    push    af                  (11)
  8F88 c39a  jp      8f9ah               (10)  ; jump straight to __div16
       8f  

  8F9A 7c    ld      a,h                 (4)  ; __div16
  8F9B aa    xor     d                   (4)
  8F9C 17    rla                         (4)
  8F9D 7c    ld      a,h                 (4)  ; hang is here or the next instruction??
> 8F9E f5    push    af                 
  8F9F 17    rla                        
  8FA0 3006  jr      nc,8fa8h           
 
(...continues, breakpointed at first hit...)

Here's a register dump...

 af   0100   af_  FFFF    CPU
 bc   0080   bc_  FFFF
 de   0038   de_  FFFF
 hl   0174   hl_  FFFF
 ix   73D0   iy   72AD
 r    39     i    00
 memp 0000   pc   8F9E


VDP0  00     VDP  0701
VDP1  E2     
VDP2  00    VDPST 00
VDP3  0E      PC  8F9E
VDP4  02      SP  73BF
VDP5  06      F   00
VDP6  01     
VDP7  01     

Of particular interest, 'SP' is the stack pointer and the CPU regs are at the top.

Of course, this is the state the /first/ time the code is hit, so it might be interesting to determine if your emulator locks up every time it runs this code or only sometimes... (and also worth double checking if my map is correct and this actually is the code you're getting stuck on).

 

 

  • Like 1
8 hours ago, NIAD said:

but just wanted to say Thanks for all your hard work on ColecoDS.

110% all this! I’m blown away by how well it works! I think the guys at work are finally getting sick of me talking about it. It’s so awesome to have a handheld Colecovision. You are the man!

  • Like 2
1 hour ago, Tursi said:

Software crashes usually can't be triggered from VDP access (yes from interrupt since it's an NMI here), but if you suspect that make sure your VRAM access correctly - check the emulation of the prefetch. That's critical to making ill-advised tricks like setting a write address but reading (or vice versa) work correctly.

Great insight into two areas to check - I'm looking at the data and prefetch stuff now and WKey is set or clear based on 0x40 in the "high byte" control write handler. 
 

ITCM_CODE void WrData9918(byte data) 
{
  if(WKey)
  { /* VDP in the proper WRITE mode */
    VDPDlatch = pVDPVidMem[VAddr] = data;
    VAddr     = (VAddr+1)&0x3FFF;
  }
  else
  {   /* VDP in the READ mode */
    VDPDlatch = pVDPVidMem[VAddr];
    VAddr     = (VAddr+1)&0x3FFF;
    pVDPVidMem[VAddr] = data;      
  }    
  if (bResetVLatch) VDPCtrlLatch = 0;
}

 

But what's strange is that the code base I have, it doesn't bother to check WKey at all for the read routine... what would a real VDP do on a data read if we were in WRITE mode?

 

ITCM_CODE byte RdData9918(void) 
{
  byte data;
    
  data      = VDPDlatch;
  VDPDlatch = pVDPVidMem[VAddr];
  VAddr     = (VAddr+1)&0x3FFF;
  if (bResetVLatch) VDPCtrlLatch = 0;  
  return(data);
}

 

Re: the triggering of interrupts I'll go over that again at with a cup of coffee in the AM based on your possible "difficult area" suggestions. I'm hugely appreciative of your time and expertise @Tursi

 

edit: and one other bit of read ahead logic in the control byte write handler:

 

if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead

Edited by llabnip
21 minutes ago, Zeptari1 said:

110% all this! I’m blown away by how well it works! I think the guys at work are finally getting sick of me talking about it. It’s so awesome to have a handheld Colecovision. You are the man!

Thanks for the encouragement, Z. This is based on the original port by Alekmaul so he’s the man and I’m the second man with more time than brains. Or something like that. 

 

The Colecovision works well on the DS (DSi/DSi-XL) as the screen resolution 256x192 - pixel perfect to real CV output. Same for MSX1, SMS, SG1000, etc.  The near-dead-nuts-perfect Nintendo engineering and a good touch screen is a nice bonus :)

 

Edited by llabnip
1 hour ago, llabnip said:

Great insight into two areas to check - I'm looking at the data and prefetch stuff now and WKey is set or clear based on 0x40 in the "high byte" control write handler. 
 

ITCM_CODE void WrData9918(byte data) 
{
  if(WKey)
  { /* VDP in the proper WRITE mode */
    VDPDlatch = pVDPVidMem[VAddr] = data;
    VAddr     = (VAddr+1)&0x3FFF;
  }
  else
  {   /* VDP in the READ mode */
    VDPDlatch = pVDPVidMem[VAddr];
    VAddr     = (VAddr+1)&0x3FFF;
    pVDPVidMem[VAddr] = data;      
  }    
  if (bResetVLatch) VDPCtrlLatch = 0;
}

 

But what's strange is that the code base I have, it doesn't bother to check WKey at all for the read routine... what would a real VDP do on a data read if we were in WRITE mode?

 

ITCM_CODE byte RdData9918(void) 
{
  byte data;
    
  data      = VDPDlatch;
  VDPDlatch = pVDPVidMem[VAddr];
  VAddr     = (VAddr+1)&0x3FFF;
  if (bResetVLatch) VDPCtrlLatch = 0;  
  return(data);
}

 

Re: the triggering of interrupts I'll go over that again at with a cup of coffee in the AM based on your possible "difficult area" suggestions. I'm hugely appreciative of your time and expertise @Tursi

 

edit: and one other bit of read ahead logic in the control byte write handler:

 

if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead

I don't know what "WKey" is... but I can almost guarantee it's wrong. Here's the gotcha which is really badly documented: the VDP does not have "Read mode" and "Write mode".

 

When you set an address, the two most significant bits are control bits. However, all address register writes first load the 14-bit address register. A flip flop tracks whether the LSB or MSB is expected - this flip flop is reset to LSB when the status register is read or the data port is accessed.

 

When the second byte (MSB) is received, the 6 valid bits are stored in the address register, always.

Next, if bit >80 is set, the data is used to write to the specified VDP register.

Finally, if bit >40 or >80 is set, the operation is complete. Otherwise, a prefetch is performed - data from the requested address is stored in the prefetch register, and the address register is incremented.
There is never again any interest in what the control bits were.

So the >40 bit is not a "write mode" specification, rather it is a "prefetch inhibit".

For any read request, the CPU gets whatever value is in the prefetch register, and another prefetch is performed.
For any write request, the data is stored in the prefetch buffer. At the VDP's next access slot, the data is stored to the current address, and then the address is incremented. (Emulation usually ignores the access slots - this is okay, it just won't be able to simulate VDP overrun).

So note that there are all kinds of edge cases with those sequences. You could write a byte to VDP RAM, then read that same byte back from the prefetch buffer even though the VDP address doesn't point there anymore. You could set a 'write' address, then read back a byte of whatever-was-there data from the prefetch while the next byte will be correct. Or you can set a read address, then without reading write to the next VDP address (I regret to note I've used this one to save bytes before ;) ).

 

So to that, the read function looks okay, though I don't know why it has a condition on the latch. The write function should always be the first case (WKey is true). And you'll want to check that the address set function handles the prefetch correctly, since the existence of that flag makes me suspect it doesn't...
 

(Small edit - the prefetch inhibit is on BOTH control bits, obviously ;) )

 

Edited by Tursi
  • Like 1

Just saw the bit in the control write:

if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead

if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead

This needs to check (value&0xc0), since both bits inhibit prefetch, and it makes the "write in read mode" code doubly wrong since it already increments the address, but with those fixes it should be pretty solid...

  • Like 1
39 minutes ago, Tursi said:

Just saw the bit in the control write:

if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead


if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead

This needs to check (value&0xc0), since both bits inhibit prefetch, and it makes the "write in read mode" code doubly wrong since it already increments the address, but with those fixes it should be pretty solid...

Yeah, I should have provided both lines in the control function:

 

    if (value & 0x80) return(Write9918(value&0x07,VAddr&0x00FF));   // Might generate an IRQ
    if (!(value & 0x40)) {VDPDlatch = pVDPVidMem[VAddr]; VAddr = (VAddr+1)&0x3FFF;} // If read request, read-ahead

 

This should inhibit if either bit 6 or 7 is set since if the high bit is set, it won't go further than the Write9918() call.

 

Interestingly enough, my last checkin didn't have the WKey read/write logic and so I've revered that change which came in when I was trying other TMS9918a codebases to see if it made a difference.  It made no difference either way. 

 

I also just checked the interrupt logic - so far it seems correct. There are exactly 2 places it can generate the interrupt:

 

1. After rendering the last scanline of a frame, we check if the the VBlank flag was not previously set and the interrupt is enabled. We set the status byte to reflect VBlank in any event.

2. If writing control register 1 and we are transitioning from interrupt disable to enable - we generate the interrupt if the VBlank is also set.

 

We clear the VBlank status when the Z80 reads the VDP control port.

 

I'll keep poking around - whatever the problem(s), they can't hide forever :)

Edit: and I tried several more runs of SSA - the Z80 seems to wedge at many different addresses so it seems somewhat random where it finally stops. 

Edited by llabnip

Ok... so much coffee consumed and I went back over this thread with a new understanding of the VDP thanks to @Tursi

 

think the VDP latching and read/write stuff is OK. I've re-documented the code to indicate that those control bits are not really related to read/write but more "inhibit" bits on read-ahead. 

 

I'm growing convinced that the interrupt stuff in the Z80 is wrong. I'll be honest that it's a weird mash-up of DrZ80 core with external interrupt handlers pulled in from an old MAME source (this is is the scheme I inherited). It uses a dual "pending_irq" and "IRQ" flag scheme which I'm struggling to wrap my head around - but I think the way it works is that when we want to trigger an interrupt, we set the pending_irq flag in the VDP code and the Z80 core will see that and trigger the interrupt. But I'm far from 100% sure.  The external part is written in C (which I understand quite well) and the Z80 core is written in ARM assembly (which I can understand the basics... but don't feel super comfortable with it). I'm really not sure if the interrupts are level triggered or edge triggered based on this "pending_irq" scheme.

 

What is happening is that if either of the 2 conditions for the NMI happen, the "pending_irq" flag is set unconditionally by the VDP code.  I assume that's not right - and the "pending_irq" flag should only be set if we are NOT currently in an active interrupt period.  Alternatively, the Z80 core could/should trigger the interrupt only if that flag is set and we are not currently handling an interrupt condition. I tried to make a change to not set the "pending_irq" flag if the Z80 core has the "IRQ" flag raised already -- but absolutely nothing changed in terms of what runs correctly and what doesn't. Which means I don't yet understand how it works :)

 

Brewing another cup...

 

Edit: with a third cup: perhaps this "pending_irq" is acting like the interrupt control line. Set high or low by the VDP based on the aforementioned 2 conditions that would cause the VDP to assert the interrupt line. I guess it should be up to the Z80 core as to whether that actually triggers an interrupt (rising edge=yes, otherwise no). Now... how to make that happen...

 

Edit#2: After some debug and some trial, I think the NMI interrupt is behaving properly. It's definitely edge triggered and it seems to trigger as expected based on everything I know up to this point.  Maybe there is just some bug in the Z80 core. Swapping that out is a bigger task... on the "ancient" (relatively speaking) DS with a 67MHz ARM processor, I've got just barely enough speed right now to render 99% of the games at full frame rate (only Oscar's Princess Quest with rapid bank switching falls below 60 and only for very brief periods). I'm concerned that a different core will bring new challenges... so I'll probably put out a release with the current DrZ80 core and then work on replacing it.

Edited by llabnip
  • Like 1

I unfortunately don't know the Z80 well enough to help there, but even today it seems that Z80 bugs are not uncommon, sadly. But it's weird that SSA just stops executing when most other software has no issues... unfortunately it's compiled code so I'm not sure what might be unusual.

 

  • Like 1

Thanks @artrag — it’s clearly something I’m doing wrong. Uridium freezes very soon after it starts. But DDA seems to be in a loop - bank switching and looking for something. If I hold the magic 5 keys on both controllers, it will go into the bonus game and play that fine. I feel like it’s expecting some input or scanning some port maybe I’m not emulating well enough. It never crashes but it also never shows anything on screen unless I get into the Easter Egg game. 

Edited by llabnip

An update of sorts.

 

I spent yesterday pouring over the 106 page TMS9900 VDP manual plus Sean Young's excellent "Almost Complete TMS9918A VDP" documentation. 

 

I removed all of the little hacks and odd bits of code in my TMS9918a code - and commented the hell out of it. 

 

And guess what?!  Drum roll please.....

 

Nothing changed. All 300 games that run fine, still run fine. The dozen games that don't, still don't.

 

Well - not quite true. There was a small graphical glitch in BitCorp's Meteoric Shower that is now fixed as well as some graphical artifacts in Nova Blast that appear to be corrected.

 

I feel better about the VDP stuff - but it's got to be down to interrupt/timing/Z80 that isn't quite right. 

 

One thing I did learn is that many emulated TMS9918/28 cores have it wrong - including ColEM. But because many games are "well behaved" in terms of their setup and use of VDP, it doesn't matter much.  Even some of the official documentation is a little confusing in spots. 

 

 

The hangs you're reporting don't feel like VDP, but it's good to eliminate it since so many Coleco issues due come back to it.

 

And yeah. I found bugs in BlueMSX that were affecting SSA's dev, too. I run my own these days.. after 25 years of dev on the same codebase I suspect I'm probably close. ;)

 

Were you able to try a Z80 testbed? I'd suggest that's the next thing to try. I know the 6502 has test suites designed to prove emulation, maybe the Z80 does too?

 

  • Like 1

I've not tried a Z80 acid test yet... the DrZ80 core is written in ARM assembly and integrated into the emulator so I'd have to find something that would run much like a Coleco ROM.  For my Atari 800 emulator, I was fortunate enough to have access to Acid800 written by Avery Lee (Altirra) who really knows his stuff. I don't pass all of the tests... but I pass enough that the compatibility is high.   On the Intellivision front, we've got FWDIAG and MOBCOLLISIONTEST - neither of which is quite an acid-test of emulation but they cover enough functionality of the STIC and PSG and various quirks of the system that you can get a good feel for how compatible your emulator is.

 

I've found Colecovision test roms of the same high quality to be somewhat sparse... the best I have is a few controller testers plus nanochess' excellent SGM Test program (which helped me get my SGM stuff working).

 

I think I'll probably graft in a 2nd (probably much slower) Z80 core that is known to have good compatibility.  Even just to prove out that the core is my issue. 

In the meantime, I'm cleaning up some other stuff that emulators seem to do but I can't find any actual proof/documentation to support it. For example, several emulators trigger MC bankswitching on "hotspot" writes as well as reads - but I think all documentation points to reads only. Also, at least one popular emulator seems to think writing to 0xFFFF is a way for Super Game Modules to bankswitch based on the 8-bit value written there... but I can't find any real support for that. I've pulled all that crap out too as some quick experiments concluded that nobody seems to actually do that in practice.  I'm prepared to be corrected on this front, however :)

Edited by llabnip
2 hours ago, llabnip said:

I've found Colecovision test roms of the same high quality to be somewhat sparse... the best I have is a few controller testers plus nanochess' excellent SGM Test program (which helped me get my SGM stuff working).

Do you have the attached?  These are about all of the system (excl. sound and controller) test roms that I have.  Perhaps one of these might help.

 

BTW - where does one get hold of the FWDIAG for the Intellivision?

ColecoVision System Tests.zip

Edited by Ikrananka
  • Like 1
6 minutes ago, Ikrananka said:

[Colecovision System Tests.zip]

Yes, I have that. One of the things that I do when I work on a new Emulator for the DS/DSi is to scour Atariage forums going back about 10 years (that took a LOT of time for StellaDS!).  Man there is a lot of drama here sometimes :D  But I scour the forums for bits of information, odd roms that might not have surfaced anywhere else, other people struggling with compatibility issues, etc.   In any event, in my CV forum scour, I did come across the zip of accumulated test roms - and they are helpful. 

 

Freewheeling Games released the FW Diag rom some time back:  http://fwgames.ca/roms_new/ 

  • Thanks 1

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...