FPGA Based Videogame System

[CZroe]

I'd agree it shouldn't come into play, but it does. For simpler systems with an 8-bit data bus, it certainly shouldn't be an issue since you are reading data in small enough chunks that it is effectively a 1:1 of the EEPROM contents. NES, Master System, and SNES for example.

 

Things get more complicated as you start dealing with 16-bit data buses present on the Megadrive, or the N64. A backup tool will generally dump the cartridge a word at a time using the cartridge bus, where a word is 2 bytes in these two examples. So when I write it out to storage, should it be LSByte first, or MSByte first? If I just record these values into a buffer and flush the buffer as a series of bytes to storage, then endianness of the system affects how the data is recorded. The 68k is big-endian, while the MIPS chip the N64 has is configurable (although I don't know which mode Nintendo used, or if it was switchable while running). So if I don't standardize on what byte order is in the ROM file itself, then it can't be read properly on the other end without some sort of detection looking for something that can be treated similar to a UTF BOM.

 

And there's an argument to be be made for both LSB and MSB when it comes to consoles like the Megadrive. LSB is more common for EEPROM programmers being used on x86 (DOS generally) at the time. MSB is how the console itself sees it. So which is "real"? At least in the case of Megadrive, the light reading I've done so far seems to suggest that big-endian is the convention, because the original dumps were done on-system. They could have used LSB by byte-swapping things before writing them out in those dumpers to more closely approximate what the raw files written to the ROMs would look like, but they didn't.

 

It should be pointed out that smd is interleaved when compared to md/bin, due to the dumper that produced them operating in Z80 mode (and likely some addressing issues in Z80 mode), so to load those, you have to know that the high and low bytes for each word are 8K apart from each other in the dump, sliced up into 16K blocks. And which ones are the high byte, and which are the low byte. Fun. 

The GameShark Pro/Action Replay Pro for N64 also has two interleaved PLCC package ROM chips... probably so that they could get away with a little CPLD to replicate the rest of the N64's addressing logic. V64jr and PV-Backup both used Altera FPGAs (much more expensive).

 

[spspspsp]

I have spent a while playing with the Noir, and overall I am extremely pleased with it.

 

I am, however, having trouble with one game. I have a Star Versus cart that _will not_ load on FW 6.5. I was able to run this on 1.0, but now it just says "RAM CHECK FAIL" and spits out a hex address. It still runs on my AV Famicom.

 

Has anyone else experienced an issue with Star Versus?

 

I was hoping to be able to use CopyNES mini to dump this (with my own script). I am hopeful that support (mapper 399) will be added in the JB because mapper 413 was just added (also a BATMAP cart).

 

On that note, has any documentation been released for CopyNES mini on how to compile custom scripts? Scripts compiled for older versions don't appear to work.

 

EDIT: Actually, it appears Kevtris addressed this here. I will try using his assembler.

Edited December 23, 2020 by spspspsp

[phoenixdownita]

@CZroe @blzmarcel @Kaide

https://github.com/SmokeMonsterPacks/Nt-Mini-Noir-Jailbreak#intv2-rom-format

 

please remember the Inty is 16bit https://en.wikipedia.org/wiki/General_Instrument_CP1600 and the code words are actually only 10bits long with 6bits "reserved" so it's an oddity already (it appears cartridges are 10bits ROM, 10bits words are called decles).

 

But in specific as the JB states:
"The existing Intellivision ROM conventions were deemed unsuitable to the Intellivision Core."

 

 

[XtraSmiley]

On 12/21/2020 at 10:33 PM, bikerspade said:

Not expecting new features, but rather fixing existing bugs in Hi-Def NES (like the Metroid FDS audio bug).

It doesn't seem like he will, but why not put in the bug report an he may surprise us!

[Pixelboy]

7 hours ago, phoenixdownita said:

But in specific as the JB states:
"The existing Intellivision ROM conventions were deemed unsuitable to the Intellivision Core."

I guess that explains why the Inty core was left out of the jailbreak on the original NT Mini. There was more work to do on it than most people realize.

 

[blzmarcel]

20 hours ago, CZroe said:

NES ROMs, for example, are largely useless without the iNES header that proceeds them. Doesn't matter that they are the exact 1s and 0s if you don't know what to do with them to replicate the rest of the addressing behavior (mappers, banks, SRAM, and such).

I've sometimes wondered why an original NES or Famicom would know what to do without any such header, yet emulators always seemed to need it. Conventional thinking would be if an emulator is doing what an original system was doing in reading and parsing the data on the cart, though there is likely something more to it otherwise this would have been solved a lot more easily.

[CZroe]

I've sometimes wondered why an original NES or Famicom would know what to do without any such header, yet emulators always seemed to need it. Conventional thinking would be if an emulator is doing what an original system was doing in reading and parsing the data on the cart, though there is likely something more to it otherwise this would have been solved a lot more easily.

Because the code works with varying hardware inside the cartridge that the NES emulator wouldn't be aware of to replicate without the header. Also, NES ROMs are generally two ROMs in one, Character (Graphics) and Program, concatenated together. Without the iNES header the emulator wouldn't know where one ROM dump ends and the other begins. For the games without a character ROM, well, they need a header to know that they need to emulate a cartridge's character RAM instead (populated with data from the program ROM).

 

An NES ROM without a header is a bit like dumping your car's ECU ROM and telling a software emulator to simulate a dyno run when it doesn't know anything else about your engine to replicate it. It doesn't even know what engine you have.

 

[Kaide]

9 minutes ago, blzmarcel said:

I've sometimes wondered why an original NES or Famicom would know what to do without any such header, yet emulators always seemed to need it. Conventional thinking would be if an emulator is doing what an original system was doing in reading and parsing the data on the cart, though there is likely something more to it otherwise this would have been solved a lot more easily.

Cartridges on these systems aren’t “inert”. For NES, the mapper chips are on the cart, and sit between the ROM chips and the system. Just dumping the ROM itself isn’t going to include the behavior of those chips.

[blzmarcel]

5 hours ago, Kaide said:

Cartridges on these systems aren’t “inert”. For NES, the mapper chips are on the cart, and sit between the ROM chips and the system. Just dumping the ROM itself isn’t going to include the behavior of those chips.

Thanks. Might I ask why the mapper chips aren't dumped along with the CHR and PRG ROMs? I'm not too keen on what mapper data looks like, I was mostly just aware of the two mentioned ROMs.

[bikerspade]

Even dumping those supplemental chips may not be enough; if there’s a volatile RAM chip or non-volatile EEPROM chip, those may not have data to be “dumped” yet are vital aspects of the cartridge that need to be represented and emulated. There could also be wiring differences on the PCB.

[Kaide]

34 minutes ago, blzmarcel said:

Thanks. Might I ask why the mapper chips aren't dumped along with the CHR and PRG ROMs? I'm not too keen on what mapper data looks like, I was mostly just aware of the two mentioned ROMs.

They are ASICs. Chunks of logic. You can’t really dump them, as there’s no data there, just volatile state.

 

When I say they sit between the ROMs and the system, I mean exactly that. The simplest mappers enable bank switching, which enable more data to be stored on the ROMs than the NES can actually address. The game will need to send signals to switch banks when it needs to access different parts of the ROM.

 

23 minutes ago, bikerspade said:

Even dumping those supplemental chips may not be enough; if there’s a volatile RAM chip or non-volatile EEPROM chip, those may not have data to be “dumped” yet are vital aspects of the cartridge that need to be represented and emulated. There could also be wiring differences on the PCB.

I totally forgot about CHR RAM.

[blzmarcel]

On 12/23/2020 at 3:08 PM, Kaide said:

The simplest mappers enable bank switching, which enable more data to be stored on the ROMs than the NES can actually address.

Thank you, this really helped, as I'm aware of other systems, like the C64, that also used bank switching techniques.

 

Cheers and Merry Christmas if applicable :)

[blzmarcel]

On 12/21/2020 at 10:00 AM, CZroe said:

On 12/21/2020 at 9:27 AM, blzmarcel said:

Could someone please explain why all these issues exist for Intellivision roms? I don't believe I've seen anything like this for emulation, etc of classic consoles from the 70s/80s/early-90s. ROM files are typically images of ROM chips (PRG, CHK, which often combined/concatenated, though sometimes separated, like in MAME) that have been dumped. Binary image should be just that, a binary image, so what is the deal with Intellivision here?

Don't know, but I recall when the byte order of N64 ROMs was a deal because your backup unit used Big or Little Endian. Considering that the Jailbreak is going to run them like a backup unit might then they need to be mapped to memory correctly. Still, I expect it should recognize something like that and flip the byte order if needed.

What I do find interesting is that the NT Mini seems to be the only platform to need the INTV2/.intv format.

 

I noticed, when I was looking through a recent Everdrive pack that I found, that the included Intellivision roms all matched the checksums of the ones that I've had since somewhere around 1999, which also match the ones from both older MESS and newer MAME sets (within the .zip files), as well as the ones used for standalone emulators and Retroarch cores, and even the Mister core (which I recently tested at a colleague's home, using the Everdrive set), which are all .int/.bin and in the format that int2intv.exe expects on the input side.

 

I know people here have explained that the issue is one of endianess, and I understand what that means, but what is strange to me is that nothing else that I can find uses the INTV2/.intv versions of those roms, so why is the NT Mini so different than all the rest?

 

 

Edited December 29, 2020 by blzmarcel
typo

[CZroe]

 

 

 

What I do find interesting is that the NT Mini seems to be the only platform to need the INTV2/.intv format.

 

I noticed, when I was looking through a recent Everdrive pack that I found, that the included Intellivision roms all matched the checksums of the ones that I've had since somewhere around 1999, which also match the ones from both older MESS and newer MAME sets (within the .zip files), as well as the ones used for standalone emulators and Retroarch cores, and even the Mister core (which I recently tested at a colleague's home, using the Everdrive set), which are all .int/.bin and in the format that int2intv.exe expects on the input side.

 

I know people here have explained that the issue is one of endianess, and I understand what that means, but what is strange to me is that nothing else that I can find uses the INTV2/.intv versions of those roms, so why is the NT Mini so different than all the rest?

From the statement acknowledging the required format, it seems that they needed it in a format that fits or better aligns with the core's native architecture. It's possible that such a format was never considered before because no one was trying to run them on original hardware or a replica of original hardware without physically duplicating the cartridge too. Since the cartridge doesn't factor here they probably decided a new ROM format was a better fit than a secondary core for replicating the cartridge and whatever code is needed to convert/process the more popular ROM formats.

[blzmarcel]

2 hours ago, CZroe said:

it seems that they needed it in a format that fits or better aligns with the core's native architecture.

I'm honestly a little confused. What do you mean by "core's native architecture" ? It is my understanding that with FPGA, the architecture of an original system is being replicated, all the chips, processors, circuits, et al, and thus all of the original behavior of said system.

 

I assum what would happen (and what appears to be the case with other FPGA systems) is that the data from a ROM file is handled the same way an original system would read the ROM data from a cart, as AFAICT, the .int/.bin ROMs that have been around for quite a while are the original binary 1:1 dumps of cart ROM which any other FPGA and emulator expects. But the NT Mini (is this also true of the Super NT and Mega SG?) needs a change of endianness which still strikes me strange that no other FPGA system or emulator requires this.

[CZroe]

I'm honestly a little confused. What do you mean by "core's native architecture" ? It is my understanding that with FPGA, the architecture of an original system is being replicated, all the chips, processors, circuits, et al, and thus all of the original behavior of said system.
 
I assum what would happen (and what appears to be the case with other FPGA systems) is that the data from a ROM file is handled the same way an original system would read the ROM data from a cart, as AFAICT, the .int/.bin ROMs that have been around for quite a while are the original binary 1:1 dumps of cart ROM which any other FPGA and emulator expects. But the NT Mini (is this also true of the Super NT and Mega SG?) needs a change of endianness which still strikes me strange that no other FPGA system or emulator requires this.

Going back to the NES example, it depends on whether the core also replicates the cartridge hardware or not. I don't know what characteristics there are with the way the Intellivision reads it's ROMs but the existing ROM dumps were probably made by literally removing the chips and dumping them which is not necessarily how the system sees them through it's data bus. Replicating the way the original system did it as accurately as possible without adding an extra layer of complexity to convert those raw dumps back into something the core can use may have required a different format. Remember: The old systems don't read them straight off the SD card so the jailbreak has to copy them to memory in a way that replicates the original cartridge. Sometimes that means replicating things like latency, read speed, and blocking RAM writes so that it appears to be running on ROM instead of RAM. Indeed, a lot of copy protections for old backup hardware worked by writing to addresses that should not be writable and then reading back what was written.

[Kaide]

1 hour ago, blzmarcel said:

I'm honestly a little confused. What do you mean by "core's native architecture" ? It is my understanding that with FPGA, the architecture of an original system is being replicated, all the chips, processors, circuits, et al, and thus all of the original behavior of said system.

 

I assum what would happen (and what appears to be the case with other FPGA systems) is that the data from a ROM file is handled the same way an original system would read the ROM data from a cart, as AFAICT, the .int/.bin ROMs that have been around for quite a while are the original binary 1:1 dumps of cart ROM which any other FPGA and emulator expects. But the NT Mini (is this also true of the Super NT and Mega SG?) needs a change of endianness which still strikes me strange that no other FPGA system or emulator requires this.

In the case of intv2, there's a couple things that jump out to me:

 

1) Specifying the binary data as 16-bit little endian words, even for 10-bit ROMs. I think we got a bit hung up on providing what was thought to be a concise example of where there is potentially no such thing as a "1:1 copy" when dumping ROMs, but still interesting in this case. 16-bit words does make the emulation of the cartridge bus a bit easier in some ways. 

2) Embedding what would normally be in a config file for .int/.bin ROMs. 

 

The second one is rather important, though.

 

It looks like outside of .rom files which embed metadata that would normally go into the .cfg file, the other formats require this configuration file to specify certain things about the ROM, including how memory is mapped so it can assign certain chunks of the ROM to specific addresses. A bit like the NES mapper example we provided before. But this data is appended to the end of the file and has to be searched for. This isn't a great engineering design, IMO, but it potentially makes it backwards compatible with emulators that only understand .bin ROMs when paired with a .cfg file.

 

.intv2 on the other hand, uses the format itself to tell where chunks of ROM data goes. So as you read the file, it's telling you where things go, making things much easier. It has performance implications for both dumping and reading of ROM files.

 

EDIT: Another thought that occurs to me is that one way to describe this is that raw ROM dumps aren't terribly useful. Much like a RAW file from a DSLR, it needs to be processed with additional information to make sense of the raw data. Be it information about circuitry that cannot be dumped, memory locations, etc, etc. Some ROM formats are closer to that raw data. MAME, and I guess Intellivision, fall into that category. Other ROM formats bundle in the metadata required. intv2 would fall more into this second category, but so would an NES ROM that contains information about which mapper was used, and size data on the CHR and PRG ROMs.

Edited December 30, 2020 by Kaide

[bikerspade]

This new format simply represents the information that something like MAME or some other emulator reads from a config file, like this: https://github.com/mamedev/mame/blob/master/hash/intv.xml

 

This new format is better, as it doesn’t have dependencies on a config file, new Intellivision homebrew can be created without needing config file updates. Not to mention it’s a PITA to have an FPGA have to parse an xml config file like this.

[blzmarcel]

3 hours ago, bikerspade said:

This new format simply represents the information that something like MAME or some other emulator reads from a config file, like this: https://github.com/mamedev/mame/blob/master/hash/intv.xml

 

This new format is better, as it doesn’t have dependencies on a config file, new Intellivision homebrew can be created without needing config file updates. Not to mention it’s a PITA to have an FPGA have to parse an xml config file like this.

I'm all for adding extra information similar to iNES headers, and agree that it's nicer to have everything together in one file, though that extra information still needs to be read upon loading, so in the end it really shouldn't make much of a different where that needed info is read from (from a header/footer, or accompanying file.)

 

16 hours ago, CZroe said:

I don't know what characteristics there are with the way the Intellivision reads it's ROMs but the existing ROM dumps were probably made by literally removing the chips and dumping them which is not necessarily how the system sees them through it's data bus.

Is it really known which order actually matches the original ROM? Even if it was read directly from the chips, from what I know, that should be the same data that would be read through the edge connector (by an Intellivision or a proper dumping tool.)

 

Aside from the NT Mini (and I assume also other Analogue systems), emulators and FPGAs that that support Intellivision all assume the byte order of the long established ROM set that's been out there for decades. So why was this necessary for just this one system and not all of the others?

 

16 hours ago, Kaide said:

but it potentially makes it backwards compatible with emulators that only understand .bin ROMs when paired with a .cfg file.

I did a test with the int2intv tool, using Frog Bog, and found that it is adding to both the beginning and the end, and not just the end of the file:

 

Frog Bog (1982) (Mattel).int
(Matching: <rom name="frog bog.50" size="0x2000" crc="37222762" sha1="58842e4425ad5a8f6ec65528c4b485039057f289"/> from the MAME intv.xml hash file.)

00 66 00 56 00 1C 00 50 00 91 00 59 00 71 00 50
....
02 90 00 80 02 0C 00 03 00 12 01 C9 02 51 02 B7

Frog Bog (1982) (Mattel).intv
(Using int2intv -m 0, with a -m value of 0 going by the spreadsheet linked from the int2intv github page.)

# Added header section.
00 50 00 00 00 10 00 00

# Same bytes as original, with every pair of bytes swapped (16-bit dword endian flip.)
66 00 56 00 1C 00 50 00 91 00 59 00 71 00 50 00
....
90 02 80 00 0C 02 03 00 12 00 C9 01 51 02 B7 02

# Added footer section.
00 00 00 00 00 00 00 00

 

Due to the added information to the start, in addition to the byte endian flipping, I don't see how INTV2 (.intv) files could be used with all the other emulators out there. If the header wasn't added (but rather just the footer) and the bytes weren't flipped, then yeah, it would probably work in most emulators, but from what I can see, people will need to have two (three when counting MAME, due to the zipping and different file naming) separate sets of files, wasn't granted, isn't a huge deal, as the full Everdrive set for Intellivision, including home brews, prototypes, demos, and all the regular games, is less than 4 MB total.

[Kaide]

1 hour ago, blzmarcel said:

I'm all for adding extra information similar to iNES headers, and agree that it's nicer to have everything together in one file, though that extra information still needs to be read upon loading, so in the end it really shouldn't make much of a different where that needed info is read from (from a header/footer, or accompanying file.)

Footer is worse than either of the other options, since you have to do a search through the file for the footer, then make sense of the raw data sitting in front of it. For a footer to be "efficient" you need to be able to stream the whole file into memory in one go and then read the footer and still be in good shape.

 

The NT Mini's ROM format here allows the memory map to be read/configured as the file is being streamed into memory, which has advantages when you don't have a lot of memory to work with, since you don't have to keep a header in memory as reference when reading the file. So this makes the approach a little more memory efficient on loading than even a header.

 

Early GPS devices used tricks like this (such as writing the R-Tree that sorts POIs into regions into the file format itself) to reduce how much memory was used, and improve speed on searching large POIs from slow flash memory by letting the GPS "skim" the file and keep very little state in RAM. 

 

1 hour ago, blzmarcel said:

I did a test with the int2intv tool, using Frog Bog, and found that it is adding to both the beginning and the end, and not just the end of the file:

I was talking about a specific format proposed for Intellivision (.ROM) that would have appended the metadata in a footer. I thought I clarified in my post that's what i was talking about here.

 

1 hour ago, blzmarcel said:

Is it really known which order actually matches the original ROM? Even if it was read directly from the chips, from what I know, that should be the same data that would be read through the edge connector (by an Intellivision or a proper dumping tool.)

It depends a lot on how the ROM chips were setup on the bus. But I'd argue that "matching the original ROM chips" is not even a good metric, as it gets too much into the weeds. A good archival format for cartridge data is one that is easy to work with, and accurately describes the cartridge enough to reproduce it. 

 

Interleaved ROM chips (i.e. two ROM chips with 8-bit data buses providing high and low bytes to a 16-bit data bus) makes things more complicated than it needs to be. For the sake of emulation or even hardware reproduction these days, it's easier to pre-interleave the data for example. If I wanted to create carts from scratch, it'd likely be easier to get a 16-bit EEPROM anyways. And if I did need to recreate interleaved EEPROMs, it's a simple task to do that from a non-interleaved dump so long as the format is well documented and consistent. 

 

In terms of endianness, like the above example, the format just needs to be well documented and consistent. That's it. 

[blzmarcel]

On 12/23/2020 at 8:43 AM, Kaide said:

Cartridges on these systems aren’t “inert”. For NES, the mapper chips are on the cart, and sit between the ROM chips and the system. Just dumping the ROM itself isn’t going to include the behavior of those chips.

What I'm wondering is if the mappers as similarly important for the Intellivision as they were on the NES, why wasn't there an equivalent of the iNES header prepended to the ROMs early on? This was recognized as a necessity for NES ROMs going at least as far back as ye olde NESticle emulator.

 

I've seen people mention there were cfg files, but packs like the Everdrive and MAME sets don't seem to have any. I tested in both MAME and on my friend's Mister again, both of which ran every homebrew, demo, and prototype from the Everdrive set that was thrown at either without any issue. In the Mister case, we browsed from a computer and couldn't find any cfg or similar files that were generated (since it's a third party core (github/source code) that didn't come with the system.) This makes me wonder if there isn't already something in the ROMs that lets newer emulators and cores know how to treat them, especially if homebrews and such run just fine.

Edited December 30, 2020 by blzmarcel
typo

[bikerspade]

And yet the original iNES header format was sorely lacking and incomplete, as there has been continued development and expansion of NES2.0 headers within the past year or so. Emulator devs had to have internal hacks in their emulators for some roms in order to represent the correct behavior that could not be expressed in iNES.

[blzmarcel]

23 minutes ago, bikerspade said:

And yet the original iNES header format was sorely lacking and incomplete, as there has been continued development and expansion of NES2.0 headers within the past year or so. Emulator devs had to have internal hacks in their emulators for some roms in order to represent the correct behavior that could not be expressed in iNES.

Some good points, though my point is that there was some kind of header early to help emulators do the right thing, but afaict nothing of the sort existed at all for Intellivision ROMs at all until recently.

[Sho]

 

I dropped this video for people who might want some palettes for use on their Noir.  Can also be used on the OG 2017 Mini.

[Kaide]

1 hour ago, blzmarcel said:

Some good points, though my point is that there was some kind of header early to help emulators do the right thing, but afaict nothing of the sort existed at all for Intellivision ROMs at all until recently.

Which would be part of the reason for the intv2 format. 
 

MAME has a similar problem. ROM sets are matched to the version of the emulator, since the configuration information lives in MAME, not the ROM. So certain versions of MAME only work with specific dumps of the ROMs, named a certain way. MAME even goes so far as including the CRCs of working ROMs to catch dumps from older ROM sets quickly so it can reject them instead of trying to load them. For Intellivision emulators that don’t use .cfg files, I would suspect a similar arrangement where the file name of the ROM gets looked up in a table for the appropriate configuration, since the configuration is likely simpler than what MAME has to deal with.