Tinkering with MSU-1 FMV

[TrekkiesUnite118]

So lately I've been tinkering with MSU-1 FMV to see how it works and get an idea of how accurate it is to actually reflecting what an actual SNES CD add-on could do as far as FMV is concerned. As far as I can tell what MSU-1 FMV is typically doing is taking a video and converting it to 256 color frames. Those frames are then converted into 8bpp SNES tiles and palettes which are then concatenated into a large MSU file with a header that says what the frame rate of the video is. During playback then each frames tiles are copied from the cart to VRAM at whatever the specified frame rate of the video is in the MSU file (typically 30fps). Audio is handled by just streaming raw uncompressed CD Quality PCM audio.

 

To test this out I tried encoding the following video to see how the results turned out:

This resulted in the following MSU-1 FMV Rom:

https://www.mediafire.com/file/z3582k33je7t6ml/miku-MSU.zip/file

 

The resulting video is 224x144 resolution at 30fps. The image came out really nice and clean looking, and why shouldn't it? It's entirely uncompressed. It's color reduced sure, but it's still uncompressed, hence how sharp it still looks. While this looks really nice, there is an issue here when people try to compare this to say the Sega CD or PC-Engine CD and say it's an accurate representation of a theoretical SNES CD's capabilities. The major issue here is CD Data transfer rates. While these FMVs look nice, they'd never be doable on any CD-ROM drives of the time period.

 

A 1x Speed CD-ROM drive (Speed used in the Sega CD and PC-Engine CD) can transfer about 150KB/s off the disc. A 2x Speed drive (3DO, Saturn, PS1) can transfer about 300KB/s. The data rate you would need to pull off these 8bpp MSU-1 FMVs at 224x144 resolution at 30fps would be about 945KB/s. And that's just for the video. The Audio is again uncompressed PCM Audio at CD quality. So that would require about another 172KB/s. So that would put us about about 1120KB/s needed to pull this kind of quality off. So that would need about an 8x Speed Disc drive, which is also approaching the data rates of a 1x DVD Drive. To put this in perspective, the Dreamcasts GD-ROM Drive when reading the low density CD-ROM portion of the disc is about on par with a 4x Speed drive, the high density GD-ROM portion is about on par with a 12x drive. So even Dreamcast homebrew when running off a MIL-CD wouldn't be able to do this.

 

So in order to get this to work on an actual CD-ROM Drive of the time period, there needs to be something done to get the data rate down. Without some form of compression, the only real options would be lowering the audio rate and the frame rate. For the Sega CD FMV they generally would allocate about 112KB/s for the video stream and use 8-bit 16000Hz Mono Audio. So using that as a guide I did some mockups for what this would look like if the audio quality and frame rate were reduced to fit into these 1x and 2x Drive limits.

 

For a 1x Drive this would be about 4fps:

https://www.mediafire.com/file/yr9mvfma4xew1cd/mikucd1x.zip/file

 

For a 2x Drive this would be about 8fps:

https://www.mediafire.com/file/bvxpm20d9de9hzg/mikucd2x.zip/file

 

Another option would be lowering the colordepth to 4bpp which would cut the data size in half, thus allowing for about 8fps for a 1x drive, and 16fps for a 2x drive. But these files all still end up pretty massive. The 30fps one for example takes up about 250MB. That's almost half a disc for a 3 and a half minute video. So that leads us to the next option, compression. To even get close to pulling off 8bpp FMV at 30fps you're going to need some form of compression to get the data down to 1x or 2x CD data rates. But it's going to need to be a form of compression lightweight enough that a CPU like that of the SNES CPU or some theoretical co-processor would be able to decompress fast enough to display it at 30fps. This would be stuff like Cinepak. Modern Codecs as well as older DCT based codecs would be out of the question. DCT Codecs like MJPEG and MPEG-1 required hardware acceleration back in the day. And modern stuff like say H.264 wouldn't really work either. To give context, the N64 Scene did get it to decode H.264, but they had to disable a lot of the more advanced profiles and the bitrate is capped at around 400-800kbps. And that's with a 90MHz CPU.

 

While Cinepak can give decent results, even it would struggle with a video this busy at 30fps as you can see in these examples I did for the Saturn. All three of them are right on the limit of the bitrate cap at about 295KB/s:

https://www.mediafire.com/file/w1vwtz6buvk3c1y/Miku30.zip/file

https://www.mediafire.com/file/2fj8e0ysa7hyj6l/miku30Dark.zip/file

https://www.mediafire.com/file/upd3clviu6j9dmy/miku20.zip/file

 

As you can see with these samples, the 30fps framerate is giving the Cinepak encoder a hard time when capped at 2x drive speeds. While it doesn't look terrible, the busy background is regularly going to lego sized pixels. And that's with trying to adjust the contrast and brightness for a dim look or a washed look to make it less noticeable. The only real way to get something decent here is to reduce the framerate again. Around 20fps seemed to be the sweet spot that gave a cleaner image with a decent level of smoothness. PS1 doesn't fair much better either at 30fps, even with it's hardware accelerated DCT based compression. Here's an example I did using the official toolset to make an MDEC STR file. You can play it in something like jpsxdec.

https://www.mediafire.com/file/xwjrsqv48sai85j/miku.str/file

 

While a tad nicer than Cinepak, when things get busy the edges become JPEG city and the background becomes macroblock soup. The reason for this is because the PS1 doesn't have interframe compression, all frames are key frames, which eats up bitrate and file size. What finally allows us to have a clean video on a 2x speed drive, is MPEG-1:

https://www.mediafire.com/file/pw5w6w63sd2kkkt/miku.mpg/file

 

While this will play on a Saturn with a Video CD card add-on, it completely violates the Video CD Spec by doubling the bitrate of the video to 2000kbps. The Saturn's Video CD card was designed to take advantage of the 2x Drive so it can play this without a problem. But it was also over $200 when it came out in 1995 on top of the cost of the Saturn itself.

 

So the general conclusion here is that while MSU-1 FMV is a neat thing to see on the SNES, it is NOT an accurate reflection of what FMV on an SNES CD would be like as it completely ignores the data rate of a disc drive. It's pretty much in the same boat as Mega Color on the Genesis. It's neat and the hardware can technically do it with a massive ROM cart mostly unassisted (both use expansion audio as things currently stand), but it's not really an accurate reflection of a CD Add-on of the time.  When it comes to FMV on CD based systems, the data transfer rate of the drive is a much harsher limit than something like colordepth. In those Saturn and PS1 examples, the colordepth is 24-bit RGB, but they still don't look as clean as the 256 color MSU-1 examples as they need to be heavily compressed to work on a 2x drive. Comparing MSU-1 FMV to something like the Sega CD or PC-Engine CD is even more ridiculous as they have an even harsher limit of a 1x drive. A real SNES CD at best would most likely be playing FMV at 4bpp with frame rates around 10-15fps with heavy amounts of audio and video compression. At that point I doubt it would be that much of a step up over the Sega CD and PC-Engine.

 

Edited November 21 by TrekkiesUnite118

[WavyGravy]

That was... Thorough!

It's good to see the actual math behind what we were saying in that old thread, and goes to the point that the MSU-1 and SNES CD would have been very different beasts. When freed from the constraints of bandwidth, storage, and processing requirements, yes, the SNES puts out nice FMV. 

I do still wonder if the SA-1 was originally intended to be the coprocessor for this thing. A fast 65816 strapped to the bitmap conversion acceleration seems like it would be the ideal setup for something like this at the time. In your experience, do you think that would yield acceptable results? I can't imagine CinePak running very well on the base S-CPU.

[TrekkiesUnite118]

I'd imagine there would need to be some kind of coprocessor to help with video decompression, plus additional memory for decompression buffers. Whether or not the SNES CPU could handle it I honestly don't know, that would be a question for someone who knows 6502 based CPUs. Maybe with something like SA1 or a DSP it could be doable though. But if Cinepak it would need to be heavily modified.

 

On Sega CD Cinepak was modified to apply the compression to 4bpp tile data instead of 24bpp bitmap data, and both the main 68k in the Genesis and the sub 68k in the Sega CD are in use while the FMV is playing.

[SlidellMan]

I have read everything that you have read, and have to say that I'm quite impressed. However, The Neo Geo CD originally used a 1x CD-ROM drive while the CDZ used a 2x drive if I remember correctly.

[TrekkiesUnite118]

4 minutes ago, SlidellMan said:

I have read everything that you have read, and have to say that I'm quite impressed. However, The Neo Geo CD originally used a 1x CD-ROM drive while the CDZ used a 2x drive if I remember correctly.

Sure. But FMV would need to be designed to work on the original 1x speed hardware otherwise it would break compatibility. Even at 2x speeds you wouldn't be able to do what MSU-1 does as seen by the Saturn and PS1 examples.

[+Biff Burgertime]

I've never loaded a MSU-1 ROM into an emulator before.

1. I see there's the ROM itself, then what I assume are the video & the audio?
2. Regardless, where do I go from here? If I simply load miku.sfc into an emulator compatible with MSU-1 ROMs, is that all it takes?
3. What emulator(s) are recommended? I assume BSNES given that the author of the emulator created it. Will the main BSNES work, or does it require a particular fork? (like how BSNES-HD is needed to run Mode7 games in widescreen)

I would've just tried a few things to see what worked, but I'm away from my devices for the time being. Thanks in advance, and an even bigger thank you to @TrekkiesUnite118for not only messing around with this stuff, but also documenting your experiences and providing a file of the result!

 

As I say: Few do, fewer document+share, fewest document+share in a way that's interesting/entertaining. So again, big thanks.

[TrekkiesUnite118]

3 hours ago, Biff Burgertime said:

I've never loaded a MSU-1 ROM into an emulator before.

1. I see there's the ROM itself, then what I assume are the video & the audio?
2. Regardless, here do I go from here? If I simply load miku.sfc into an emulator compatible with MSU-1 ROMs, is that all it takes
3. What emulator(s) are recommended? I assume BSNES given that the author of the emulator created it. Will the main BSNES work, or does it require a particular fork? (like how BSNES-HD is needed to run Mode7 games in widescreen)

I would've just tried a few things to see what worked, but I'm away from my devices for the time being. Thanks in advance, and an even bigger thank you to @TrekkiesUnite118for not only messing around with this stuff, but also documenting your experiences and providing a file of the result!

 

As I say: Few do, fewer document+share, fewest document+share in a way that's interesting/entertaining. So again, big thanks.

Simply load the SNES ROM, the .sfc file. That will boot in things like Mesen, bSNES, etc. It will even boot on an actual SNES with an FX Pak Pro.