VGM Compression Tool

[Tursi]

23 hours ago, TheMole said:

Oh yeah, the tools are awesome for sure. I am a bit hampered on my side because I haven't gone through the effort of porting the testplayer to SDL or something similar to allow it to run on my Mac, but that's entirely my problem. It does make it harder to 'debug' the process, but by and large everything always works first time for me, so I don't often have the need either. This was actually the first time that I got a result markedly different from what I was expecting, and it coincided with the first time I tried to use Deflemask instead of just ripping the music from another game...

 

Ahhhh... right. I should do more with SDL, I just have this innate dislike of massive cross platform libraries. Maybe one of the simpler libraries can do audio, I'll take a peek sometime. The stream itself is simple enough so it doesn't need much, but I just ripped the audio code from Classic99 out of laziness.

 

 

[TheMole]

On 7/24/2023 at 3:21 AM, Asmusr said:

@TheMole I'm I right in assuming you're working on converting the tune from GhostBlaster ripped using Meka?

Close, but not really :).

 

I decided to add new renditions instead of ripping from existing games, so my girlfriend has re-created the main themesong in Deflemask, which I've now succesfully converted to Tursi's soundblock format.

[Asmusr]

On 7/25/2023 at 9:41 AM, TheMole said:

Close, but not really :).

 

I decided to add new renditions instead of ripping from existing games, so my girlfriend has re-created the main themesong in Deflemask, which I've now succesfully converted to Tursi's soundblock format.

Your version of the tune in the last demo is great and very cheerful. It need a little bit of percussion, however, which it sounds like it should be possible to add in between the other notes. Especially in the beginning, the crescendo of the hi-hat is important.

[TheMole]

Hi, me again, with yet another question...

 

 

How does the TI player determine which channels to use for sfx playback? I have created some effects that only use the first channel (in deflemask), but when I play them back on the TI while the main song is playing, the bass notes (periodic noise-based) drop out, indicating the sfx defaults to channel three perhaps? Not sure, still playing around with the different command line tools and parameters to see if there's anything I've done wrong, but at the very least the PSG seems to indicate it is correctly converted to use the first channel:
 

0x000D6,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
0x000D4,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
0x000D4,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
0x000D7,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
0x000DB,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
0x000DA,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
0x000D5,0x5,0x00001,0xF,0x00001,0xF,0x00005,0xF
...

 

 

Edited October 28, 2023 by TheMole

[Tursi]

By whether there is any data present. You are creating a sound effect with 4 channels there - even though three are silent. It doesn't try to be clever, it does what you tell it to do. Encode the sound effect with the unused channels unspecified, like so:

 

prepare4sn chan1.60hz - - - output.snpsg

 

The 'dash' indicates an unused channel.

 

Treat the SFX player with a healthy dose of skepticism, I'm still not 100% sure it plays and recovers the music correctly (though I'm more confident of the TI player than the Coleco one )

 

[TheMole]

7 hours ago, Tursi said:

Encode the sound effect with the unused channels unspecified, like so:

 

prepare4sn chan1.60hz - - - output.snpsg

 

The 'dash' indicates an unused channel.

 

This is what I'm using and what produces the outcome I pasted above:

./prepare4sn beam.vgm_ton00.60hz - - - beam.vgm.psg

 

However, I have a theory... this is what my vgmcomp2 command looks like:

./vgmcomp2 -sn -forcechan 2 themesong.vgm.psg blip.vgm.psg vacuum.vgm.psg beam.vgm.psg trap.vgm.psg boom.vgm.psg themesong.dat

 

Note the -forcechan 2 parameter. This is needed to ensure the noise channel uses channel 2's frequencies for the notes in the main theme song (otherwise channel 2 is discarded, since all notes are muted). I'm assuming that vgmcomp uses that parameter on all input files, in which case playing back the effect will ofcourse override the main theme's channel 2 frequencies.

 

I'm assuming I can just create two separate sbf files, one for the main song and a separate one for the sfx and that should solve my issue. I'm going to go ahead and try that right now. I'll report back.

 

[TheMole]

22 minutes ago, TheMole said:

I'll report back.

Yup, that did the trick!

[Tursi]

Ahhh, interesting, yeah. forcechan would be applied to all files. For most files joining them into one sbf doesn't save all that much data, so you didn't lose much splitting them.

 

[retrodroid]

Okay, I'm attempting to get this working in my assembly game project.

 

I'm trying to understand the memory reqs:

Quote

The amount of memory needed depends on the player you choose to use, from 124 bytes of data and 608 bytes of code for the 60hz music only version, to 254 bytes of RAM and 1004 bytes of code for the 30hz music plus sound effects version. All players also require a workspace, which can be used for scratch data inbetween calls. (I store it in the scratchpad).

Quote

tiSfxSN.asm - SFX and Music playback, 60hz version.

 

In ALL cases, SongLoop (and other xxxLoop functions) must be called with workspace >8300, and ALL registers should be assumed trashed.

So when I call either bl @StartSong or bl @StartSfx I see almost the entirety of my workspace memory getting thrashed. I guess this is expected? If I wanted to use this with my project (that doesn't use 32KB memory expansion), I would have to move my workspace data into VRAM to make the necessary space for the sound routines?  I was thinking/hoping it would use VRAM I guess.

 

Edited March 7 by retrodroid

[Tursi]

14 hours ago, retrodroid said:

Okay, I'm attempting to get this working in my assembly game project.

 

I'm trying to understand the memory reqs:

So when I call either bl @StartSong or bl @StartSfx I see almost the entirety of my workspace memory getting thrashed. I guess this is expected? If I wanted to use this with my project (that doesn't use 32KB memory expansion), I would have to move my workspace data into VRAM to make the necessary space for the sound routines?  I was thinking/hoping it would use VRAM I guess.

 

Yes, the entirety of your workspace is trashed. Although, less from startsong or startsfx than from the actual play function, which uses nearly all registers. The documentation on register usage is right at the top of the player file (CPlayerCommonHandEdit.asm):

 

* register usage for interface functions (StartSong and StopSong)
* R0 =                      R8 = 
* R1 = arg1,scratch,return  R9 = 
* R2 = arg2,scratch         R10= stack pointer (if used from C, not touched)
* R3 = scratch              R11= return address
* R4 = scratch              R12= 
* R5 =                      R13= 
* R6 =                      R14= 
* R7 =                      R15= 

* register usage for SongLoop and getCompressedByte
* R0 = SongLoop scratch     R8 = address of sound chip
* R1 = scratch,return       R9 = SongLoop scratch 
* R2 = scratch              R10= stack pointer (if used from C, not touched)
* R3 = scratch              R11= return address
* R4 = scratch              R12= SongLoop scratch
* R5 = scratch              R13= address of getCompressedByte
* R6 = contains >0100       R14= SongLoop scratch
* R7 = SongLoop scratch     R15= getCompressedByte arg1, SongLoop scratch

 

If you're coding in assembly as it sounds, you are probably better off just using a different workspace for your code than >8300 (like >8320). 

 

As for VRAM, you can't put registers in VRAM. The remaining player memory usage is roughly 166 bytes for both music and sound effects - most software can afford that. VRAM would slow things down quite a bit as the memory access pattern is random.

 

[Tursi]

Just noticed the bit about not using 32k. It's not designed for that, no. You would have to rewrite the code to move storage into VRAM. I started it once with the older one but I got the memory usage on this one low enough that I didn't want to. It's aimed at performance more than memory, as decompressing the streams takes a lot of cycles.

 

That, or live with just 26 bytes for your own software - likely not acceptable.

 

However, you could cut that RAM usage in half by not using a separate SFX player. You don't need the complexity of decompression for most sound effects anyway, so you could write a much simpler one. You can either mute the music channels it uses as the current SFX player does, dedicate a channel to SFX and the rest to music so they never conflict, or just play your SFX immediately after calling the music player - if you always retrigger your sample it will /probably/ be fast enough to sound okay.

 

You could reduce needing two workspaces if your own code can get away with less, too. If you can live with just 8 registers, for instance, you could put your workspace at >8310 and only use registers R8-R15. In addition, none of the register data used by the player is persistent, meaning you are free to use all of it for scratch data between calls. That arrangement would let you use R0-R7 as temporaries and R8-R15 as permanents (excluding R11 for return, which is why I used the top half). You could probably think of other arrangements.

 

If you can afford the CPU time, you could also dedicate a small block of VDP RAM for your registers - backup to VDP before calling the player and restore on return. Even all 32 bytes will not take very long to save/restore, letting you share the workspace and keep all your data.

 

[retrodroid]

37 minutes ago, Tursi said:

Just noticed the bit about not using 32k. It's not designed for that, no. You would have to rewrite the code to move storage into VRAM. I started it once with the older one but I got the memory usage on this one low enough that I didn't want to. It's aimed at performance more than memory, as decompressing the streams takes a lot of cycles.

 

That, or live with just 26 bytes for your own software - likely not acceptable.

 

However, you could cut that RAM usage in half by not using a separate SFX player. You don't need the complexity of decompression for most sound effects anyway, so you could write a much simpler one. You can either mute the music channels it uses as the current SFX player does, dedicate a channel to SFX and the rest to music so they never conflict, or just play your SFX immediately after calling the music player - if you always retrigger your sample it will /probably/ be fast enough to sound okay.

You could reduce needing two workspaces if your own code can get away with less, too. If you can live with just 8 registers, for instance, you could put your workspace at >8310 and only use registers R8-R15. In addition, none of the register data used by the player is persistent, meaning you are free to use all of it for scratch data between calls. That arrangement would let you use R0-R7 as temporaries and R8-R15 as permanents (excluding R11 for return, which is why I used the top half). You could probably think of other arrangements.

.

 

Thanks, this is very helpful and confirms my thinking re: ramifications and possible solutions. Using decompression is probably massive overkill for my simple sound requirements but I am struggling a bit on how to author the sounds I require and was thinking sampling would might be a faster/simpler way to do so.

 

I think I'll go back to the drawing board and look at implementing a simple player or maybe even hand-coded sound routines for the effects I need, though I guess I could at least quickly test swapping out the required number of workspace bytes to/from VDP RAM each frame to see if I can afford the cycles. On that note, I'm still a bit confused as to exactly how many bytes out of the 256 workspace are required for the player:

57 minutes ago, Tursi said:

The remaining player memory usage is roughly 166 bytes for both music and sound effects

Does this mean that the player requires the 32 bytes at >8300 for the registers, plus an additional 166 bytes of workspace memory? Where in the workspace are those bytes accessed?  So in total I'd need to swap out/in 198 bytes of the 256 workspace memory, correct?

 

Thanks for your help on this.

[Asmusr]

15 hours ago, retrodroid said:

Okay, I'm attempting to get this working in my assembly game project.

Have you created your own music in DefleMask or Mod2PSG2, or are you planning to use VGM music/sfx from other sources? If not then maybe a simpler sound player is a better choice. Even the console built-in sound list player might fulfil your needs.

[Tursi]

1 hour ago, retrodroid said:

Thanks, this is very helpful and confirms my thinking re: ramifications and possible solutions. Using decompression is probably massive overkill for my simple sound requirements but I am struggling a bit on how to author the sounds I require and was thinking sampling would might be a faster/simpler way to do so.

 

I think I'll go back to the drawing board and look at implementing a simple player or maybe even hand-coded sound routines for the effects I need, though I guess I could at least quickly test swapping out the required number of workspace bytes to/from VDP RAM each frame to see if I can afford the cycles. On that note, I'm still a bit confused as to exactly how many bytes out of the 256 workspace are required for the player:

Does this mean that the player requires the 32 bytes at >8300 for the registers, plus an additional 166 bytes of workspace memory? Where in the workspace are those bytes accessed?  So in total I'd need to swap out/in 198 bytes of the 256 workspace memory, correct?

 

Yeah, even in Super Space Acer, my own title, I use vgmcomp2 for the music but I rolled a really basic player for the sound effects, just a duration and bytes (similar to the console player). Mind you, I use separate sound chips so I don't have to worry about conflicts.

 

So for memory usage, the 166 bytes are used to track the state of the current song and the current sound effect. (If you only use the music player, then it is 88 bytes). This is located in RAM where-ever you put it, I'd have to see your build process. They are defined at the bottom of CPLayerTIHandlers.asm and are in a section named songDatVars for the sake of GCC. You're probably using one of the combined versions of source? Look for "def strDat" to find the first one.

 

The workspace is separate from this, yes, and it's the registers used for the code running the decompression.

 

If you need to swap out the music space, then yeah, but swapping 200 bytes is a lot more expensive than swapping 32. Depending on your program it may still be quick enough.

 

[retrodroid]

5 hours ago, Asmusr said:

Have you created your own music in DefleMask or Mod2PSG2, or are you planning to use VGM music/sfx from other sources? If not then maybe a simpler sound player is a better choice. Even the console built-in sound list player might fulfil your needs.

So far all I have done is created some snazzy sound effects using https://harmlesslion.com/sn_sfxr/.  So I have some cool VGM files. Is there a way to translate those into something I can play using a simpler sound player?  I also just got Mod2PSG2 running on Wine but it's pretty indecipherable to me. Haven't spent much time on it yet.

 

My needs are a few sound effects, a couple of which are dynamic in nature (sweeps, etc.),  and a couple of very short tunes, so not a lot. Could probably just hand-code something for them.

 

Edited March 7 by retrodroid

[Asmusr]

17 hours ago, retrodroid said:

So far all I have done is created some snazzy sound effects using https://harmlesslion.com/sn_sfxr/.  So I have some cool VGM files. Is there a way to translate those into something I can play using a simpler sound player?  I also just got Mod2PSG2 running on Wine but it's pretty indecipherable to me. Haven't spent much time on it yet.

 

My needs are a few sound effects, a couple of which are dynamic in nature (sweeps, etc.),  and a couple of very short tunes, so not a lot. Could probably just hand-code something for them.

 

Nice tool, I haven't seen that before. But I don't know the answer to your question. If you just want to produce some sound effects for the built-in sound list player you could take a look at the Sound List Ripper.

[Tursi]

Once you parse the VGMs with vgm_psg2psg, you have the simplest possible format - a text file with one row per frame, tone and volume command every frame, broken out by channel.

 

Since my toolchain is purpose built around the compressor, I don't offer much for simpler formats, but the intent was this was simple enough for anyone to write a parser for. That said, there is a psg2soundlist tool that will convert those text files into a console-rom-compatible soundlist.

 

[retrodroid]

Okay, I've found your post related to converting and implementing a simple sound list player here:  

Will do some exploring around that idea.  

 

 

[Tursi]

Documentation is really meant to be the included PDF, but that works too

 

Obviously you don't need the player listed there, since if you're using that format you can use the console player. You'll still need separate code if you want to play sound effects on top of the music though.

 

[retrodroid]

11 minutes ago, Tursi said:

Documentation is really meant to be the included PDF, but that works too

 

Obviously you don't need the player listed there, since if you're using that format you can use the console player. You'll still need separate code if you want to play sound effects on top of the music though.

 

Yes I have been looking through the PDF doc, it's certainly very decent and appreciated. However, I did find the specific steps outlined in that old post regarding converting from a .vgm to a TI ASM format soundlist format very helpful. At this point I'm experimenting with a simple soundlist player approach, but using all your vgm creation and conversion tools to get my sounds there. 😎

Edited March 8 by retrodroid

[Tursi]

Cool, good luck!