RMT2LZSS: convert RMT tunes to LZSS for fast playback!

[ivop]

39 minutes ago, rensoup said:

Consider that you must be able to export to RMT binary format just so that you can patch the tune into the 6502 memory on top of maintaining a 6502 player ?

RMT binary format is not needed if the player is expanded to play its own binary format. A small 6502 player is still useful, if not all the effects are used. Just a small tune and some sound effects might not be suited for a full blown LZSS decompression player. That is, if one can afford the cycles for a normal player.

 

There's a Dutch saying, it's like lead or old iron, which means it makes no difference. You either have to maintain a C/C++/C# player, or you maintain and use 6502 player (not necessarily limited to the RMT player) with CPU emulation. Bonus of the latter is that it also runs on real hardware if you stay away from the screen, ROM and DOS area.

 

Whatever player is used, after that it can be run through a SAP to LZSS stage.

[rensoup]

2 hours ago, ivop said:

RMT binary format is not needed if the player is expanded to play its own binary format. A small 6502 player is still useful, if not all the effects are used. Just a small tune and some sound effects might not be suited for a full blown LZSS decompression player. That is, if one can afford the cycles for a normal player.

 

?

So if you want to play a song inside your editor, how exactly do you get from you QT derived class data to SAPR ?

[ivop]

1 hour ago, rensoup said:

?

So if you want to play a song inside your editor, how exactly do you get from you QT derived class data to SAPR ?

Either the player plays the data directly from the class data (a C++player, similar to your C# player) and generates SAPR events, or it has an export function that you can use to populate the 6502 memory later used by the 6502 player, and then run a CPU/POKEY emulation cycle and send the audio data to the audio pipeline, or, in the case of the new Altirra backend with its own audio pipeline, do nothing.

 

[rensoup]

21 minutes ago, ivop said:

Either the player plays the data directly from the class data (a C++player, similar to your C# player) and generates SAPR events

Yes that's the sane path. 

21 minutes ago, ivop said:

or it has an export function that you can use to populate the 6502 memory later used by the 6502 player

Yes, that's the not so sane one. 

 

You can call it export function but it is the RMT binary save which would indeed be injected into the 6502 mem and played with the stock 6502 RMT player or another binary save which would be similar to the RMT format but would require another full blown 6502 player which would be a lot more work...

 

 

So at the minimum if you used a 6502 player you'd need an RMT binary save, agreed ?

 

[ivop]

1 hour ago, rensoup said:

Yes that's the sane path. 

But a lot of work.

 

Quote

Yes, that's the not so sane one. 

But a lot less work

 

1 hour ago, rensoup said:

You can call it export function but it is the RMT binary save which would indeed be injected into the 6502 mem and played with the stock 6502 RMT player or another binary save which would be similar to the RMT format but would require another full blown 6502 player which would be a lot more work...

So at the minimum if you used a 6502 player you'd need an RMT binary save, agreed ?

Yes, either the current best player, or a future better player.  Why do you think it would be a lot more work as you say? This code is already there.

Note that this New Atari Music Tracker is still fictional. Spending a lot of time on a new C/C++ player doesn't really help imho. We want a new GUI with as near to real hardware replay during composing as possible.

Edited September 29, 2021 by ivop

[rensoup]

19 hours ago, ivop said:

Yes, either the current best player, or a future better player.  Why do you think it would be a lot more work as you say? This code is already there.

Note that this New Atari Music Tracker is still fictional. Spending a lot of time on a new C/C++ player doesn't really help imho. We want a new GUI with as near to real hardware replay during composing as possible.

I don't know if we're talking about the same thing anymore, I'm not sure why you keep mentioning a new player but don't mind me, I don't want to get in the way of your procrastination investigation ?

 

[ivop]

4 minutes ago, rensoup said:

I don't know if we're talking about the same thing anymore, I'm not sure why you keep mentioning a new player but don't mind me, I don't want to get in the way of your procrastination investigation ?

 

Well, this discussion started when you were asked to open source your C# player. After that you said I was going the not sane path, et cetera...

 

Your code is not yet available and it is not C or C++ but Microsoft .NET C#, so it needs porting to be used by a new cross-platform tracker. If you can deliver an RMT player in C++, that would be great!

The A8 music community @emkay wants a new tracker with proper timing and a proper backend for about 12 years or more.

 

The backend and timing is mostly fixed now. Close enough to Altirra/real hardware. So now we need a new GUI. First start with the RMT format, but then extend. Arpeggio commands on a per note basis for example! (instead of needing several instruments for each arp). Radek's and Vinscool's code is there to support that and extend the format. And after that it could be run through SAP to LZSS if needed.

 

Procrastination? That reminds me of something

 

 

 

[VinsCool]

I feel personally called out there lol

Oh hey, ivop did the music!

Edited September 30, 2021 by VinsCool

[ivop]

I returned to my rmtdump utility I did in 2012: https://github.com/ivop/rmt-sa-libs/tree/main/rmtdump and the description on http://atariki.krap.pl/index.php/RMT_(format_pliku)

 

There are not many possibilities to extend the TRACK format. Note $3C could be abused to add new features, like portamento or arps. I don't see any other option.

 

I never realized that portamento is instrument based. Never used it. That's really backwards And vibrato and arpeggios should be on a note basis, too.

 

I think I'm almost convinced that a 6502 player export should be LZSS only (could be extended for sound effects), and the internal player should be C/C++ (or C# if it is cross-compilable and does not pull in a load of .NET libraries). RMT import of course, but drop the format after that. I assume Rensoup is OK with this idea, so when can we expect your cleaned up player code?

 

Edited October 2, 2021 by ivop

[rensoup]

3 hours ago, ivop said:

I think I'm almost convinced that a 6502 player export should be LZSS only (could be extended for sound effects), and the internal player should be C/C++ (or C# if it is cross-compilable and does not pull in a load of .NET libraries). RMT import of course, but drop the format after that. I assume Rensoup is OK with this idea, so when can we expect your cleaned up player code?

You're asking if I'm agreeing with what I've been saying all along ? ?

 

I did that dxRMT save in RMT2LZSS, so a RMT file can be converted to that, it's basically RMT format in plain text  + the extended instruments stuff that I did, I'm working on the loading part...

 

Once I'm done, it'll be possible to either load a dxRMT file in a text editor, edit it, save it and convert it with RMT2LZSS... it serves no purpose of course but it just tests a possible workflow for a real editor.

 

So that'd be a pretty good candidate for a release I think... there isn't much left to be done but I've been finishing that other project ?

 

[ivop]

19 hours ago, rensoup said:

You're asking if I'm agreeing with what I've been saying all along ? ?

Haha, yes It took me a look at the .rmt file format after all these years, to realize you were right.

19 hours ago, rensoup said:

I did that dxRMT save in RMT2LZSS, so a RMT file can be converted to that, it's basically RMT format in plain text  + the extended instruments stuff that I did, I'm working on the loading part...

Once I'm done, it'll be possible to either load a dxRMT file in a text editor, edit it, save it and convert it with RMT2LZSS... it serves no purpose of course but it just tests a possible workflow for a real editor.

So that'd be a pretty good candidate for a release I think... there isn't much left to be done

OK, I'll wait.

19 hours ago, rensoup said:

but I've been finishing that other project ?

Oh yeah, that one is great! Love the mouse!

[VinsCool]

3 hours ago, ivop said:

Oh yeah, that one is great! Love the mouse!

The music was painful to get working with everything.

Thanks god (rensoup) for restless programming and thanks god for the marvelous things called tables, having been able to literally make a RMT hack for this project exclusively really made it much much easier for my contribution lol

 

[clth]

Thanks for this tool, it is really neat. I would like to ask - is at this point possible explicitly do the RMT output conversion for NTSC? This would be helpful for me to automate making raw SAP-R dumps for sfx effects with minimal player so RMT is used only as offline sound generator.

[rensoup]

1 hour ago, clth said:

Thanks for this tool, it is really neat. I would like to ask - is at this point possible explicitly do the RMT output conversion for NTSC? This would be helpful for me to automate making raw SAP-R dumps for sfx effects with minimal player so RMT is used only as offline sound generator.

Not sure I understand your question... the tool always outputs all the SAPR data, it's up to you to call the player at 50/60fps to get the correct playback speed.

[clth]

22 hours ago, rensoup said:

Not sure I understand your question... the tool always outputs all the SAPR data, it's up to you to call the player at 50/60fps to get the correct playback speed.

My assumption was that while initially having 50 hz SAPR data, it would be sort of better have 60 hz capture too instead of somehow using 50 hz ones while running 60 hz. However I've today discovered that RMT actually is not able to do "authentic" 60 hz mode even though such usage is supported, result is kind of tradeoff done while internals are still more or less 50 hz based. So, you are right that only option is find way playing 50 hz SAPR data while having different framerate what will lead to slightly different and inaccurate output compared to "original".

[rensoup]

1 hour ago, clth said:

My assumption was that while initially having 50 hz SAPR data, it would be sort of better have 60 hz capture too instead of somehow using 50 hz ones while running 60 hz. However I've today discovered that RMT actually is not able to do "authentic" 60 hz mode even though such usage is supported, result is kind of tradeoff done while internals are still more or less 50 hz based. So, you are right that only option is find way playing 50 hz SAPR data while having different framerate what will lead to slightly different and inaccurate output compared to "original".

Hmm.. if you set RMT to 60hz, your SAPR will still contain the right amount of data for 60hz playback.

 

The RMT2LZSS player doesn't care about 50 or 60, whatever you set in RMT, you need to set your machine region accordingly for correct playback speed.

 

Not sure if we're saying the same thing worded differently.

[clth]

45 minutes ago, rensoup said:

The RMT2LZSS player doesn't care about 50 or 60, whatever you set in RMT, you need to set your machine region accordingly for correct playback speed.

Thats probably key information for me, gotta find some other way around. Thanks for making that clear.

[solo/ng]

Thanks for AWESOME work!

Our new production for xl/xe (at Lost Party 2022) going to use it !

 

 

 

 

[tebe]

yeah, another WIP production

[VinsCool]

If you guys are interested, very soon I will release a new proper version of RMT, with full LZSS export functionality, which will be infinitely easier to use than RMT2LZSS in many ways.

It will be recording registers writes directly from it, and tuning lookup tables will also be automatically generated in realtime, so there won't be any confusion anymore about the expected sound vs what it really sounds like after data is exported.

 

Note that, most of the results may be virtually identical to RMT2LZSS, but still, it's always nice to have an alternative ?

[solo/ng]

7 hours ago, tebe said:

yeah, another WIP production

not this time ;- )

[solo/ng]

1 hour ago, VinsCool said:

If you guys are interested, very soon I will release a new proper version of RMT, with full LZSS export functionality, which will be infinitely easier to use than RMT2LZSS in many ways.

It will be recording registers writes directly from it, and tuning lookup tables will also be automatically generated in realtime, so there won't be any confusion anymore about the expected sound vs what it really sounds like after data is exported.

 

Note that, most of the results may be virtually identical to RMT2LZSS, but still, it's always nice to have an alternative ?

 

of course we are interested

[rensoup]

On 6/26/2022 at 10:58 PM, solo/ng said:

Thanks for AWESOME work!

Our new production for xl/xe (at Lost Party 2022) going to use it !

 

 

 

 

Good to hear (hopefully )!

 

Note that if your tune has long repeated songline sequences, you can chop up the tune manually into different subtunes to save some memory, then instead of having 1 part (tune repeating at songline 0) or 2 parts (tune repeating at non 0 songline ), you'd have more parts.

Yes that should be automated but I'll leave it for later.

 

The drawback is that starting a new part causes a new init of the player which is slightly slower (perhaps 1 extra scanline?)

[miker]

@rensoup Maybe I've didn't read this thread carefully, but in what format is expected for "1.28 custom tebles"?

Is it a part of rmt player with those or what?

 

Update: managed to find that file in package, but still haven't dealt with it...

Edited July 4, 2022 by miker
update

[miker]

Never mind, I hacked-in some code.