[RELEASE] RASTER Music Tracker, v1.34.00

[emkay]

8 hours ago, ivop said:

If you enable Audio scope, too, you can see the combined waveform.

LOL. 

Thanks for reminding me to update Altirra

[VinsCool]

Oh yeah, I would love to see a oscilloscope display.

No idea how or when I could make this happen, unfortunately.

[VinsCool]

Tonight I did something really dumb but I think it is going to be very useful later.

So first, I added this somewhere in the RMT code...

void Get_Driver_Version()
{
    const char driver[] = "RMT driver ";
    char version[65] = {0};
    for (int i = 0; i < 64; i++)                //64 characters are more than enough...
    {
        WORD adr = RMT_ATA_DRIVERVERSION + i;
        version[i] = g_atarimem[adr];
        if (!i && !g_atarimem[adr])             //if i is 0 and byte is 0x00...
        {
            sprintf (version, "version unknown. Are you using an older version of tracker.obx?");
            break;
        }
    }
    g_driverversion.Format("%s%s", driver, version);
}

This thing will essentially read some special memory from the tracker.obx binary during initialisation, and display the text in the About dialog.

Seems to work pretty nicely with my idea, as far as I could tell!

Old tracker.obx, without the plaintext data:

New tracker.obx, with the plaintext data, this is also the format I will use from this point onward:

I know, this is very pointless, I did it simply because I can, so, why not? :3

As for the Bilinear Filtering, I reverted the test I did the other day, so scaling with no filter will remain until I find a better way to do it.  

 

Test version:

rmt132test.06-1.zip
- Removed the crappy bilinear filter due to giving mixed results.
- Added the new tracker.obx version display, including an updated .obx for the purpose. Works exactly the same otherwise.

Edited February 4, 2022 by VinsCool
Added a new test version

[emkay]

About the "analyzer" . It's a nice feature in Altirra, but the real helpful stuff would be to have joined channels showing the resulting wave. 

So  people could "see what they create" 

You hear some triangles and sawtooths, but you don't see them ...

 

 

 

 

[+Stephen]

33 minutes ago, emkay said:

About the "analyzer" . It's a nice feature in Altirra, but the real helpful stuff would be to have joined channels showing the resulting wave. 

So  people could "see what they create" 

You hear some triangles and sawtooths, but you don't see them ...

 

 

 

 

On 2/3/2022 at 2:34 AM, emkay said:

On 2/2/2022 at 6:06 PM, ivop said:

If you enable Audio scope, too, you can see the combined waveform.

LOL. 

Thanks for reminding me to update Altirra

Did the combine waveform not work?  Found the option:

[VinsCool]

Combined definitely works, it's pretty much what the actual output looks like from the real thing.
Individual has some nice purpose too, because you see the resulting waveforms from filtering and distortions as the channel itself outputs it.

So a combination of both visualisation is the best option  

[emkay]

16 minutes ago, VinsCool said:

Combined definitely works, it's pretty much what the actual output looks like from the real thing.

???

 

To me this looks like "3 ?" independent analyzers were running.

 

 

 

[rensoup]

On 2/2/2022 at 12:21 AM, VinsCool said:

I have plans to make this happen, but this is far away in my list of things I want to do.

Currently, I don't think have the skills necessary to make certain things possible.

You could just output SAPR data ?

[VinsCool]

1 hour ago, rensoup said:

You could just output SAPR data ?

I imagine that can technically be not too difficult to do, just redirect the register writes to a file every m_play() calls ?...Especially when the sequence of bytes is quite literally a matter of reading the emulated memory, enter a loop and copy the bytes over and over, and then get processed by the POKEY plugin.

Speaking of which I noticed something I overlooked a while ago, it seems like there is a way to actually set a number of cycles per frame. Turns out it was hardcoded to PAL as a 312*114 constant, or something like that. Maybe useful, maybe not. 

I don't really have the time to look at this for the week end.

The tiny experiment with reading data from the driver binary wasn't much but opened my eyes on many more possibilities related to input/output of data. heh I'm not very smart I admit.

[emkay]

I did show this video some time ago. But THIS is doing, what is needed:

 

Check from 1:40 (again)

 

The Video shows on the 2nd channel from above what happens with the waves.

You see everything you hear, directly on the screen. 

The addition of the waves, the resulting "edgy waveforms" the resulting PWM between them.... and so on. 

THIS is physics

 

 

[VinsCool]

12 hours ago, emkay said:

THIS is physics

I have broken down this entire video a while ago, and again, the visualisation is a result of inaccurate emulation.

The rendering fucked up upon recording channels individually, resulting with channel 1 and 2 being mixed together into channel 2.

 

In most videos using ASAP, and making use of the highpass filter, the output is also incorrect, such as this video, for example at 1:30:

The same tune emulated with Altirra, with all 4 channels being proper:

 

A proper way of visualise all effects like this would be to have all 4 channels mixed together, and displayed in the oscilloscope view, maybe as a 5th channel labeled "mix" or something like that.

 

In any case, the individual 4 channels in these old video are mostly wrong, due to using very old ASAP emulation in particular.

 

In case you don't believe me, I have the exact same kind of oscilloscope render, which I *have* recorded from a real PAL 800xl (mine), and rendered the video using each individual channels, so what is shown there is *really* what each single channel actually have output when it is properly done.

Having all 4 channels mixed together, would display exactly what you want to see, the physics of all 4 channels being mixed together

 

Maybe next time I make such a soundtrack, I should make labels and show what all channels mixed together looks like

 

[emkay]

1 minute ago, VinsCool said:

I have broken down this entire video a while ago, and again, the visualisation is a result of inaccurate emulation.

The rendering fucked up upon recording channels individually, resulting with channel 1 and 2 being mixed together into channel 2.

 

This is not "fucked up" . This is what you hear, where the music gets the flanger FX and others... 

This is also strongly recommended to have full control over wave creation with POKEY. 

It's THE visualizer that helps people to understand POKEY better. 

Put the relevant channels together, for synth adjustments and show the result immediately while editing. 

 

You don't get the point, if you only show separated channels, and you don't get it, if you show all channels acting together. 

Believe me. It will surely open their eyes

 

1 minute ago, VinsCool said:

 

Maybe next time I make such a soundtrack, I should make labels and show what all channels mixed together looks like

 

 

 

As a technician, I still see the phase variations and the base wave. 

But, do you think, any musician will see it while editing music?

[VinsCool]

2 hours ago, emkay said:

This is not "fucked up" . This is what you hear, where the music gets the flanger FX and others... 

Yes, this is correct.

However, what was visualised in that channel 2, again, was incorrect due to the incorrect emulation going there.

The principle you described is correct, but the actual things happening were wrong.

I cannot blame anyone for being mislead about it, because I didn't actually know that until I started to study the chip's behaviour really seriously.

2 hours ago, emkay said:

It's THE visualizer that helps people to understand POKEY better. 

Put the relevant channels together, for synth adjustments and show the result immediately while editing. 

Yeah, I agree.

I totally want to see the waveforms in realtime too, but this will need to be coded in the first place, something I am not yet able to provide, unfortunately.

Currently, best I was able to offer was a more detailed registers viewer, with accurate pitch calculations.

That needs work as well, so I can understand this isn't that useful unless the user knows exactly what these infos are for.

2 hours ago, emkay said:

You don't get the point, if you only show separated channels, and you don't get it, if you show all channels acting together. 

Believe me. It will surely open their eyes

Nah, I get the point, don't worry about it.

There is a purpose to have 4 channels separated, and there is also a purpose for displaying all at once, in 1 single oscilloscope wave dancing.

My only point was regarding the fact that a single channel's display must be accurately what it does output alone, without the interference of another one layered on top of it.

I agree with you, if that was not clear, about the usefulness of displaying the full modulation, when all channels interact together.

2 hours ago, emkay said:

As a technician, I still see the phase variations and the base wave. 

But, do you think, any musician will see it while editing music?

They need to hear it, first and foremost.

To see it, is a nice bonus, and of course I also want to see it  

 

 

...

Anyway, turns out whatever I had planned for the week end was cancelled last minute... so I have plenty of free time again to try improving what I have been doing with RMT's code.

So let's see where I can go with this.

Edited February 5, 2022 by VinsCool
typo, more infos

[VinsCool]

Finally addressed something cosmetic but worth being customisable:

The option also becomes grayed out when the "Display rasterbar" option is also unchecked  

The patch was also simple to add on the export binaries:

			if (!dlg.m_meter || !dlg.m_msx_shuffle)	//no rasterbar, or no colour shuffle
			{
				const BYTE shuffle[9] = { 0xF0, 0x07, 0xEA, 0xEA, 0xEA, 0xEA, 0xEA, 0xEA, 0xEA }; //BEQ label nn1, and 7 NOP instructions
				for (i = 0; i < 9; i++) mem[adr_msx_shuffle + i] = shuffle[i]; 
			}

I've also applied the same patch on SAP exports, since the same player binaries are used.

Those don't actually make use of the rasterbar display, so there's no point editing a random memory address for no purpose, right?  

 

Next, based on the same idea, I'll add the options to set the export region manually, so you could decide to export to NTSC, without patching the executable to adjust the speed for PAL, etc

Ideally, a better method to detect region would be useful, but I think having the choice to manually set the values will be a good compromise for now, I know this was asked a couple times already.

[VinsCool]

Alright, another one added:

This one is pretty straightforward: if the option "Automatically adjust playback speed" is checked, the same PAL/NTSC check based on the setting currently used will apply.

If the box is unchecked, the check will simply be NOPed out of existence when the binary is loaded, meaning there won't be any adjustment at all, useful for particular setups that may be detected incorrectly.

			if (dlg.m_region_auto)	//if the automatic adjustment is enabled...
			{
				if (g_ntsc)
				{
					mem[adr_region_XEX] = 0xd0;	//BNE (is_pal)
					mem[adr_region_XEX + 3] = 0xbb;	//(LDX) #187
				}
				else
				{
					mem[adr_region_XEX] = 0xf0;	//BEQ (is_ntsc)
					mem[adr_region_XEX + 3] = 0x82;	//(LDX) #130
				}
			}
			else
				for (i = 0; i < 9; i++) mem[adr_region_XEX - 5 + i] = 0xEA;	//NOP, for 9 bytes

I'm aware there is a different way to detect the region automatically (thanks PPS!), and I will try make use of that in a future 6502 ASM code update.

But for now, that should do the trick  

 

---

Test version:

rmt132test.06-2.zip

- Make the XEX export format have the colour shuffling optional.
- Also make sure the SAP exports skip the colour shuffling instructions, since both export formats use the same player binaries.
- Added an option for XEX exports to enable or disable the automatic region detection, in case a particular setup may work incorrectly with it.
 

Edited February 6, 2022 by VinsCool
typo

[rensoup]

On 2/5/2022 at 3:52 AM, VinsCool said:

I imagine that can technically be not too difficult to do, just redirect the register writes to a file every m_play() calls ?...Especially when the sequence of bytes is quite literally a matter of reading the emulated memory, enter a loop and copy the bytes over and over, and then get processed by the POKEY plugin.

 

hmm.. yeah I don't know how straightforward that would be actually... not sure how things are processed with Altirra pokey.

 

Not sure if you just patch the 6502 RAM with the tune and just run the emu as a black box... meaning the 6502 emu produces Pokey data which gets passed on to the Pokey emu which produces the samples which get passed on to the audio driver without any kind of intervention.

 

Normally you'd just run the 6502 emu continuously (without any kind of sync and no pokey emu) and grab the Pokey data into a buffer until reaching the loop point then you'd save that buffer.

[VinsCool]

22 hours ago, rensoup said:

hmm.. yeah I don't know how straightforward that would be actually... not sure how things are processed with Altirra pokey.

Yeah, I would have to actually study the way Phaeron did implement his emulation plugins.

Right now I speak strictly from how I think I can understand the way RMT is coded for this purpose.

Ideally I would like to get more advanced and actually have the entire thing become cycle accurate without relying on external hacks.

22 hours ago, rensoup said:

Not sure if you just patch the 6502 RAM with the tune and just run the emu as a black box... meaning the 6502 emu produces Pokey data which gets passed on to the Pokey emu which produces the samples which get passed on to the audio driver without any kind of intervention.

As far as my understanding with the process goes, this is pretty much what happens.

 

The emulation for play call first run a 6502_JSR() function, at a specified memory address, where all of the RMT player routines are loaded, and it ends as soon as the final RTS instruction is met.

From there, the next step is to run a memory copy loop from the emulated memory, to the emulated pokey registers.

 

Then, what is done is handled by the POKEY plugin to produce audio samples, and finally RMT reads the data produced into a buffer and outputs it through DirectSound, in a semi constant rate defined by the timing constant I haven't yet fully understood.

22 hours ago, rensoup said:

Normally you'd just run the 6502 emu continuously (without any kind of sync and no pokey emu) and grab the Pokey data into a buffer until reaching the loop point then you'd save that buffer.

Yeah, that makes sense to me.

That's more or less what I imagine this would work.

That may not be too complex to exploit for producing SAPR streams, and then compress it into a LZSS output.

 

I honestly have a lot of things I have not fully understood, so I may be wrong for many parts.

 

As far as my understanding goes, it may not be too hard to produce a SAPR stream based on the observation above.

Run the exact same 6502 emulation without a timing restriction, and save all the buffered data into a SAPR output, instead of timing writes to the POKEY emulator, and that's it?

 

Something worth noting here, is the more I have studied the RMT code, the more I realise the 6502 emulation may actually not be needed at all, if all of the driver is recreated in C++, and executed much of the same way as the 6502_JSR() calls.

 

The JSR themselves are surprisingly straightforward now.

They run at a specified address, optionally with a set of values in the registers A, X and Y, and finishes once a RTS is read.

 

The JSR themselves are about exactly the same as how RMT binaries operate, too.

The only major difference is in the way they are executed, with a few tracker specific subroutines, that again could easily be replicated in C++.

 

My guess for why this was implemented in such a way was probably so it makes it easier for Raster to update the driver without having to spend hours to port the code from one to another, something I can definitely get behind after many weeks spent into trying to update my own patched version of the driver.

The code is quite literally the same in both cases, so that certainly works quite well for that purpose.

 

Something interesting I observed however is how the timing involved to all these actions are not done in a proper timing setup.

They depend on the program timing in miliseconds, and the execution speed is all done from that.

 

20ms works well in PAL, but NTSC timing was all screwed up since it was running too slow, sitting at a constant of 17ms.

I sorta mitigated that by adding a counter which would set a timing groove of 17 16 16 ms, a pretty hacked up method but this at least solved the issue of running too slow... At least temporarily.

That part alone is very much a good example of why the RMT timing is all over the place, and why cycle accuracy is currently not happening unless you use the Altirra Plugins, lol

 

 

So anyway, rambling aside, I believe that capturing the SAPR stream could work without too much issues in the current setup.

Would I be able to make it happen? Maybe.

When? I have no idea

 

if I get a genius spark, or if I have the permission to share the entire source code, that certainly could be done in almost no time.

 

Edited February 7, 2022 by VinsCool
typo

[VinsCool]

Test version:

rmt132test.06-3.zip

- Fixed a bug related to scaling, causing the output to have choppy appearance due to incorrectly having set the boundaries of the display to draw every frame. Now it's a smooth display!

 

I'm actually really stupid sometime, but I fixed this error.

Anyone who tested the scaling from previous versions may have noticed some incoherent parts of the screen flickering or refreshed at weird timing, or generally felt sluggish.

There was a few parameters related to the device context that were still using the dimensions of unscaled elements, so the output was only properly refreshed on the older boundaries, unless something specifically forced a screen refresh.

A dumb 5 minutes fix, thankfully.

 

[VinsCool]

Okay, so now I changed the 6502 ASM code involved in the region detection, and borrowed the technique @pps posted in the page 1 of this thread, with some tiny modifications for my modified player:

...
	ldx #100		; load into index x a 100 frames buffer
wait_init   
	jsr wait_vblank		; wait for vblank => 1 frame
	dex			; decrement index x
	bne wait_init		; repeat until x = 0, total wait time is ~2 seconds
region_init			; 50 Hz or 60 Hz?
	stx vcount		; x = 0, use it here
	ldx #156		; default value for all regions
region_loop	
	lda vcount
	beq check_region	; vcount = 0, go to check_region and compare values
	tay			; backup the value in index y
	bne region_loop 	; repeat
check_region
	cpy #$9B		; compare index y to 155
	IFT REGIONPLAYBACK==0	; if the player region defined for PAL...
	bpl region_done		; negative result means the machine runs at 60hz
	ldx #130		; NTSC is detected, adjust the speed from PAL to NTSC
	ELI REGIONPLAYBACK==1	; else, if the player region defined for NTSC...
	bmi region_done		; positive result means the machine runs at 50hz
	ldx #187		; PAL is detected, adjust the speed from NTSC to PAL
	EIF			; endif
region_done
	stx ppap		; value used for screen synchronisation
...

Gave it some testing, no more false positive from the PAL register, meaning NTSC-50 and PAL-60 will work properly with the expected speed.

Thanks again pps for the suggestion!

I'll update the binaries for the next proper RMT release I will post later.

[VinsCool]

Changes in RMT 1.32.06
-------------------
- Added a new tracker.obx function, where the driver version is written in plaintext, and then displayed in the About dialog. This is purely cosmetic, otherwise, there is no difference.
- Make the XEX export format have the colour shuffling optional.
- Also make sure the SAP exports skip the colour shuffling instructions, since both export formats use the same player binaries.
- Added an option for XEX exports to enable or disable the automatic region detection, in case a particular setup may work incorrectly with it.
- Fixed a bug related to scaling, causing the output to have choppy appearance due to incorrectly having set the boundaries of the display to draw every frame. Now it's a smooth display!
- Improved the PAL/NTSC detection code in the exports player (thanks pps for the sample code I used as a reference!).
- Updated all binaries with the last 6502 ASM changes, the old export binaries will no longer be compatible due to several memory address changes!

rmt132.06.zip

 

Hopefully I did not break everything.

Now the export binaries also make use of the more aggressive code I have been working on, so it isn't optimal currently.

If everything goes like expected, there shouldn't be any obvious difference, so if there's anything abnormal related to exports, it's my fault again.

[VinsCool]

Well somehow I managed to fix yet another thing that bothered me for a while now.

Having known earlier that would have been in the release I posted last night, but oh well, it will be in the next one  

The maths involved with the tracks line position and cursor position just didn't work right, especially with the scaling being applied on top of it, but I finally figured it out.

So now things are *properly* centred on screen, and dynamically will be adjusted based on the scaled dimensions, and the window height, without being off by 1 or several lines with variable levels of precision.

I only had to improvise a very crappy MessageBoxA debug display to figure it out after 20 minutes, thankfully.

 

So yeah, another thing out of the way after weeks of being somehow pretty off the whole time.

[VinsCool]

Instrument Speed up to 16xVBI is now supported.

Yes, 16x.

Seriously, I forgot who even insisted telling me the RMT driver could never achieve speeds above 4, let alone 8, but I can clearly demonstrate 16x is perfectly doable.

And that was in fucking NTSC mind you, PAL blows this with better performance and still cycles to spare.

More interesting is the fact this could be exploited in greater depth to allow finer BPM control through the entire driver tweak.

And those things could definitely be done in real time with the right amount of tweaks in the code, so there goes more crazy ideas for another day!

16xVBI NTSC test.obx

 

[emkay]

It's really relaxing, if one puts some changes to an instrument, and it is still usable for a broad range of notes. 

Vins, where have you been in the last 40 years ?

 

And, well, it's some idea I had today, listening to the "beep" threads

 

 

 

[VinsCool]

4 hours ago, emkay said:

It's really relaxing, if one puts some changes to an instrument, and it is still usable for a broad range of notes. 

I've commented on Youtube already, but holy shit, this sounds REALLY good!

I'm so happy to see the efforts are paying off, especially since I'm busy right now making sense of all the 6502 code, in order to improve it even more later.

4 hours ago, emkay said:

Vins, where have you been in the last 40 years ?

Well... For the first 14 years before... I was living a parallel universe, until I crossed a wormhole in 1995...

Then, for the next 20 years, I grew up but had not yet touched anything musical until the age of... 23?

As for programming, this did not happen until... last summer during the month of July I believe? ?

 

Clearly, I missed a lot, and I have a lot of the Wacky Race to catch up.

Could you imagine, me running bare feet in the slushy snow, wearing only the coureur des bois attire to the strict minimum (including the beaver hat, obviously),.

I've since been trying to catch up with the train of all the older folks in hope I could eventually prove a point by sinking a lot (too much) of my free time into this kind of wacky projects ?

[rensoup]

On 2/7/2022 at 10:03 PM, VinsCool said:

As far as my understanding goes, it may not be too hard to produce a SAPR stream based on the observation above.

Run the exact same 6502 emulation without a timing restriction, and save all the buffered data into a SAPR output, instead of timing writes to the POKEY emulator, and that's it?

Pretty much, broadly speaking...

 

On 2/7/2022 at 10:03 PM, VinsCool said:

Something worth noting here, is the more I have studied the RMT code, the more I realise the 6502 emulation may actually not be needed at all, if all of the driver is recreated in C++, and executed much of the same way as the 6502_JSR() calls.

No shit, that's what RMT2LZSS does and what I've argued for for about a year but encountered great resistance from the dutchs and the canadians ?

 

Once you go that road you can't output regular xex players anymore and must use LZSS though.

 

On 2/10/2022 at 1:53 PM, VinsCool said:

Instrument Speed up to 16xVBI is now supported.

Yes, 16x.

? That doesn't look stable at all... You don't seem to use effects... if you look at emkay's latest vid, it wobbles a lot more depending on what he's using.

RMT can do 8x but it potentially leaves no CPU for anything else.