RMT Hacking Ideas and Progress

[emkay]

Just now, _The Doctor__ said:

I want this please!

geneve.xex

[ivop]

3 hours ago, VinsCool said:

OH, my bad! I misunderstood the instruction.
let me fix this right away.

RMT STRIPPED OFF BINARIES.exe 398 kB · 5 downloads

This is not an RMT 1.28 binary:

 

331776 May 10  2009 Rmt.exe

 

vs

 

407552 Sep  2 23:58 'RMT STRIPPED OFF BINARIES.exe'

 

I also got this dependency error:

 

0009:err:module:import_dll Library mfc100.dll

 

which was not needed for RMT 1.28.

 

Can you post a patched RMT 1.28 binary?

 

I have proof of concept code injection code for 0x3182-0x3fff in place, and perhaps later a true XEX loader with code all over the place

 

 

Edit: OK, I can find the same 6502 binaries in the original RMT 1.28 file, and can probably patch the tags in there, but what is your 407552 bytes long .EXE file?

Edited September 2, 2021 by ivop

[VinsCool]

8 minutes ago, ivop said:

This is not an RMT 1.28 binary:

 

331776 May 10  2009 Rmt.exe

 

vs

 

407552 Sep  2 23:58 'RMT STRIPPED OFF BINARIES.exe'

 

I also got this dependency error:

 

0009:err:module:import_dll Library mfc100.dll

 

which was not needed for RMT 1.28.

 

Can you post a patched RMT 1.28 binary?

 

I have proof of concept code injection code for 0x3182-0x3fff in place, and perhaps later a true XEX loader with code all over the place

Arrrgh ? I did not realise that would have been a problem since 1.30 was pretty much the same internally!

Okay! That's pretty promising!
I'll get you that 1.28 binary stripped (I PROMISE I did not know and did not do the wrong thing on purpose there!)

I'll be right back in a few minutes, hopefully that's not gonna be a wrong one again LOL

[VinsCool]

Okay, take 3, hopefully this is the good one!
I can say for sure the binaries header and size were identical, at least  

Let's see how this goes now  

RMT 128 STRIPPED OFF BINARIES.exe

 

I'm actually surprised there.
I just tried running the .exe, it pretty much just worked, except there was no output nor response at all.
Just 00's. I expected a crash or an error, but nothing, that's pretty funny! haha

Edited September 2, 2021 by VinsCool

[VinsCool]

33 minutes ago, ivop said:

Edit: OK, I can find the same 6502 binaries in the original RMT 1.28 file, and can probably patch the tags in there, but what is your 407552 bytes long .EXE file?

it's RMT 1.30, the unofficial version that was used as a base for my own hack, since the programs has several cosmetic improvements such as resizeable window.
No idea what the other extra bytes are for.

[ivop]

59 minutes ago, VinsCool said:

I'm actually surprised there.
I just tried running the .exe, it pretty much just worked, except there was no output nor response at all.
Just 00's. I expected a crash or an error, but nothing, that's pretty funny! haha

Yes, that's nice. The CPU emulation just quits when it encounters 0x00 (BRK). It helps with my experiments.

 

Question: when is the MONO or STEREO player used? Whatever I do (load/play mono/stereo tracks), it always triggers the TRACKER player.

[VinsCool]

29 minutes ago, ivop said:

Yes, that's nice. The CPU emulation just quits when it encounters 0x00 (BRK). It helps with my experiments.

That makes sense! No wonder my experiments just never seemed to break things in really bad ways-- it either worked perfectly each time, or not at all (output garbage etc).

30 minutes ago, ivop said:

Question: when is the MONO or STEREO player used? Whatever I do (load/play mono/stereo tracks), it always triggers the TRACKER player.

Correct! This is specifically the reason I labelled it this way, it's only used in the tracker itself.
MONO and STEREO binaries are for exports exclusively, slapped to either the SAP or XEX format, but other than that, it's almost exactly the same code everywhere.

[ivop]

1 hour ago, VinsCool said:

Correct! This is specifically the reason I labelled it this way, it's only used in the tracker itself.
MONO and STEREO binaries are for exports exclusively, slapped to either the SAP or XEX format, but other than that, it's almost exactly the same code everywhere.

So basically, once we go larger than 2264 bytes, the export players don't matter, as they won't work anymore. But we want the best player during composition, and an accompanying RMT2LZSS     

 

Here's my sa_c6502 library that injects code into memory based on the tag at 0x3182:

 

https://github.com/ivop/rmt-sa-libs/tree/main/real/sa_c6502_inject

 

If you place this DLL in an RMT directory, and copy tracker.obx there, too, your previously posted stripped Rmt.exe with the TRACKER tag, suddenly starts to work. It loads tracker.obx. It skips the first six bytes, and then loads the rest into 0x3182 and further. Your player can reach up to 0x3fff

 

Edited September 3, 2021 by ivop

[VinsCool]

1 minute ago, ivop said:

Here's my sa_c6502 library that injects code into memory based on the tag at 0x3182:

 

https://github.com/ivop/rmt-sa-libs/tree/main/real/sa_c6502_inject

 

If you place this DLL in an RMT directory, and copy tracker.obx there, too, your previously posted stripped Rmt.exe with the TRACKER tag, suddenly starts to work. It loads tracker.obx. It skips the first six bytes, and then loads the rest into 0x3182 and further. Your player can reach up to 0x3fff

Holy shit, incredible, I must give it a try-- I've got several .obxes around that can be swapped in and out in a few seconds!
Thank you so much!

2 minutes ago, ivop said:

So basically, once we go larger than 2264 bytes, the export players don't matter, as they won't work anymore. But we want the best player during composition, and an accompanying RMT2LZSS      

That sounds incorrect to me.
They will work regardless of the size. 

Things can be ORGed in other memory location, and would still work.
e.g.: my Visual Player project, I can slap pretty much anything I want.
And everything that is data before it will be moved to a different page at a later time, too.
for example, I made it so the Player still uses the same old address, RMT module still sits at $4000 but is at the literal end of the binary, then the visual player portion was moved to $8000, leaving a pretty comfortable buffer for the module in the next $4000 blocks.

But I'm getting offtrack for now, so I'll try this new .dll out!

[VinsCool]

Man this is so fucking AWESOME
It works, just like that! hahahaha!

Now the very first thing I can see now: The engine speed seems too fast? Like instruments speed 2 it sounds like.
I'm testing with the 1.28 stripped exe, and the tracker.obx from patch16 beta5.

But other than that, it's definitely taking my code!

 

[Edit] yep, it really seems to run at a 2x engine speed, but that's the only problem I could see for now!
I'll give a nice little thorough test and reports more detailed results
[Edit2] Well, more like... between 2? either way, that's something worth checking, but I cannot really see anything related to the speed in the github code.

Edited September 3, 2021 by VinsCool

[ivop]

12 minutes ago, VinsCool said:

Now the very first thing I can see now: The engine speed seems too fast? Like instruments speed 2 it sounds like.

That's weird. It is just the 6502 emulation, and the speed at which it is called depends on the caller, i.e. Rmt.

I have played numerous songs with this stripped 1.28 and tracker.obx injecting code, and I heard no discrepancies.

 

12 minutes ago, VinsCool said:

I'm testing with the 1.28 stripped exe, and the tracker.obx from patch16 beta5.

It should also work with the stripped 1.30 binary.

 

12 minutes ago, VinsCool said:

But other than that, it's definitely taking my code!

Good to hear Bed time again

[VinsCool]

Just now, ivop said:

That's weird. It is just the 6502 emulation, and the speed at which it is called depends on the caller, i.e. Rmt.

I have played numerous songs with this stripped 1.28 and tracker.obx injecting code, and I heard no discrepancies.

I can do some comparison later tonight, because I can totally tell something is a little off! haha!
Things do play, that's not the problem, it's just the timing that's off from what I can hear.
 

1 minute ago, ivop said:

It should also work with the stripped 1.30 binary.

Awesome, I'll also try and compare.
 

2 minutes ago, ivop said:

Good to hear Bed time again

Sleep well! Thanks a ton, this is seriously amazing shit there! ?

[VinsCool]

Okay, so I think it was indeed not the CPU emulation, because that exact same .dll in the non stripped version works pretty much exactly like the old one, I cannot hear any difference right now.
But the stripped player definitely exhibits something odd during the initialisation, because it's definitely not my .obx binary ?

I'll keep poking around and see!

[VinsCool]

Ok so I tested things out between the Stripped and Unstripped .exe with the new hacked sa_pokey.dll

The good news is, for everything I tested, as far as I could tell, the Unstripped .exe worked perfectly.

It was my RMT Patch16 Beta5 executable with that new hacked 6502 plugin, and it played everything without any problem at all.

For the Stripped .exe, it DID work for the majority of the things I tested, but there were many things I could observe:
- ALL of my custom hijacked code (15khz, 16-bit, etc) WAS working perfectly
- Timing and overall engine speed was ALL screwed up, even if the tempo WAS pretty much correct.
- There was several instances of odd garbage noise going in some sounds, like if channels were desynced from each others, it was particularly obvious during arps using the filter. 

- Stereo WORKS for all of the above, and the same observation could also be made for anything that was not correct.

So from all of this, I can say, thanks a ton @ivop for this new entrypoint, it is still a bit rough from what I have been able to witness, but the results so far are seriously blowing me away!
Can't wait to see what else could be done, now that I know I may be able to do even more code changes in the rmtplayr and not worry about compatibility, it makes me really excited for what will come next  

Here's video evidence of me testing that stuff, I think it's pretty easy to guess which played is which, even by not reading me rambling in a .txt file, testing everything in real time, because the element of surprise is always a good one to have right? hahaha.
 

that video took a fucking eternity to upload, but here is it anyway.

[ivop]

10 hours ago, VinsCool said:

- Timing and overall engine speed was ALL screwed up, even if the tempo WAS pretty much correct.
- There was several instances of odd garbage noise going in some sounds, like if channels were desynced from each others, it was particularly obvious during arps using the filter. 

These are related I think.

 

I have thought a lot about this today. Why does it behave differently when the obx is injected in the Rmt.exe binary, and when it is injected in the 6502 memory space?

 

I think RMT changes some bytes in the obx while or after it is copied to the 6502 memory space for emulation. When the obx is just "TRACKER " with zeroes, RMT changes values somewhere in there, but they are overwritten by the injection code. I'm going to test this hypothesis later.

 

As a last resort, I also thought about plan B. Use your code injection into Rmt.exe as usual (which works), and call out to library functions at, say, $2800, which are put in place by sa_c6502.dll code injection That way, you can still expand the player, without the current problem. Hopefully

Edited September 3, 2021 by ivop
typoo

[VinsCool]

I've been thinking about it a bit too.

I know there are some values that do get created in the same memory location the binary gets loaded, and some that did not.

As far as I could tell, the things such as engine speed or tempo are not part of that stuff, but would be in a different memory page.

 

I know the rmtplayr combined to a player for stand alone execution would do some things of its own, but for the actual tracker itself, I have yet no idea about this behaviour, does it use some sort of player that is bond to the tracker interface? Maybe... I did not think about this until yesterday when I did test the obx.

 

I think I really should try investigate a bit deeper in this system today.

While I am now a lot more familiar with the 6502 player + visual interface combination (my custom visual player, which is based on Pigu's own code, WAS based directly on the simple player by Raster, after all!), The way the tracker makes use of the rmtplayr is still a bit unknown to me.

 

For example, the ways I was able to break certain features without any logical explanation, or the fact certain things seemed to be hardcoded to very specific location or breaking down when technically unused in my own edited code, may be a good hint regarding this matter.

 

I already know where the simple visual player is located in the .exe, but what if the tracker also has its own of such?

That could probably explain the behaviour we both have observed.

 

That said, I am not a skilled reverse engineerer at all, but maybe now I can try a bit harder to find out what could be related, and see what could be involved to cause problems.

 

Out of curiosity, if you remain faithful to the limits of the original binary size (2270 bytes) and load the .obx, would it break something?

If no, then you might be on the right track assuming something gets overwritten.

 

I'd be interested to try some things of my own too, assuming I can even build a dll in the first place...

[rensoup]

6 hours ago, ivop said:

I think RMT changes some bytes in the obx while or after it is copied to the 6502 memory space for emulation. When the obx is just "TRACKER " with zeroes, RMT changes values somewhere in there, but they are overwritten by the injection code. I'm going to test this hypothesis later.

Not sure if you're aware or not but the the tracker source code for version 1.0 is still available on Raster's page so maybe there's stuff to learn there ?

[VinsCool]

17 minutes ago, rensoup said:

Not sure if you're aware or not but the the tracker source code for version 1.0 is still available on Raster's page so maybe there's stuff to learn there ?

That was exactly what I was looking at as well.
I think I found a possible explanation for my tests last night.

---

In the Release folder there are a bunch of very familiar files that have very interesting clues inside.

Specifically, let's compare this: 

 

In this image, I compared, in this order, my TRACKER binary, the RMT.exe that makes use of it, and then, a strangely familiar rmt_ata.sys
For obvious reasons the .obx and the binary inside of the .exe are identical, except...
See the rmt_ata.sys extra data? Doesn't that look very similar to what also is also the patched .exe?

 

Now let's take look at this: 

 

Doesn't this look familiar again?

The highlighted parts are the only differences between these 2 files.
This is the rmt_ata.sys file compared to the rmt_sap8.sys file.
And what else did I notice was mostly identical before?

The TRACKER and STEREO binaries I had assembled and inserted into RMT.exe, of course!

It seems like I really was not that far off the reality of what may have happened now.
There is some extra data right after the TRACKER binary, and this flew above my head all this time until now.

Now I think I may have an idea regarding what happened with the hijacked .dll  
It overwrote the whole area that had more infos, and it looks like this is precisely what caused most of the issues.

I will experiment some time later tonight with the .obx loader, and specifically, what the .obx file in question is, and see what happens.  
My first guess is, by adding the missing data at the end of the .obx, things may work exactly like tjey should, but we never know, haha
 

[VinsCool]

Hmm I think I am onto something now.
So it turns out, there are several things hardcoded in RMT.

Remember when I mentioned having a weird issue with stripping away unused code?
Well it turns out, it just happened to move a block of code doing so, and that block WAS actually hardcoded to that specific location.

Just taking a look here: 

  

After the block of nop, it's the extra TRACKER data I had added earlier, but now it does give a few clues regarding what exactly it did.
jmp $3576 and org $35FC in particular caught my attention.

 

Where did I see these 2 numbers before? Right in this part of the rmtplayr.a65:
 

Spoiler

 

SetUpInstrumentY2
3576 B1 CB                	lda (p_instrstable),y
3578 9D B0 30             	sta trackn_instrdb,x
357B 85 D7                	sta nt
357D C8                   	iny
357E B1 CB                	lda (p_instrstable),y
3580 9D B8 30             	sta trackn_instrhb,x
3583 85 D8                	sta nt+1
3585 A9 01                	lda #1
3587 9D 58 31             	sta trackn_filter,x
358A A8                   	tay
358B B1 D7                	lda (nt),y
358D 9D 38 31             	sta trackn_tablelop,x
3590 C8                   	iny
3591 B1 D7                	lda (nt),y
3593 9D C8 30             	sta trackn_instrlen,x
3596 C8                   	iny
3597 B1 D7                	lda (nt),y
3599 9D D0 30             	sta trackn_instrlop,x
359C C8                   	iny
359D B1 D7                	lda (nt),y
359F 9D 10 31             	sta trackn_tabletypespeed,x
35A2 29 3F                	and #$3f
35A4 9D 40 31             	sta trackn_tablespeeda,x
35A7 B1 D7                	lda (nt),y
35A9 29 40                	and #$40
35AB 9D 18 31             	sta trackn_tablemode,x
35AE C8                   	iny
35AF B1 D7                	lda (nt),y
35B1 9D 70 31             	sta trackn_audctl,x
35B4 C8                   	iny
35B5 B1 D7                	lda (nt),y
35B7 9D E0 30             	sta trackn_volumeslidedepth,x
35BA C8                   	iny
35BB B1 D7                	lda (nt),y
35BD 9D F0 30             	sta trackn_volumemin,x
35C0 C8                   	iny
35C1 B1 D7                	lda (nt),y
35C3 9D F8 30             	sta trackn_effdelay,x
35C6 C8                   	iny
35C7 B1 D7                	lda (nt),y
35C9 A8                   	tay
35CA B9 92 31             	lda vibtabbeg,y
35CD 9D 00 31             	sta trackn_effvibratoa,x
35D0 A0 0A                	ldy #10
35D2 B1 D7                	lda (nt),y
35D4 9D 08 31             	sta trackn_effshift,x
35D7 A9 80                	lda #128
35D9 9D E8 30             	sta trackn_volumeslidevalue,x
35DC 9D A8 30             	sta trackn_instrx2,x
35DF 0A                   	asl @
35E0 9D D8 30             	sta trackn_instrreachend,x
35E3 9D 78 30             	sta trackn_shiftfrq,x
35E6 A8                   	tay
35E7 B1 D7                	lda (nt),y
35E9 9D 30 31             	sta trackn_tableend,x
35EC 69 00                	adc #0
35EE 9D C0 30             	sta trackn_instridx,x
35F1 A9 0C                	lda #INSTRPAR
35F3 9D 28 31             	sta trackn_tablea,x
35F6 A8                   	tay
35F7 B1 D7                	lda (nt),y
35F9 9D 20 31             	sta trackn_tablenote,x

xata_rtshere
35FC 4C 6E 35             	jmp p2x0

 

 

 

So it turns out, my weird workaround that was to not remove the few trackn_outnote related lines was coinciding with the memory map that was just the right one for it to work properly.
So that's one thing I can easily change at a later time, very easily, which is nice  

But wait! There's more to this, because I had overlooked yet another crucial element.

In that same screenshot, there were some other interesting values: L3060,x, L3150,x, L30A8,x, L3068,x, L30B8,x, L30B0,x, L3168,x
Now where did I see these before... let's take a look at my rmtplayr.lab file:

3060 TRACKN_NOTE
3068 TRACKN_VOLUME
30A8 TRACKN_INSTRX2
30B0 TRACKN_INSTRDB
30B8 TRACKN_INSTRHB
3150 TRACKN_OUTNOTE    ; HMMMMMMM I wonder where I saw this one before...
3168 TRACKN_AUDC

At least, now I know what to do if I want to have a better control of the rmtplayr code in the more sensible sections, because now I know exactly why things break when move things around  

...
Now regarding the .obx loader, I did get mild success to add that extra part at the TRACKER binary to the .obx file, and that did sort of fix a few things, but the timing still was off, so there may be more things to look at...

What puzzles me is that I am sure there is nothing in my labels, the one last one is at 3A5A being RMTPLAYEREND.
So... there are certainly more things to look at.

I did take a look at the very old RMT code, and found something that more or less confirmed my observation, even without any understanding of C languages, I can immediately understand what is going on there:
 

Spoiler

 

int Atari_LoadRMTRoutines()
{
	//nacte rmt rutinu $3400, setnoteinstrvol $3d00, setvol $3e00
	WORD min,max;
	int r = LoadBinaryFile((char*)(LPCSTR)(g_prgpath+"rmt_ata.sys"),g_atarimem,min,max);
	return r;
}

int Atari_InitRMTRoutine()
{
	int r = GO(0x3400,0,0x00,0x3f);			//adr,A,X,Y
	return r;
}

void Atari_PlayRMT()
{
	//GO(0x3403,0,0,0);					//SetPokey + jeden prubeh RMT rutinou
	GO(0x3406,0,0,0);					//(bez SetPokey) jeden prubeh RMT rutinou, ale az od rmt_p3 (zpracovani obalek)
	GO(0x340c,0,0,0);					//SetPokey
	//prepis z g_atarimem do POKEYe
}

void Atari_Silence()
{
	GO(0x3409,0,0,0);
	//Silence rutina
}

void Atari_SetTrack_NoteInstrVolume(int t,int n,int i,int v)
{
	GO(0x3d00,n,t,i);		//adr, nota, track, instrument
	GO(0x3e00,v,t,0);		//adr, volume, track
}

void Atari_SetTrack_Volume(int t,int v)
{
	GO(0x3e00,v,t,0);		//adr, volume, track
}

 

 

So indeed, few things do exist at $3D00, $3E00 and $3F00 (the RMT8 header with the entire alphabet) and seem to be hardcoded, and most of the same does exist in the current version too.

Hopefully this will help figuring out more workarounds.
The hardcoded addresses are a very obvious choice, if we can more the memory map to other locations, surely, much larger player binaries would work, as long as the other hardcoded addresses don't overlap, and point to the right locations  

Edited September 4, 2021 by VinsCool
typo

[rensoup]

8 hours ago, VinsCool said:

So indeed, few things do exist at $3D00, $3E00 and $3F00 (the RMT8 header with the entire alphabet) and seem to be hardcoded, and most of the same does exist in the current version too.

Could these be the addresses of a dummy track to play a single sound when entering notes ?

[ivop]

10 hours ago, VinsCool said:

The hardcoded addresses are a very obvious choice, if we can more the memory map to other locations, surely, much larger player binaries would work, as long as the other hardcoded addresses don't overlap, and point to the right locations  

Great work! Still, I don't see any location outside of the $3182 - $3a59 area?

 

I have to check memory changes in between calls to execute 6502 code, i.e. save a copy of the memory after the last RTS, and compare it with current memory when called again. I can also check, with a non-stripped exe, if the memory still matches what was injected into the Rmt.exe, or if RMT changed it after writing to the memory array. Couldn't find the energy yesterday, perhaps tonight.

 

BTW if you want to build the injector library yourself, you need to install a cross-compilation toolchain. As you are also on Linux, that shouldn't be too much of a problem. Debian based distros it's 'apt-get install gcc-mingw-w64-i686', which should pull in all dependencies, too. Install git, clone the repo, cd to the right directory, and make

Edited September 4, 2021 by ivop

[VinsCool]

6 hours ago, rensoup said:

Could these be the addresses of a dummy track to play a single sound when entering notes ?

Honestly, I have no idea when even these are being read.

All I know is altering the contents of these will cause various side effects in the playback time.

 

I did not even know these were actual code until I tried to read a little more thoroughly there.

[emkay]

6 hours ago, rensoup said:

...

Btw. 

As now things have been changed tremendously. 

Could you suggest an Amiga Mod file to convert?

[ivop]

Hypothesis seems to be correct:

 

$ wine Rmt.exe 
C6502_Initialise: memory is zeroed
C6502_JSR: tracker memory changed
C6502_JSR: TRACKER tag found, loading file at $3182
load_raw: read tracker.obx and skip 6 bytes
load_raw: injected 2264 bytes
C6502_JSR: tracker memory changed
C6502_JSR: tracker memory changed
C6502_JSR: tracker memory changed
... ad infinitum

 

I do a memcmp(previous_memory+0x3182, current_memory+0x3182, 2264), and detect that it has been altered in between 6502 emulation calls. Conclusion: bytes are changed within the player memory area in between calls to C6502_JSR(). Now find out which, and why

 

All code to replicate this is on github.

Edited September 4, 2021 by ivop

[VinsCool]

Great!

 

Personally I managed to get myself fucked.

 

No idea how the fuck that is even possible but I installed MinGW and now I am stuck with a laptop that no longer boots ? 

So much for just trying to build a .dll... orz

 

I need to troubleshoot this shit for now... Lol

 

[Edit] ok I fixed it. Guess it's simply a matter of booting to a cli screen and update from there lol

Edited September 4, 2021 by VinsCool