RMT Hacking Ideas and Progress

[ivop]

7 hours ago, Wrathchild said:

Can you delta these with the replacement ones people should be using

I have been thinking about updating both sa_pokey and sa_c6502 with the latest implementations that are currently in atari800. No idea how much work that is

[ivop]

2 hours ago, ivop said:

I have been thinking about updating both sa_pokey and sa_c6502 with the latest implementations that are currently in atari800. No idea how much work that is

I have done a diff -biNur cpu.c ~/git/atari800/src/cpu.c | less, and it seems there's a lot of renaming of variables (a good thing), but I don't see any bugfixes in the way the CPU is emulated. That's pretty solid already in the 2003 version.

 

I want to cleanup the standalone CPU code a bit and check all the commit messages for atari800 after 2003 for the cpu.c file if there was a bugfix instead of only cosmetics.

Edited September 1, 2021 by ivop

[VinsCool]

5 hours ago, emkay said:

Normally the deep 15kHz sounds on all channel results in a suffocating mashup. Correct gen 2 handling stabilizes the whole thing .

 

 

 

Most importantly, my patch 16 code for handling 15khz tuning plays a really important role there.

First it was adjusted to be mostly correct in 15khz for both Distortion A and C, and then, the Distortion 2 (1.79mhz) tuning was also adjusted to be in-tune with any situation , so between 64khz and 15khz, there won't be any severe penality.

 

The only difference is the automatic handling of the 1.79mhz clock for Distortion 2 on top of the already improved 15khz code

 

Try making a module bouncing between modes for example, 15khz set from only 1 instrument, you will see things should remain pretty robust compared to the original code used in RMT

 

Even RMT2LZSS cannot handle that sort of trigger automatically, yet

 

 

 

[VinsCool]

23 hours ago, VinsCool said:

So to make a quick summary, here's what I finally stuck to after I realised how much I was on the wrong track:
- check for Distortion 2 for the first step
- proceed to the hijack like earlier (since that part did work perfectly fine after all)
- ORA the new AUDCTL value ($20 or $40) after the checks passed with trackn_audtl,x, then ORA with a new variable called v_dist2flag in the zeropage, then STA that value in v_dist2flag
- at the moment all 4 channel's own trackn_audctl are ORA'd together, include the v_dist2flag in the process, making all AUDCTL values written in memory
- once the step above is done, LDA #0, and STA into v_dist2flag to reset it for the next time the rmtplayr is called

I just realised I did an extra unnecessary step in this implementation, that could save some bytes and some cycles...

 

That part:

ORA the new AUDCTL value ($20 or $40) after the checks passed with trackn_audtl,x, then ORA with a new variable called v_dist2flag in the zeropage, then STA that value in v_dist2flag

Could have been just:

ORA the new AUDCTL value ($20 or $40) after the checks passed with a new variable called v_dist2flag in the zeropage, then STA that value in v_dist2flag

And would have worked just as well.

 

I think I first did this because I originally intended to save in trackn_audctl,x, but forgot what I did earlier would not be reset until a new note was played, and so that step would not do anything as a result, oops.

 

Not gonna post a new version just for a tiny change, it could be NOPed out with a hex editor if I wanted to, but that's pointless to bother for such a small edit, hehe.

I will make sure to remember when I revise the code later.

 

That variable change could actually lead to even more hijacks involving the AUDCTL value too, actually.

As long as I don't write anything into trackn_audctl,x or v_audctl, I could use that v_dist2flag as a more generic AUDCTL flag, in which I can hijack anything I want, until the time all channels are ORA'd together, effectively enabling the hijacked trigger to work with anything if I wanted to.

Maybe an even more robust implementation of 16-bit, who knows?

 

I did mention that the current 16-bit code was temporary disabled because I ran out of space, and ideally I would bring it back, or, I could use the space the tables used and add more 8-bit tables, I dunno yet.

 

I don't exactly like the idea of branching my hack into 2 different versions, so that would just be a only 1 time improvement, really, if I do some changes specific to the Distortion 2 hack to help having all the space used intelligently instead of leaving a full 128 bytes of tables unused.

That 128 bytes was salvaged from the old volume code, and really was a big improvement on the memory side.

 

I even wonder if I could go as far as reducing it to 64 bytes... But I worry about the precision of the volume envelope would suffer as a result...

 

 Baaah sorry, I'm rambling again about that stuff, hahaha

Just having random ideas out loud helps me think about it, and then anyone better than me could provide some advices about what I try to achieve. Haha.

 

 

[VinsCool]

2 hours ago, ivop said:

I have done a diff -biNur cpu.c ~/git/atari800/src/cpu.c | less, and it seems there's a lot of renaming of variables (a good thing), but I don't see any bugfixes in the way the CPU is emulated. That's pretty solid already in the 2003 version.

 

I want to cleanup the standalone CPU code a bit and check all the commit messages for atari800 after 2003 for the cpu.c file if there was a bugfix instead of only cosmetics.

Do you think you can also cleanup the Atari800 POKEY code for a more up to date sa_pokey.dll too?

Even if not ideal, at least, we would no longer have to chose between a very old emulation plugin, or a much better apokeysnd.dll that has a foggy origin ?

 

[emkay]

Everything fine. Except I'm still not trusting the sound in RMT

So how to get out of the sandbox ?

 

 

Edited September 1, 2021 by emkay

[VinsCool]

6 minutes ago, emkay said:

Everything fine. Except I'm still not trusting the sound in RMT

So how to get out of the sandbox ?

Export .xex, .sap works fine too.
The code is already in place for the exports to work.

I can recognise Secret of the Monkey Island played on a different key there, sounds nice.

Edited September 1, 2021 by VinsCool
Addendum

[emkay]

3 minutes ago, VinsCool said:

Export .xex, .sap works fine too.
The code is already in place for the exports to work.

 

Have to check it

3 minutes ago, VinsCool said:

 

I can recognise Secret of the Monkey Island played on a different key there, sounds nice.

The point is that it sounds unbelievable calm. 

 

[Wrathchild]

6 hours ago, ivop said:

No idea how much work that is

I recall looking at it a long time ago but the codebase changes between major versions of Atari800 put me off. 

I would think making a variant from Altirra sources may be the better option?

[ivop]

4 hours ago, ivop said:

I want to cleanup the standalone CPU code a bit and check all the commit messages for atari800 after 2003 for the cpu.c file if there was a bugfix instead of only cosmetics.

OK, I have done so on github. The only bug that was fixed (partly) was the page zero wrap around bug. All the other commits were cosmetics. I didn't like the way it was sort-of fixed, so I did not apply those changes. It's not a bug you are likely to encounter, especially not within RMT. But I left the comment at the top in place. Other than that, I removed all dead code and cruft. It's a lot more readable right now and can be used to patch the memory/player on the fly.

2 hours ago, VinsCool said:

Do you think you can also cleanup the Atari800 POKEY code for a more up to date sa_pokey.dll too?

Yes, my next "project" is to add the Raster provided sa_pokey DLL source code to github, and clean it up similar to the CPU code. Then check all the commit messages in the atari800 repo (which were surprisingly little on cpu.c in 18 years), and hopefully apply them

2 hours ago, VinsCool said:

Even if not ideal, at least, we would no longer have to chose between a very old emulation plugin, or a much better apokeysnd.dll that has a foggy origin ?

I'll do my best

28 minutes ago, Wrathchild said:

I recall looking at it a long time ago but the codebase changes between major versions of Atari800 put me off. 

Now I'm scared

28 minutes ago, Wrathchild said:

I would think making a variant from Altirra sources may be the better option?

Have you ever looked at the Altirra Pokey sources? It's so intertwined with the main codebase, being cycle exact and all. It's beyond my capabilities to turn it into a standalone library. And it's C++.

 

A totally new C implementation based on the latest atari800 C code, apokeysnd's CI code, and Altirra's C++ code might be an endeavour for the future.

 

[ivop]

Okidoki

 

https://github.com/ivop/rmt-sa-libs/tree/main/real

 

I added the sa_pokey DLL from 2003. And cleaned it up, too.

 

Basically, for both sa_c6502 and sa_pokey, all ifdeffery has been removed. If it was not compiled in the first place, and/or was atari800 related (breakpoints, savestates), it's all gone

 

The if(n)defs were a pain in the behind BTW. #ifndef NO_SOME_FEATURE. Now what does that mean?

 

Note that the Pokey emulation is sub-par. It's 2003 quality, but the code is a lot more readable now, and I plan to go over the commits to pokey.c and pokeysnd.c in the atari800 repo that have been done since February 2003.

 

Also note how terribly bad the resampling is in Pokey_Process(), which is under the #ifndef NOSNDINTER block of code. (read as: if not defined no sound interpolation )

 

The libraries can be (re)compiled with the makefiles if you have a proper (debian) Linux i686-w64-mingw32 cross-compilation toolchain, or something similar with msys32 or cygwin. Again, the compiler improvement in 18 years is remarkable!

 

Now it's time to go to bed.

[VinsCool]

Freaking awesome ivop! Thank you very much for going the extra miles for these parts!
You absolutely deserve having some rest now! haha  

On my side, I continue exploring the 6502 world of programming, and I will hopefully be able to share new progress some time later  

[ivop]

14 hours ago, VinsCool said:

On my side, I continue exploring the 6502 world of programming, and I will hopefully be able to share new progress some time later  

Have fun

 

I have cleaned up the code some more, dos2unix, reindent, helps a lot to increase the readability of pokeysnd.c. Its indentation was a mess. Now I have to go over all the commits since 2003

 

About sa_c6502.dll, it's pretty much done, and ready to be patched for larger players! For now, I think each of the players in the Rmt.exe binary need to be replaced by a TAG and zeroes. For example, the MONO player is just:

 

    .byte "MONO"

 

and the rest doesn't matter. Once the 6502 emulation is called, it can check for this TAG and put the right player in place before running code. Next time, it won't find the tag and just leave the memory alone. Once you switch to STEREO for example, it'll suddenly finds the "STER" tag, puts the right player in place, and continue..

 

Could you provide such an Rmt.exe binary, i.e. with all players replaced by a 4-byte or 8-byte tag?

 

Edit: it would also help if you share the exact .org of each player, and where the .rmt file is injected into the memory array.

 

Edit: and you can just use apokeysnd.dll, no need for an obsolete sa_pokey.dll, even though it is not (yet) known which frequency it uses. For either PAL or NTSC it is wrong (similar to sa_pokey.dll which is always resampled as if it were NTSC), or perhaps both

Edited September 2, 2021 by ivop

[VinsCool]

55 minutes ago, ivop said:

About sa_c6502.dll, it's pretty much done, and ready to be patched for larger players! For now, I think each of the players in the Rmt.exe binary need to be replaced by a TAG and zeroes. For example, the MONO player is just:

 

    .byte "MONO"

 

and the rest doesn't matter. Once the 6502 emulation is called, it can check for this TAG and put the right player in place before running code. Next time, it won't find the tag and just leave the memory alone. Once you switch to STEREO for example, it'll suddenly finds the "STER" tag, puts the right player in place, and continue..

 

Could you provide such an Rmt.exe binary, i.e. with all players replaced by a 4-byte or 8-byte tag?

Awesome!

Sure I can go get you a patched .exe with the necessary changes you need.

Just off the computer for the moment so replying on mobile isn't ideal.

 

I assume it's as early as the Atari binaries start? So right in the header?

 

57 minutes ago, ivop said:

Edit: it would also help if you share the exact .org of each player, and where the .rmt file is injected into the memory array.

That is also easy, I just need to check on my computer before saying anything incorrect.

 

58 minutes ago, ivop said:

Edit: and you can just use apokeysnd.dll, no need for an obsolete sa_pokey.dll, even though it is not (yet) known which frequency it uses. For either PAL or NTSC it is wrong (similar to sa_pokey.dll which is always resampled as if it were NTSC), or perhaps both

Very nice, I am aware of it

i just wondered what would a up to date sa_pokey.dll be since I believe Atari800 had gone through a lot of improvements since 2003

No harm in having that for sure, even if probably pointless now that you say it that way, haha

[ivop]

32 minutes ago, VinsCool said:

I assume it's as early as the Atari binaries start? So right in the header?

Yes, the first four or eight bytes that are written to memory, so that are the bytes right after the six byte FFFF header.

 

Note that this way to increase the player size will probably work, you can even run code in the OS area. There's no OS, no ROM, no real hardware registers.

 

But exporting SAP or XEX will cease to work. If it doesn't fit in the Rmt.exe binary, it won't work. But in the end, we run the .rmt file through RMT2LZSS, right?

Edited September 2, 2021 by ivop

[emkay]

The settings really allow to do some "multichannel" harmonics and to use it on a broad range of notes. 

 

Using the generators for some special FX makes even more fun then. 

 

 

[VinsCool]

Alright so here I quickly edited in the HEX editor the last RMT Patch16 Binary to strip away the binaries, and replace them with a FF FF header followed by 8 bytes of characters to identify them: 'TRACKER ', 'MONO    ' and 'STEREO  ', in that order with the spaces as filler ($20), all filled with Zeroes for their entire size.

- Both the TRACKER and STEREO binaries are 2270 bytes, and virtually identical, however there is a couple things left different on purpose or else the tracker program would malfunction. I suspect hardcoded memory locations being used there

- The MONO player is only 2002 bytes, and the most difficult to get the code fitting since it is considerably smaller than its stereo counterpart, which just has a bunch of extra blocks that are usually duplicates with slight changes for the second POKEY

Now regarding the ORG of them each:
- All binaries begin with the header FF FF, then 82 31 for the ORG, the next part depends on the binary size, as expected, so it will always be different between exports, however,
- In the TRACKER binary, I left it exactly like it was assembled, so it's FF FF 82 31 then 59 3A 
- RMT modules will always ORG at $4000, it is hardcoded in the files themselves, and behave like proper Atari executables
- In exports, the Player first loads in the memory, then, the small player routines (which is just a bunch of calls for graphic, speed, etc), that one is slapped immediately at the end of the Player binaries, after that, the RMT module is slapped there, until the end of the file.

I have no clue about how exactly this setup works regarding the second part of the player in exports, so let's not bother about this one for now.

So below I'll attach the stripped RMT.exe, and the 3 Binaries I had built for patch16 Beta5.
When I originally inserted them into RMT, I only fully overwrote the TRACKER binary since the header data was identical, the other ones, I skipped the first 6 bytes in case anything broke, and that probably wouldn't have done any difference so whatever  

rmtplayr Patch16 Beta5 (MONO).obx rmtplayr Patch16 Beta5 (STEREO).obx rmtplayr Patch16 Beta5 (TRACKER).obx RMT STRIPPED OFF BINARIES.exe

[ivop]

It looks like you overwrote the start and end address in the RMT binary?

 

What I meant was FF FF 82 31 xx xx 'M" "O" "N" "O", so it loads at 0x3182 and I can detect it in the sa_c6502 DLL and replace it with your .obx code.

[VinsCool]

9 minutes ago, ivop said:

It looks like you overwrote the start and end address in the RMT binary?

 

What I meant was FF FF 82 31 xx xx 'M" "O" "N" "O", so it loads at 0x3182 and I can detect it in the sa_c6502 DLL and replace it with your .obx code.

OH, my bad! I misunderstood the instruction.
let me fix this right away.

 

RMT STRIPPED OFF BINARIES.exe

[ivop]

3 hours ago, emkay said:

The settings really allow to do some "multichannel" harmonics and to use it on a broad range of notes. 

 

Using the generators for some special FX makes even more fun then. 

 

 

Love it! It sounds so crisp. No muffling capacitors

[emkay]

11 minutes ago, ivop said:

Love it! It sounds so crisp. No muffling capacitors

Gen. 2 helps a lot to correct thing, particular to de-yell almost everything. 

It's musically impressive, even if no 16 bit is used. 

What's still missing is the final stabilization of the polycounter basses and the direct programming of PWM. 

Everything else is satisfying enough, to do tunes with a broad range of usable music FX. 

[VinsCool]

1 minute ago, emkay said:

Everything else is satisfying enough

I agree, even in the current shape, the Patch16 Beta5 hack really have improved several peculiar things, in my opinion.
I do think much much more is still possible, however  

[emkay]

1 minute ago, VinsCool said:

I agree, even in the current shape, the Patch16 Beta5 hack really have improved several peculiar things, in my opinion.
I do think much much more is still possible, however  

Of Course. But your latest added Gen.2 feature is a literal key changer  

[_The Doctor__]

now emkay can fix all that stuff he's been mulling over for year and put the xex's up... he may win a day or two! Nice

[_The Doctor__]

3 hours ago, emkay said:

The settings really allow to do some "multichannel" harmonics and to use it on a broad range of notes. 

 

Using the generators for some special FX makes even more fun then. 

 

 

I want this please!