RMAC/RLN

[ggn]

Right, so I managed to pass through the maze of twisty little tunnels, all alike, and came out with parts of my sanity :).

 

I have a fix for the situation but it's super tentative for now - I'll have to run this past Shamus and we'll need to check with our regression suite to see if nothing breaks (because yes, we do have such a thing nowdays!). Without further ado:

 

Open object.c and go to line 175. There should be an "else" in that line, a closing brace directly above and an opening brace directly below. Change the "else" to "if (w1 & DEFINED)", recompile rmac and you should be good to go!

 

(unless I screwed something up. Never underestimate this!)

Edited November 6, 2019 by ggn

[+BitJag]

On 11/6/2019 at 12:35 PM, ggn said:

First of all, no worries about the supposed wild goose chase - it's all good!

 

As for which sound.o I used: well, I just unpacked the "Jaguar_linux_dev_setup_rv6.zip" and started looking for the filenames of the objects I wanted to link against. It might have been from the "old" folder or the "new" but I'm not sure. The thing is, all sound.o files in that archive (and there are 3 of them) are byte for byte identical between each other. I also did some byte comparisons between the files in the folders the 3 sound.o files are and the files in the .zip file I uploaded above. I still get byte match for all files.

 

Next thing I tried was to assemble sound.s with rmac myself. The resulting .o file isn't byte match identical to the archive's .o files. So that's progress I guess! I'll see what happens from here, there were quite a few changes that happened for v2.0.0 of rmac so it's not impossible that something upset the jrisc parts. I'll have a look and will report back when I have something. Stay tuned....

 

[EDIT] Now that I think about it: do you happen to know what version of rmac you used to assemble the sound.o in the archives? Or have I got it wrong and it was assembled using mac?

Sorry for taking so long to respond.  I will respond to this quoted post, and then to your latest post shortly after.

 

I am running two completely different tool chains right now.  My "old" one, using mac/aln and old versions of jlibc/rmvlib (I am attempting to completely move away from this static tool chain, now unusable with the latest linux kernel).  Then there is my "new" tool chain, this is the latest versions of rmac/rln/jlibc/rmvlib from git hub (I assume they are the latest since these sources are from the links I was provided in the other thread about a.out support).   

 

These links are where I am getting what I assume are the latest sources for my new tool chain.

git clone http://shamusworld.gotdns.org/git/rmac
git clone http://shamusworld.gotdns.org/git/rln
git clone https://github.com/sbriais/jlibc.git
git clone https://github.com/sbriais/rmvlib.git

 

The Jaguar_linux_dev_setup_rv6.zip archive I provided is my old tool chain and everything is built with mac (nothing is built with rmac as far as I know).  I dropped the ball by not providing you a built version of my new tool chain (I only provided the bash script that builds this new tool chain from scratch in linux).   I didn't register in my head that you weren't planning on getting a linux install going to test my environment, and I don't expect you to do so.

 

I already implemented the suggested change to rmac/object.c that you have in your latest post, and my new tool chain is now compiling the new program just fine.  But I still want to post archives of my old tool chain, and my new tool chain (before the change to object.c) for comparison, so you are able to reproduce this scenario if necessary.  Keep in mind that both these tool chains are completely separate in their use and testing in my case, and are separated by two different physical computers.  I am compiling the same program in both tool chains, my old tool chain gives me a working program (the object files I believe you have been testing against in the post I have quoted above), my new tool chain, minus your latest suggested edit to rmac/object.c, does not give me a working program (object files you haven't been able to test until now). 

 

Old Tool Chain - (this is the google drive link to Jaguar_linux_dev_setup_rv6.zip, which you have already downloaded once before  - https://drive.google.com/file/d/10Zk1lYoclQo72DWjewiEvX_hmgvCip6G/view?usp=sharing

New Tool Chain - (you haven't had access to this until now.  This is built using the bash script posted from before) - See File Attached to This Post

Program to Compile - (same program that can be compiled in either tool chain, after makfile.config changes.  compiled program works in old tool chain, but is not working in new tool chain) - Posted multiple times and can be found in one of my posts above

 

Thanks again, I will be responding to your latest post in just a moment.

11-7-19_new_tool_chain.zip

[+BitJag]

On 11/6/2019 at 2:34 PM, ggn said:

Right, so I managed to pass through the maze of twisty little tunnels, all alike, and came out with parts of my sanity :).

 

I have a fix for the situation but it's super tentative for now - I'll have to run this past Shamus and we'll need to check with our regression suite to see if nothing breaks (because yes, we do have such a thing nowdays!). Without further ado:

 

Open object.c and go to line 175. There should be an "else" in that line, a closing brace directly above and an opening brace directly below. Change the "else" to "if (w1 & DEFINED)", recompile rmac and you should be good to go!

 

(unless I screwed something up. Never underestimate this!)

Thank you for you work here!  This tentative workaround seems to be working.  I have tested out with the example program and it works great in my new tool chain.  I have attached a prebuilt version of the tool chain (still need to manually download m86k-atari-mint-gcc through ubuntu ppa if you want to compile jag programs from source), along with an updated version of the bash script for anyone interested installing everything from scratch in Linux.

 

Just to clarify, there isn't anything wrong with the rmvlib source, specifically rmvlib/sound/sound.s, this bug seems to be all on rmac?

11-7_19_RMAC_RLN_JLIBC_RMVLIB_Setup_Scripts.zip 11-7-19_RMAC_RLN_JLIBC_RMVLIB_linux_static_build.zip

[ggn]

7 hours ago, lachoneus said:

Just to clarify, there isn't anything wrong with the rmvlib source, specifically rmvlib/sound/sound.s, this bug seems to be all on rmac?

Short answer: yes.

Long answer (skip if you're bored): At the bottom of sound.s there are a few labels that are at the same time exported as globals and equated (EQU). rmac's COFF file writer hadn't expected that possibility and simply exported the labels as absolute. The problem with that is that the linker isn't allowed to modify absolute labels, so it simply uses whatever value it's fed. BUT: the equates are in fact relocatable addresses! Oops!

 

Of course the most important thing is:

8 hours ago, lachoneus said:

Thank you for you work here!  This tentative workaround seems to be working.

which is good! I will try to create your dev environment above but I'm glad we at least fixed this without breaking anything else :).

 

Also, since you did so much work and want to share it with people perhaps you should make a new topic so it'll have more visibility?

[Chilly Willy]

6 hours ago, ggn said:

Long answer (skip if you're bored): At the bottom of sound.s there are a few labels that are at the same time exported as globals and equated (EQU). rmac's COFF file writer hadn't expected that possibility and simply exported the labels as absolute. The problem with that is that the linker isn't allowed to modify absolute labels, so it simply uses whatever value it's fed. BUT: the equates are in fact relocatable addresses! Oops!

So is that really a bug in rmac, or in the sound code? Should the labels have been SET rather than EQUed? I could see the author of both arguing the other is wrong.

 

[joeyaage]

Forgive me if I am mistaken but as far as I can tell the only difference between the two versions of the mod_play_crash program are that one is linked with original rmvlib made with MAC (it works) and the other is made with rmvlib made with RMAC including the tiny changes in display.s, sound.s, paula,s (it doesn't work).

The changes made to enable the rmvlib to be made with RMAC are at a presumeably sensitive part of code that is to do with padding 'nops'. The change is from:

         padding_nop    (D_RAM+$20-*)  

to:

         pc1=*

         padding_nop    (D_RAM+$20-pc1)

This adds extra bytes to the code to contain the PC value (pc1=*).

Would it not be better to change the values in display.s,sound.s,paula.s to

pc1:  padding_nop    (D_RAM+$20-pc1)          ;; and etc for other files...

so that the byte count remains the same (I will try to get to testing this, I am nearly there).

 

((OOPS!!, using a label in an expression doesn't work like this)), maybe try

         padding_nop    (D_RAM+$20-[length of pc1=* which I think maybe 2])

 

BTW, padding_nop is a macro that is in the files sound.s.ORIG and sound.s.LONG but not in sound.s (???)

 

Edited November 16, 2019 by joeyaage
Not working yet solution...

[swapd0]

I've been looking at the documentation but I haven't found it, it's possible to undefine an equr?

[LinkoVitch]

30 minutes ago, swapd0 said:

I've been looking at the documentation but I haven't found it, it's possible to undefine an equr?

Yep EQURUNDEF is your friend.

[+DrTypo]

It seems I've found a bug in the way rmac handles registers. It looks like rmac doesn't care about the "r" in the register name.

"r15" and "15" are the same.

For example, rmac doesn't complain in the following cases:

add 7,1   => assembled to add r7,r1

store 5,(2) => assembled to store r5,(r2)

addq #r31,3 => assembled to addq #31,r3

movei #r8,2 => assembled to movei #8,r2

 

I'm looking at the rmac source code but this whole tokenizer/parser thing is a bit black magic for me

 

 

[JagMod]

If you're looking at the parser...

 

This does not work:

x        REGEQU    r1
y        REGEQU    r2
z        REGEQU    r3

 

Error: syntax error; expected symbol

 

 

however, this does work:

xx        REGEQU    r1
yy        REGEQU    r2
zz        REGEQU    r3

 

[42bs]

21 hours ago, swapd0 said:

I've been looking at the documentation but I haven't found it, it's possible to undefine an equr?

Maybe here:
http://rmac.is-slick.com/

[ggn]

12 hours ago, DrTypo said:

It seems I've found a bug in the way rmac handles registers. It looks like rmac doesn't care about the "r" in the register name.

"r15" and "15" are the same.

For example, rmac doesn't complain in the following cases:

add 7,1   => assembled to add r7,r1

store 5,(2) => assembled to store r5,(r2)

addq #r31,3 => assembled to addq #31,r3

movei #r8,2 => assembled to movei #8,r2

 

I'm looking at the rmac source code but this whole tokenizer/parser thing is a bit black magic for me

 

 

Interesting, thanks for the report, I'll take a look at this as I personally haven't given the Jaguar side of rmac a lot of testing.

 

The tokenizer is easy, it just scans for bits of text inside the line and tries to match them to existing keywords or registers, and then creates tokens for them ;). But, I see your point, it's not easy to dig into the specifics by just browsing the code. I usually step through that thing using a debugger attached.

12 hours ago, JagMod said:

If you're looking at the parser...

 

This does not work:

x        REGEQU    r1
y        REGEQU    r2
z        REGEQU    r3

 

Error: syntax error; expected symbol

 

 

however, this does work:

xx        REGEQU    r1
yy        REGEQU    r2
zz        REGEQU    r3

 

This is a side effect of the tokenizer matching x and y as registers (the 6502 and 56001 CPUs have those). It's a known issue but it hasn't bothered us too much to look into yet. A quick look into the tokeniser shows

			// If the symbol is small, check to see if it's really the name of
			// a register.

So if we add a small check to omit this step in the first line token it should be fixed. Stay tuned!

 

 

Thanks to everyone for the feedback so far!

 

[JagMod]

One more small suggestion for rln.

Change from malloc() to calloc() in rln.c  to expose several bugs.

 

[42bs]

1 hour ago, ggn said:

Interesting, thanks for the report, I'll take a look at this as I personally haven't given the Jaguar side of rmac a lot of testing.

 

The tokenizer is easy, it just scans for bits of text inside the line and tries to match them to existing keywords or registers, and then creates tokens for them ;). But, I see your point, it's not easy to dig into the specifics by just browsing the code. I usually step through that thing using a debugger attached.

This is a side effect of the tokenizer matching x and y as registers (the 6502 and 56001 CPUs have those). It's a known issue but it hasn't bothered us too much to look into yet. A quick look into the tokeniser shows

			// If the symbol is small, check to see if it's really the name of
			// a register.

So if we add a small check to omit this step in the first line token it should be fixed. Stay tuned!

 

 

Thanks to everyone for the feedback so far!

 

Or you may add support for an optional ":" after symbols to differ them from reserved words.

[42bs]

13 hours ago, DrTypo said:

It seems I've found a bug in the way rmac handles registers. It looks like rmac doesn't care about the "r" in the register name.

"r15" and "15" are the same.

For example, rmac doesn't complain in the following cases:

add 7,1   => assembled to add r7,r1

store 5,(2) => assembled to store r5,(r2)

addq #r31,3 => assembled to addq #31,r3

movei #r8,2 => assembled to movei #8,r2

 

I'm looking at the rmac source code but this whole tokenizer/parser thing is a bit black magic for me

 

 

? If you ever programmed PowerPC, it is default there. Which is a constant source for error:

add 1,2,3 => r1 = r2 + r3

addi 1,2,3 => r1 = r2 + 3

 

So a small typo is a big difference. So luckily the RISC mnemonics use "#" for immediate values.

[+cubanismo]

1 hour ago, ggn said:

This is a side effect of the tokenizer matching x and y as registers (the 6502 and 56001 CPUs have those). It's a known issue but it hasn't bothered us too much to look into yet. A quick look into the tokeniser shows

			// If the symbol is small, check to see if it's really the name of
			// a register.

So if we add a small check to omit this step in the first line token it should be fixed. Stay tuned!

 

 

Thanks to everyone for the feedback so far!

FYI, @ggn, this is what I ran into and reported here http://jlhconsulting.gotdns.com/bugs/show_bug.cgi?id=159, though I like your idea for a fix a lot better than the patch I proposed.

[ggn]

Hi,

31 minutes ago, 42bs said:

Or you may add support for an optional ":" after symbols to differ them from reserved words.

 

IIRC the ":" is handled as a separate token, so by the time the parser kicks in (which would probably be the appropriate place to check for this) the damage has already been done. i.e. the "x" is classified as a register. However, since we can pretty much exclude that a valid instruction starts with a register name I think it's simpler to just add that small check to the tokenizer.

 

9 minutes ago, cubanismo said:

FYI, @ggn, this is what I ran into and reported here http://jlhconsulting.gotdns.com/bugs/show_bug.cgi?id=159, though I like your idea for a fix a lot better than the patch I proposed.

Yeah I was wondering who submitted bug reports/fixes to the tracker and wasn't me :). Thanks very much for taking the time to do all this work. I'll check the reports you added in detail after work and will comment on them :).

 

Oh, and welcome to rmac!

Edited May 8, 2020 by ggn

[42bs]

5 minutes ago, ggn said:

Hi,

 

IIRC the ":" is handled as a separate token, so by the time the parser kicks in (which would probably be the appropriate place to check for this) the damage has already been done. i.e. the "x" is classified as a register. However, since we can pretty much exclude that a valid instruction starts with a register name I think it's simpler to just add that small check to the tokenizer.

 

Yeah I was wondering who submitted bug reports/fixes to the tracker and wasn't me :). Thanks very much for taking the time to do all this work. I'll check the reports you added in detail after work and will comment on them :).

 

Oh, and welcome to rmac!

Most assemblers I have worked with do this:

- First column => symbol

- not first column but with a ":" => symbol

- starts with a "." => local symbol

 

Edit: And an instruction may never start in the first column.

Edited May 8, 2020 by 42bs

[+DrTypo]

5 hours ago, 42bs said:

? If you ever programmed PowerPC, it is default there. Which is a constant source for error:

add 1,2,3 => r1 = r2 + r3

addi 1,2,3 => r1 = r2 + 3

 

So a small typo is a big difference. So luckily the RISC mnemonics use "#" for immediate values.

Now this is evil  

I had a PowerPC machine a long time ago but I didn't do much assembly with it, mostly C++.

 

@ggn Thank you for looking into this. Actually I did use the VS debugger . But the various arrays (kwtab, kwaccept, kwcheck) confuse me.

 

[ggn]

30 minutes ago, DrTypo said:

Now this is evil  

I had a PowerPC machine a long time ago but I didn't do much assembly with it, mostly C++.

 

@ggn Thank you for looking into this. Actually I did use the VS debugger . But the various arrays (kwtab, kwaccept, kwcheck) confuse me.

 

Oh, that's mr Dyer's magic hash tables :). Those are generated using the supporting build tools like kwgen, 68kgen etc. and I too haven't looked too deep in them. My mind just goes "oh, that's clever hash stuff, let's breakpoint past that" :D.

[Shamus]

Those "magic hash tables" are really DFAs (Deterministic Finite Automatons).  Ever since I learned about them in my uni days, I've always thought they were the closest things in all of CompSci that you could label as "magic". 

[LinkoVitch]

Why am I only just learning about these now! did a quick google and they look AWESOME fun! we never did them on my Comp Sci course ? (or if we did I was too busy not attending or asleep).  Cheers for the info shamoo

[ggn]

On 5/7/2020 at 7:56 PM, DrTypo said:

It seems I've found a bug in the way rmac handles registers. It looks like rmac doesn't care about the "r" in the register name.

"r15" and "15" are the same.

For example, rmac doesn't complain in the following cases:

add 7,1   => assembled to add r7,r1

store 5,(2) => assembled to store r5,(r2)

addq #r31,3 => assembled to addq #31,r3

movei #r8,2 => assembled to movei #8,r2

 

I'm looking at the rmac source code but this whole tokenizer/parser thing is a bit black magic for me

Sorry it took that long - sometimes things get too hectic!

 

Anyway, the risc parser has been hardened against these kinds of instructions. A patch has been submitted and if Shamus agrees with what I did then it'd be added to the main tree soon .

 

(You can of course download the patch file and try it out yourself if you compile rmac on your own!)

[+DrTypo]

2 hours ago, ggn said:

Sorry it took that long - sometimes things get too hectic!

 

Anyway, the risc parser has been hardened against these kinds of instructions. A patch has been submitted and if Shamus agrees with what I did then it'd be added to the main tree soon .

 

(You can of course download the patch file and try it out yourself if you compile rmac on your own!)

Thank you for the correction!

 

[dilinger]

Hi,
I think I've found an issue with RLN V1.6.1.
If the options related to symbols are not used, the PSymbolTable and NumberOfSymbols fields in the coff will still be set.
It will create a problem if the fields are used by a tool or a debugger, because no symbols will be found in the file.
Could it be possible to fix this problem?
If the source code is available on Git, please let me know, I will try to have a look at the issue.

Edited July 29, 2020 by dilinger