swapd0 Posted August 8, 2020 Share Posted August 8, 2020 (edited) I've done a full compile for BurgerTime and my engine library and It looks that there are something wrong. I'm using gcc and rmac, I've a some .c and .s files for the gcc boot code (brownboot.s, zerocrtfini.c, zerolib.cpp, etc) I've included into this "library" the skunk.s, also I've my c/asm library common to all my games, and then I've the current game that I'm developing. With the latest rmac version I've the following errors when I compile BurgerTime. /opt/local/bin/gcc68k/lib/libcxx-jaguar/zerocrtfini.o: In function `_exit': zerocrtfini.c:(.text.exit+0x0): multiple definition of `_exit' /opt/local/bin/gcc68k/lib/libcxx-jaguar/tinyalloc.o: In function `_ta_init': tinyalloc.c:(.text.ta_init+0x0): multiple definition of `_ta_init' /opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o: In function `_skunkRESET': (.text+0x0): multiple definition of `_skunkRESET' /opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o: In function `_skunkNOP': (.text+0x3a): multiple definition of `_skunkNOP' /opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o: In function `_skunkCONSOLEWRITE': (.text+0x6e): multiple definition of `_skunkCONSOLEWRITE' obj/main.o: In function `_main': main.c:(.text.startup.main+0x0): multiple definition of `_main' /Users/Shared/apps/Jaguar/projects/lib/libkatu.a(debug_asm.o): In function `_skunkCONSOLEWRITE': (*ABS*+0x0): multiple definition of `_skunkFILEOPEN' /opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o:(.text+0x170): first defined here /Users/Shared/apps/Jaguar/projects/lib/libkatu.a(debug_asm.o): In function `_skunkCONSOLEWRITE': (*ABS*+0x0): multiple definition of `_skunkFILECLOSE' /opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o:(.text+0x2a0): first defined here /Users/Shared/apps/Jaguar/projects/lib/libkatu.a(debug_asm.o): In function `_skunkCONSOLEWRITE': (*ABS*+0x0): multiple definition of `_skunkFILEWRITE' /opt/local/bin/gcc68k/lib/libcxx-jaguar/skunk.o:(.text+0x1ca): first defined here /Users/Shared/apps/Jaguar/projects/lib/libkatu.a(gpu_asm.o): In function `DRAM': (*ABS*+0x0): multiple definition of `_gpu_events_list' /Users/Shared/apps/Jaguar/projects/lib/libkatu.a(sprite.o):(.bss._op_start+0x0): multiple definition of `__op_start' Ok, let's have a look at _skunkRESET, it's defined in skunk.s like this. ; skunkRESET() ; Resets the library, waits for the PC, and marks the console up or down _skunkRESET:: movem.l a1-a2,-(sp) move.w #$4001,_skunkReadMode ; ensure normal 4MB mode set continueRESET: move.l #-1,_skunkConsoleUp ; optimistic! bsr setAddresses ; get HPI addresses into a1 & a2 ; try and get both buffers, that tells us the console is up bsr getBothBuffers ; also sets skunkConsoleUp bsr restoreMode ; set correct flash mode movem.l (sp)+,a1-a2 ; Restore regs rts And it's referenced in brownboot.s with a .extern _skunkRESET and somewhere in the same file I've a jsr _skunkRESET, it's not called from my engine library, neither from my game, and it's only defined in skunk.s... so why I've a "multiple definition" error? edit: After some testing looks like the symbol must be referenced from at least two different files to get the error of multiple definitions, and must be defined in a third file. nm dump for skunk.o and brownboot.o swapd0$ nm skunk.o 000000b0 T _skunkCONSOLECLOSE 000000f6 T _skunkCONSOLEREAD 0000006e T _skunkCONSOLEWRITE 000004a4 t _skunkConsoleUp 000002a0 T _skunkFILECLOSE 00000170 T _skunkFILEOPEN 00000214 T _skunkFILEREAD 000001ca T _skunkFILEWRITE 000002dc T _skunkMEMREAD 00000356 T _skunkMEMWRITE 0000003a T _skunkNOP 00000000 T _skunkRESET 00000028 T _skunkRESET6MB 000004a4 t _skunkReadMode 000004a4 t _skunkTimeout 000003dc t checkBuffer1 000003e8 t checkBuffer2 000003ba t checkbuffer 0000000c t continueRESET 00000436 t getBothBuffers 000003f4 t getBuffer 000003b2 t restoreMode 000003a0 t setAddresses 0000047e t waitforbufferack swapd0$ nm brownboot.o 00f02200 a A1_BASE 00f02208 a A1_CLIP ... 00000152 T __exit 00000000 A _exit 000004c8 T _init_debugger 000004c2 T _jaguar_reset 00000000 A _main 00000154 T _memcpy 000002e2 T _memmove 00000272 T _memset 00000466 T _putchar 0000047c T _rand 00000000 A _skunkCONSOLEWRITE 00000000 A _skunkNOP 00000000 A _skunkRESET 0000049e T _srand If I use the old rmac I get an 'U' for _skunkRESET instead of an 'A' Edited August 8, 2020 by swapd0 Quote Link to comment Share on other sites More sharing options...
ggn Posted August 9, 2020 Share Posted August 9, 2020 Ok, so what probably happens now is that rmac optimistically exports all global symbols as Absolute. Reading your post (thanks for bearing with this btw) I think the problem isn't in skunk.s but brownboot.s, or rather the way symbols are exported in brownboot.s. Last time I asked you to remove these two lines in object.c: else if (globflag) st_shndx = 0; // Global, not absolute This time I ask of you to replace them with else if (globflag && !(w1 & DEFINED) && (w1 & REFERENCED)) { st_shndx = SHN_UNDEF; } This will hopefully fix all your "multiple definition" issues (unless there's some other case I'm not considering right now, which is possible!). So give it a go and let us know what happens. Thanks Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 9, 2020 Share Posted August 9, 2020 Now it compiles but the game doesn't start. Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 9, 2020 Share Posted August 9, 2020 With the older version it runs, at the start I print this through the skunk board, with the new version it hangs after the Starting console... message. I'll have a look later at the symbol table. Starting console... init memory alloc at 00032c00 find_file_system at 00802000 Found at 0080209c init_file_system at 0080209c with 26 files NTSC Jaguar Quote Link to comment Share on other sites More sharing options...
ggn Posted August 9, 2020 Share Posted August 9, 2020 If it's only rmac that's different between the 2 builds you could even binary compare the two different output files. Also, producing a map file ("-Xlinker -Map=output.map" for gcc) when linking might also shed some clues. Or, even better, binary diffing the object files produced by the 2 rmacs would help tremendously. Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 10, 2020 Share Posted August 10, 2020 I've only changed the rmac version, same compiler options. old rmac output size is 309,872 bytes new rmac output size is 290,760 bytes There are a lot of labels that are shifted. I don't understand this, old rmac 00f0323c A _GPU_MAX_VIDEO_X 00f03240 A _GPU_MAX_VIDEO_Y 00f03234 a _GPU_SPRITE_OFFSETS 00f03234 A _GPU_SPRITE_OFFSET_X 00f03238 A _GPU_SPRITE_OFFSET_Y New rmac 00f03228 A _GPU_MAX_VIDEO_X 00f03228 A _GPU_MAX_VIDEO_Y 00f03228 a _GPU_SPRITE_OFFSETS 00f03228 A _GPU_SPRITE_OFFSET_X 00f03228 A _GPU_SPRITE_OFFSET_Y On the new version the labels are in a lower memory address and they are from an assembly file, so they are using the same instructions (same size). Also, why _GPU_SPRITE_OFFSET_Y are at the same address as _GPU_SPRITE_OFFSET_X? They are defined like this. .long _gpu_sprite_dst:: dc.l 0,0 ; destination address _GPU_SPRITE_OFFSETS: _GPU_SPRITE_OFFSET_X:: dc.l 0 _GPU_SPRITE_OFFSET_Y:: dc.l 0 _GPU_MAX_VIDEO_X:: dc.l 0 _GPU_MAX_VIDEO_Y:: dc.l 0 Any tool for the binary-diff? I've my own but the output it's something like this. out-new.bin and out-old.bin has different sizes. differs at offset 0x22 5 4f differs at offset 0x23 38 e0 differs at offset 0x46 8 53 differs at offset 0x47 c4 64 differs at offset 0x4a a6 eb differs at offset 0x4b ac 4c differs at offset 0x201f b8 bc differs at offset 0x2061 0 1 differs at offset 0x2062 0 e5 differs at offset 0x2063 0 48 differs at offset 0x213d d0 d4 differs at offset 0x2142 49 4b differs at offset 0x2143 50 20 differs at offset 0x2148 49 4b differs at offset 0x2149 8a 5a differs at offset 0x214e 73 7d differs at offset 0x214f 4e 2 differs at offset 0x2158 90 9a differs at offset 0x2159 62 16 differs at offset 0x2471 0 1 differs at offset 0x2472 0 e5 differs at offset 0x2473 0 50 differs at offset 0x2477 0 1 differs at offset 0x2478 0 e5 differs at offset 0x2479 0 50 differs at offset 0x247e 49 4b out-new.map.txt out-new.txt out-old.map.txt out-old.txt 1 Quote Link to comment Share on other sites More sharing options...
ggn Posted August 10, 2020 Share Posted August 10, 2020 Hi there, Thanks a lot for the files. From the little I can gather it seems like the linker is rejecting the data from the symbols. So my guess is that while it's importing the symbols correctly, it thinks that the data associated with the symbols is 0 in length, maybe that's the reason you see symbol addresses clashing. Also, since we get such a large difference in the final binaries, it makes little sense to binary compare them. Instead, it'd be really useful if you could binary diff the object files generated by the two rmacs. For example brownboot.o. At least there I more or less expect to see the same file sizes but some bytes will be different, most likely in the symbol table. So it'd be great if I could have such a diff. Also, as for a visual hex diff client you can try https://winmerge.org - just go to edit->options under "binary" and append the file extension you'll diff (most likely ".o"). Then drag the two object files and it'll do its job. Thanks for going through with this so far! Quote Link to comment Share on other sites More sharing options...
ggn Posted August 10, 2020 Share Posted August 10, 2020 I just realised that I actually have some of these files since they're from dml's tiny startup code - d'oh! So I tried assembling brownboot.s using a pre-2.0.0 version of rmac, as well as the latest with all the the above proposed fixed applied. Then I binary diffed them and something stood out: Seems like the name of the section names was turned to lower case at some point and nobody complained until now. So, current theory: the linker is anal about case sensitivity in section names, sees the label names, loads them into a different section, then sees that the sections aren't used, and then zaps them from orbit. So let's try something. Around line 561 of object.c there should be a line like shstSize += DepositELFSHSTEntry(&shstPtr, ".text"); and a similar line below for ".data" and ".bss". Change those to "TEXT', "DATA", and "BSS" (note: no leading dot "."). Recompile rmac and test things out. I have a good feeling about this Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 10, 2020 Share Posted August 10, 2020 Done, ".text" changed for "TEXT", and the same for data and bss Now the file size is sightly bigger (290,932 bytes) but still smaller than the one using old rmac. The labels are still clashing. Quote Link to comment Share on other sites More sharing options...
ggn Posted August 10, 2020 Share Posted August 10, 2020 11 minutes ago, swapd0 said: Done, ".text" changed for "TEXT", and the same for data and bss Now the file size is sightly bigger (290,932 bytes) but still smaller than the one using old rmac. The labels are still clashing. Are you able to diff a few object files now? Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 10, 2020 Share Posted August 10, 2020 They are quite different. I think that it's better if I send you several object files. In the gpu.o is where _GPU_SPRITE_OFFSET_X, _GPU_SPRITE_OFFSET_Y, _GPU_MAX_VIDEO_X are defined gpu-new.o gpu-old.o main-new.o main-old.o video-new.o video-old.o brownboot-new.o brownboot-old.o Quote Link to comment Share on other sites More sharing options...
ggn Posted August 10, 2020 Share Posted August 10, 2020 Without going any further I noticed some weird things. For starters, brownboot.o: That's 68000 code in there, and it seems that after that 4E75 (RTS) where everything starts mismatching very badly there is a "10a8 60c1 4e75" (which is move.b $6061(a0),a0) / rts) snippet inserted in the "old" version instead of just a "4ef9" which appears on the "new" version. Similarly, in gpu.o: A "0000 0000" is inserted at offset $78 which seems to derail the whole binary afterwards. The other two sets of .o files seem to be an identical match, but I'm puzzled about these two. It seems like they're assembled from a different version of the source codes. Is it possible that this is the case? Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 10, 2020 Share Posted August 10, 2020 20 minutes ago, ggn said: The other two sets of .o files seem to be an identical match, but I'm puzzled about these two. It seems like they're assembled from a different version of the source codes. Is it possible that this is the case? I have only a version of the source code, and I've made a make clean of each project. Anyway I've done this just to be sure. swapd0$ rmac-old -fe brownboot.s -o brownboot-old.o swapd0$ rmac-new -fe brownboot.s -o brownboot-new.o brownboot-new.o brownboot-old.o Quote Link to comment Share on other sites More sharing options...
ggn Posted August 10, 2020 Share Posted August 10, 2020 (edited) Sorry, but the invocation is wrong, it should be: swapd0$ rmac-old -fe -o brownboot-old.o brownboot.s swapd0$ rmac-new -fe -o brownboot-new.o brownboot.s (yes, it's a remnant from the old MadMac times, the idea behind this is that you can assemble multiple files from the command line that have different options you can switch on and off. So whenever rmac encounters an input file, it goes and assembles without parsing the rest of the arguments) (I still believe there's an issue since the register clashing is real fwiw) Edited August 10, 2020 by ggn Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 10, 2020 Share Posted August 10, 2020 34 minutes ago, ggn said: Similarly, in gpu.o: A "0000 0000" is inserted at offset $78 which seems to derail the whole binary afterwards. I'm 100% that the extra "0000 0000" are from a .long directive to align a data. It' this part of the code. movei #_gpu_events_list,list_ptr movei #.execute_list,r1 jump (r1) load (list_ptr),events .long _gpu_events_list:: dc.l 0 ; pointer to events list "D020" it's the jump (r1) and the "A672" it's the load (list_ptr),events, then two bytes with zeros for the alignment but in the new version it appends 4 extra bytes. Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 10, 2020 Share Posted August 10, 2020 (edited) 9 minutes ago, ggn said: Sorry, but the invocation is wrong, it should be: swapd0$ rmac-old -fe -o brownboot-old.o brownboot.s swapd0$ rmac-new -fe -o brownboot-new.o brownboot.s (yes, it's a remnant from the old MadMac times, the idea behind this is that you can assemble multiple files from the command line that have different options you can switch on and off. So whenever rmac encounters an input file, it goes and assembles without parsing the rest of the arguments) (I still believe there's an issue since the register clashing is real fwiw) Ok, I will fix all the makefiles Edit:It was correct in the makefiles... Edited August 10, 2020 by swapd0 Quote Link to comment Share on other sites More sharing options...
ggn Posted August 13, 2020 Share Posted August 13, 2020 Oh hi, I lost track of this thread because I missed the edit in the last post. So, my guess that now you should be able to make equal sized brownboot.o files but with some slight differences. Could you try that? Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 13, 2020 Share Posted August 13, 2020 I've downloaded the latest version 2.0.20 (sorry but I think that I was using 2.0.16), and I've done the modifications. Almost the same size. swapd0$ rmac-old -fe -o brownboot-old.o brownboot.s swapd0$ rmac-new -fe -o brownboot-new.o brownboot.s swapd0$ ls -l brownboot* -rw------- 1 swapd0 wheel 18548 13 ago 16:59 brownboot-new.o -rw------- 1 swapd0 wheel 18552 13 ago 16:58 brownboot-old.o brownboot-new.o brownboot-old.o Quote Link to comment Share on other sites More sharing options...
ggn Posted August 13, 2020 Share Posted August 13, 2020 Oh, I see. Well, v2.0.20 doesn't have the proposed fixes of this thread yet (because we're still investigating). So what you could do is to grab the latest snapshot of the source code (currently http://shamusworld.gotdns.org/cgi-bin/gitweb.cgi?p=rmac;a=tree;h=234f42879ceda29cd6a37a50ff287730f4ac632e;hb=HEAD - just click "snapshot") and then apply the things on this thread. This means posts #102 and #108. Then recompile and generate brownboot.o. I expect that with this you'll get equal sized files but with a few diffs. I hope we're getting to the bottom of this soon! Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 13, 2020 Share Posted August 13, 2020 Done, downloaded and patched, I think that I get the same result. brownboot-new.o brownboot-old.o Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 13, 2020 Share Posted August 13, 2020 By the way I get a compiler error with SHN_UNDEF but I've googled it and changed to 0. Quote Link to comment Share on other sites More sharing options...
ggn Posted August 13, 2020 Share Posted August 13, 2020 (edited) Hmm, that's odd The differences start yet again on the name of the segments - old has ".text", new has "TEXT" etc. Did you apply that fix? [EDIT] On second thoughts, could you attach brownboot.s? I'll take a look at it here and not have to bother you. Edited August 13, 2020 by ggn Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 13, 2020 Share Posted August 13, 2020 Yes, I changed .text to TEXT and the same for .data and .bss (I hope that I didn't understand in the wrong way) Maybe you can remove the include "debugger.s" and the jsr _init_debugger to fix the bug. brownboot.s debugger.s Quote Link to comment Share on other sites More sharing options...
ggn Posted August 15, 2020 Share Posted August 15, 2020 Okay, my computer gave me the day off yesterday as it needed to install some very important updates, but I'm back at the helm now. What I did just now was to try and match my brownboot.o with the brownboot-old.o in post #120. I assume that one is linking fine for you. Skipping swiftly past the fact that jaguar.inc was different than the stock one and had to be recreated, I'm now able to produce a 1:1 copy of brownboot-old.o. My version is built from head (http://shamusworld.gotdns.org/cgi-bin/gitweb.cgi?p=rmac;a=snapshot;h=821279dcb7b5d069d175767edcd413b833b0cae3;sf=tgz) with the following changes: object.c: line 235 replaced with: uint16_t st_shndx = SHN_ABS; // Assume absolute (equated) number lines 243 and 244 replaced with: else if (globflag && !(w1 & DEFINED) && (w1 & REFERENCED)) { st_shndx = SHN_UNDEF; } object.h: the following inserted in the file (around line 35 but shouldn't matter) // // ELF special section indices (field st_shndx) // Lifted from glibc (https://sourceware.org/git/?p=glibc.git;a=blob;f=elf/elf.h) // #define SHN_UNDEF 0 /* Undefined section */ #define SHN_ABS 0xFFF1 /* Associated symbol is absolute */ #define SHN_COMMON 0xFFF2 /* Associated symbol is common */ and that's it. If I want to match brownboot-new.o from post #120 I can do so by replacing line 561 in object.c with shstSize += DepositELFSHSTEntry(&shstPtr, "TEXT"); and "DATA", "BSS" in lines 570 and 579 respectively. Now, again, if brownboot-old.o links ok for you then this should also link as it should produce the same object file! 2 Quote Link to comment Share on other sites More sharing options...
swapd0 Posted August 15, 2020 Share Posted August 15, 2020 (edited) Ok, I've done all the changes except .text -> TEXT, .data -> DATA and .bss -> BSS. After a clean make both versions of the game are the same!!! Thanks a lot for all the work! Edited August 15, 2020 by swapd0 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.