Lynx 3D Experimenting

[VladR]

On 8/5/2019 at 3:43 PM, 42bs said:

VladR, if you like, I can run your code on real HW if it is an ".o" file, not a complete .lnx file as I cannot yet flash cards.
"real" Suzy can only write to the video buffer and I would wonder if Handy has such a grave bug and nobody detected it so far.

I absolutely don't believe this is a Handy bug. I don't claim I have an intimate knowledge of Suzy's behavior - far from it.

On 8/5/2019 at 3:54 PM, Stephen said:

I can also test on real HW (I have a "saint" SD card.  Obviously nothing I receive would ever be shared.

This is not a jag. The game will be released for free, anyway. So, why would I have any issue with sharing of those builds ? There's like Two and half Men in Lynx area.

 

You're taking things out of a context and deliberately "forgetting" that those creatures in jag area weren't interested in the builds themselves, they merely wanted something they could just thrash resulting in eventual thread closure.

 

On 8/5/2019 at 9:55 PM, karri said:

bs42 mentioned that Suzy does not see the rom. In order for the CPU to see the RAM you need to set some register bits to swap out the ROM and part of the registers.

You mean $FFF9. Yes, I played with it earlier. That was a separate issue, but due to the fact that at the time I was still using TGI for doublebuffering and clearscreen, that solution brought another set of interrupt conflicts.

 

But, now I have 2 framebuffers that don't collide with Suzy/Mike, so that is taken completely out of the equation (at the cost of slight RAM waste).

On 8/5/2019 at 10:49 PM, enthusi said:

I had similar issues. Are you certain that your spritedata is fully complete and SUZY doesnt i.e. read beyond your data and starts interpreting god-knows-what as sprite afterwards? 

Well, yeah - it's obvious, that occasionally something happens, and then it starts overwriting RAM - as I can see it in the Framebuffer contents. Question is - what is it and why doesn't it happen 100% of the time but only at certain xpos,ypos ? How does position at lower scanlines (but left of screen center) NOT result in glitch, but right of screen center, even though it's upper screen half, it does glitch ? Really makes no sense. And I have a pretty extensive experience with Blitter from Jag.

 

 

So, the only way to debug this remotely - I would need somebody who is willing to spend an afternoon with me going back and forth about a dozen times and trying the incremental builds till I narrow it down. And I don't mean one, or two builds, and not getting back to me a day or two later after I upload it, but up to a dozen builds (perhaps less, if I get lucky) and turnaround within 10 minutes (e.g Skype/IM).

 

 

Obviously, I disregarded such a sci-fi scenario.

 

Today I have finished the transition from TGI (doublebuffering+clearscreen) to my SW rasterizer in Assembler and fixed all glitches, so now I have a fully working 6502-based rasterizer.  It's pretty smooth under the emulator.

 

There's 21 days till deadline, I will loose about 5 days next week, so I got ~16 days to implement some very basic gameplay now. Hmmmm....

[VladR]

Forgot the screenshot with the sky+land shaded background:

[42bs]

VladR, if you would have just send an .o file which shows the bug, one could have tested it already and you would know if it is a Handy, Suzy or VladR-Bug.

 

I have to withdraw my offer leaving for vacation. Good luck

 

BTW: Stop comparing the Lynx with the Jaguar. They have _nothing_ in common.

Edited August 10, 2019 by 42bs

[VladR]

44 minutes ago, 42bs said:

VladR, if you would have just send an .o file which shows the bug, one could have tested it already and you would know if it is a Handy, Suzy or VladR-Bug.

Even in the highly unlikely scenario that it was actually a Handy bug - how would I continue development if I can't run the builds because they crash/glitch? Knowing that wouldn't help at all as I would be stuck.

 

I can't just revert to deploying builds on Lynx as there is no Lynx. I needed a solution that would cover all scenarios.

 

Which is exactly what I did. I don't care if my first game on Lynx doesn't have best performance available. Considering the framerate of other Lynx 3D games, it's quite alright.

[42bs]

4 hours ago, VladR said:

Even in the highly unlikely scenario that it was actually a Handy bug - how would I continue development if I can't run the builds because they crash/glitch? Knowing that wouldn't help at all as I would be stuck.

 

I can't just revert to deploying builds on Lynx as there is no Lynx. I needed a solution that would cover all scenarios.

Debugging starts with changing parameters. One parameter is emulator vs. real HW. But who am I to tell ...

[+karri]

Making inventions with a fixed timeline is hard.

 

My projects follow the guideline: first make it work, then make it fast.

 

Unfortunately I very seldom have the time or energy for making it fast...

 

I am interested in a superfast 16-bit 3D engine with a small footprint. Perhaps my best bet is to reverse engineer Battlezone 2000. It is fast enough and appears to be without too many bugs.

 

The crashes in Suzy are likely to be because of the end-of-scanline bug.

 * If the last bit on a line is in use you need to add a byte of zeroes
 * A sequence 00000 ends a scan line

I used to have these kind of bugs hitting me while developing the sp65 utility.

 

Also: never use packed mode for 1 pixel.

[VladR]

7 hours ago, karri said:

Making inventions with a fixed timeline is hard.

 

My projects follow the guideline: first make it work, then make it fast.

 

Unfortunately I very seldom have the time or energy for making it fast...

 

I am interested in a superfast 16-bit 3D engine with a small footprint. Perhaps my best bet is to reverse engineer Battlezone 2000. It is fast enough and appears to be without too many bugs.

 

The crashes in Suzy are likely to be because of the end-of-scanline bug.

 * If the last bit on a line is in use you need to add a byte of zeroes
 * A sequence 00000 ends a scan line

I used to have these kind of bugs hitting me while developing the sp65 utility.

 

Also: never use packed mode for 1 pixel.

I believe you mean discovering quirks of new platform, right ?  I don't think there's anything I invented during this coding spree:

 - 8-bit clip-space  : I already had that last year

 - Quad rasterizer : I developed it in C last ~November, then ported to 68000 Asm and then to GPU RISC Asm. So, I merely ported the algorithm to 6502 Asm.

 

Though, to be fair, some algorithm improvements were made:I did one major change to scanline traversal. I now have two versions: 16-bit Fixed-Point and 8-bit Bresenham.

It was, indeed, a discovery for me, how awesome and clean Bresenham looks. Its performance characteristics are horrendous on Jaguar, but are really good on 6502.What's more, there was an additional and unexpected benefit - I could remove 2 safety conditions (per scanline) and 3-pass case became unheard of - now great majority of quads are single or double pass.

 

The most notable change is that I ditched the math tables and did all math on Suzy - which as this thread has documented turned out to be a very, very bad idea given its performance characteristics. But, you live, you learn

 

I would advise against reverse engineering any other engine. Hell, even with commented source code it can be a mine field. You don't have the documentation on all hacks to the initial design, which will result in very costly debugging sessions. At which point you will likely find out it would have been faster to just do this from scratch by yourself anyway.

[VladR]

7 hours ago, karri said:

I am interested in a superfast 16-bit 3D engine with a small footprint.

That's curious. Is this for the space game you are working on or do you have some other game in mind ? I thought you already had an optimized 3D engine working, no ?

[+karri]

17 minutes ago, VladR said:

That's curious. Is this for the space game you are working on or do you have some other game in mind ? I thought you already had an optimized 3D engine working, no ?

For some strange reason the drawings get corrupted after a while. My guess is that some bugs or rounding errors creep in when you do multiple small transformations and things start flying around in totally unpredicted ways.

 

I tried to add steps to ensure that the transformation matrix is orthogonal but I never had the time to really dig into it.

[+karri]

I was also hoping to generate a small matrix library for cc65. But should it follow DirectX or OpenGL convention? I mean do you have the vectors horizontal or vertical?

[VladR]

1 minute ago, karri said:

For some strange reason the drawings get corrupted after a while. My guess is that some bugs or rounding errors creep in when you do multiple small transformations and things start flying around in totally unpredicted ways.

 

I tried to add steps to ensure that the transformation matrix is orthogonal but I never had the time to really dig into it.

And that's exactly why I chose to implement my own 6502 dev emulator from within Visual Studio so that I can debug these kind of issues within an hour or two. You notice some weird behavior, put a condition into inner loop to detect it (with a breakpoint) and boom - within 5 minutes you have a full debugger access to all variables, arrays and can step through the code and see what the hell is going on.

 

Debugging that via printing values on screen on the device - that's nasty and painful

 

[VladR]

1 minute ago, karri said:

I was also hoping to generate a small matrix library for cc65. But should it follow DirectX or OpenGL convention? I mean do you have the vectors horizontal or vertical?

Honestly, that doesn't matter. You get to implement, so you get to choose

 

People will bitch and moan on StackOverflow till DoomsDay, regardless of which one you choose. Hell, they'd bitch even if you gave them both options (just on principle)

 

I'm personally used to DirectX's way - despite having started on OpenGL, but that's just because I spent more time in DirectX 8.1 land, and later XNA (XBOX360 + WindowPhone7).

 

Choose whatever makes sense for you.

[+karri]

Does Vulcan follow either convention? Or is it just a GPU optimisation language where this is not relevant?

[VladR]

No idea. Modern APIs don't really interest me anymore.

 

The only way how to survive in that arena is to hire an army of 3D artists who will keep chiseling dents into eyebrows of fleas hiding in the shadow of the seventh strand of hair behind the left ear of a mouse that pops out of hero's backpack for a total of 3 seconds during whole game and I really need no part of that 

 

 

With the proliferation of Unity and Unreal, engine programmers have become obsolete, hence scores of games are being released these days. Way too many amazing games are simply shovelware now due to the sheer number of them...

[+karri]

I get the point. The current game I am playing is Cuphead. All graphics hand drawn with great music.

[VladR]

Are you willing to consider a cooperation on your game ? Not sure of your desired timeframe, but I might be talked into writing a 3D engine from scratch for the target gameplay handling all the corner cases of clipping, sub-pixel edges and a dozen other issues that routinely pop up in such endeavor. Then, you could just focus on the fun gameplay stuff and any issues that you'd encounter would be my turf to debug and fix.

 

But, I have a lot of experience with automated testing on jag. I routinely write unit tests that run through the levels, randomize input and expose all corner scenarios. I would write the same thing in my DevEmu in C++ first, to catch as many of them as possible before going for the final 6502 rewrite.
 

I am sure you know I currently work full-time on my Atari stuff (fully self-financed) and regularly take breaks from Jag coding for the 6502. This competition made me realize I might be able to finance those 6502 experiments with actual Lynx games as with a finished 3D engine and core game framework elements (HUD, Menus, Stat Screens, Audio, Input, GameState management, ...) creating a new Lynx 3D game shouldn't take more than a month, given how barren the 3D game landscape is in Lynx (there's less than 10 such games as far as I can see from YT vids, so I can literally pick any genre I want). I might be too optimistic here, but I believe I can sell 50-100 units of each such game, across eBay/forums/retro conventions.

 

After this compo, I'll dive deep into Jag, as I will have only about 6 weeks till the PRGE, but between PRGE and Jan-01, I might take a 6502 break and could entertain creating a space 6-DOF 3D engine that I would reuse for Star Raiders-style game. At that time I should have the Lynx with some deployment setup and hopefully capability to create carts.

 

Don't feel bad if you wanna do this completely alone - I can totally respect that. Just think about it.

[+karri]

My dilemma is that real 3D math sucks in games. Nobody wants to look at a small dot for 10 minutes and then it suddenly zooms past you in a second.

 

So my "3D" world needs to have constant speed.

 

If my feet are at line 101 (bottom of screen) and horizon is at line 0 (top of screen). I want my object to get closer at a constant speed of 1 line per tick. Like 0, 1, 2, 3... until it reaches me at 102 ticks.

 

But I want to have the 3D orientation vectors correct.

 

Another thing is that I do not want my view to rotate. The reason is that for the game I would like to use sprites instead of polygons. There may be some rough sprite rotations like 0, 45, 90 degrees for front, side, back views of an object. So a little more like this

 

than classic 3D flying.

 

My games are usually free to play but of course I cannot donate carts so there is some money changing hands. I believe that "Always winter, never Christmas" and "Shaken, not stirred" both sold around 60 carts which I consider a success. Mainly the sales were at events. A few guys found their way to my whitelynx shop also. But I have stayed away from ebay. Plus the shop is only open if I have packed games to send.

 

I will likely close the shop and grab with me everything I have for sale when I fly to PRGE.

 

I really get the idea of starting from C++. The C++ compilers are amazing in optimizing the code. Perhaps there is some close-enough CPU target to optimize for and then translate the asm file to 65SC02.

[VladR]

I feel you with the "sudden zoom". My current games solve it by design -e.g. the track is always at the lowest screen scanline thus greatly achieving the sense of speed.

 

That is obviously impossible in empty dark space. Moving stars don't really help with that.

 

I'm glad to hear you sold 60 carts. That's reassuring, actually.

 

My point with C++ was not optimizations but ultra fast prototyping. You whip up few classes, constructors automatically take care of initialization and other design patterns help with cross-component communication without accidentally introducing a hard to find bug.

 

Still, I am amazed at how many things I catch that would otherwise take days to debug. Only then it's time to port it to 6502.

[+karri]

I get the speed by space debris. Here is a small build of the intro. You can turn to see the debris move.

stardreamer.lnx

 

I could work with a coordinate system where the distance is between 0..101, x-axis between 0..431 and y-axis between 0..271.

 

So direction forward 0,0 backwards 432/2, 272/2.

 

Perhaps an optimized compilation could reduce some of the math to byte operands...

[VladR]

Thanks for the build. You got a lot of debris there so it does help with the sense of speed quite a lot.

 

Forget about compiler doing optimizing. Accessing unsigned char array via unsigned char index results in almost a page of code instead of LDA array,X. No wonder my early builds were so slow, till I found the linker option to produce the .S file.

 

You gotta get the high-level code working first, outside of an emulator environment, like in VS with full debugger, and then handcraft the ASM.

 

 

Now I'm confused. First you mentioned you want full 3D, but now you mentioned sprites instead of polygons ?

 

So, if I get it right, all you want from 3D space is the WorldSpace cube where all ships move around (e.g. full XYZ position and vectors), but the rendering would be just scaled sprites ?

[+karri]

My goals are not fixed. The problem with polygons is the low pixel count on the Lynx. It is hard to make a game with a lot of depth in Z direction. So I have lately been thinking of combining large graphics to create a 2D game with 3D movement.

 

It may be possible to use 3D polygons if all objects are close to the viewer. Further away you just get noisy pixels.

[VladR]

2 hours ago, karri said:

My goals are not fixed. The problem with polygons is the low pixel count on the Lynx. It is hard to make a game with a lot of depth in Z direction.

I did a lot of research an experimenting on my 8-bit clip-space around January, facing the exactly same issue. There are several contradicting paradigms working against each other:

- you have 256 values of Z, that's it - now how do you want to distribute it

- If you want a lot of depth, the perspective must be very steep - e.g. vanishing point - that sucks

- if the perspective isn't so steep, you then don't get a lot of Z-values near the camera, so the objects, even if you would achieve 60 fps, still "jump" 10-20 pixels from one frame to another - that sucks

 

I've spent a full week trying to come up with a scenario that would still give you enough precision near the camera, yet enough depth in the distance. This, of course, needs to be tweaked on a world-by-world basis.

 

On Lynx, I just went for 16-bit coordinate space, because I got fooled by Suzy math, which in turn (when I finally implemented it) turned out to be just a hoax and I wish I hadn't done that and just stayed with my tables. But, it's an experience.

 

So I have lately been thinking of combining large graphics to create a 2D game with 3D movement.

I would be remiss not to mention the genius design of Star Raiders 2: The Last Starfighter. There's a constant arcade action tension present. Batches of ships don't leave you to breathe for more than a second.

Contrast that with Star Raiders on Atari ST, where 99% of time you flow through empty space with an occasional asteroid resulting in an incredibly boring gameplay.

 

If one gives players freedom to roam the 3D empty space freely, there must be some action sequence that will make up for it (e.g. large battles). Because 3D, especially at 4 MHz is very taxing, technology directly dictates the gameplay - you can't just process dozen ships at human framerates. But it doesn't have to be like that, if you go for 3D/2D hybrid (much more work, because you have to debug 2 systems and seamlessly integrate them).

 

It may be possible to use 3D polygons if all objects are close to the viewer. Further away you just get noisy pixels.

Yep,  especially for an empty world (the space), where objects (spaceships) comprise less than 0.0001% of volume player goes through.

 

It's also incredibly wasteful (performance-wise), to do this for each frame, - when in the end - ship makes up 2 pixels on screen:

- 3D transform

- clipping

- sorting points

- initializing edges

- running scanline loop

 

I've done a detailed depth-distribution analysis in excel (with exact cycle costs of each pipeline stage vs on-screen pixel coverage) on exactly same scenario last year. Even on Jaguar, which has two 26.6 MHz RISCs + 13.3 MHz 68000, I wouldn't want to do that without some sort of LevelOfDetail scheme. And that's, like, 2-orders-of-magnitude performance difference to Lynx, given that you have 5 processors working in parallel:

- OP rendering the last framebuffer

- Blitter rendering the scanlines of current framebuffer

- GPU running the scanline loop and feeding Blitter

- DSP computing other stages of pipeline

- 68000 - handling all the other gameplay/engine code that didn't fit into tiny 4/8 KB of cache or those 2 RISCs

 

Basically, 95% of the gameplay time, ships are pre-rendered and only for those couple frames that they actually zoom past you, you switch to a full-blown 3D engine. You still compute their pathfinding and waypoints in full 3D, but that's where it ends and they're merely 2D bitmaps (at the cost of RAM).

 

[laoo]

22 hours ago, karri said:

I really get the idea of starting from C++. The C++ compilers are amazing in optimizing the code. Perhaps there is some close-enough CPU target to optimize for and then translate the asm file to 65SC02.

Sadly not. Modern compilers rely heavily on data on stack and stack relative addressing modes with register and/or constant offsets. There is no such thing on 65(SC)02. It's a pity that Lynx does not have WDC 65c802 core. There are such addressing modes there, namely LDA off,s and LDA (off,s),y and much more good things like 16-bit indexing.

 

[+karri]

This obviously sucks.

 

How about designing the movements as projected trajectories on a flat canvas?

I mean to tie the objects to fixed coordinates and just move them around convincing arcs? Perhaps creating typical flight paths as splines with spawn and exit points using splines. The size would be minimum at the end points and maximum half way.

 

For fights the spacecraft could follow a few pseudo-random attack patterns and the distance would be the only thing you can affect by your own steering.

 

This would also solve my desire for the flying paths :). I just let the trajectory width be thicker or smaller depending on your distance to the target. I could also draw the object with polygons to maintain the illusion of 3D.

 

OMG! This is so simple comparing to the 3D nightmare.

[Clint Thompson]

On 8/9/2019 at 10:02 PM, VladR said:

You're taking things out of a context and deliberately "forgetting" that those creatures in jag area weren't interested in the builds themselves, they merely wanted something they could just thrash resulting in eventual thread closure.

It goes without saying that there absolutely are others (Jag creatures) genuinely interested in checking out whatever builds you have in the Jag area and not solely to trash any WIP you post or intentionally find a way to get the thread closed (what good would that do anyway)

 

I think it would just be easier to announce that you don't intend to share most of these builds or sources publicly instead of avoiding/ignoring/sidestepping those willing to volunteer their time to test a file for you on real hardware, in this case the Lynx.

 

If anything, posting a current WIP build (or even video) to the Jag area would reinforce the idea that you're actually progressing and no, not just to trash talk you. When progress is moving at a glacial pace, you need thick skin.