Phantasia (Atari 7800 ARPG)

[+Bruce-Robert Pocock]

3 minutes ago, RevEng said:

Nice dev info! The crash detection info is very comprehensive.

 

mksmith started a thread on compression schemes a while back. There were some great insights by several members - especially @Eagle who tested various schemes and provided comparisons. Well worth reading, if you haven't yet.

That's great, I'll be reading that in more detail soon. I started an implementation based on DEFLATE but ran into just garbage output, with the added “fun” that I can't really tell if it's the compressor or decompressor at fault, but I'm sure something better than RLE will be the final goal. Y'all's collective efforts w.r.t. LZ4 look quite promising there!

 

That particular pain point I'm pushing off until the initial demo map is working, though, as running out of ROM for the demo should hopefully not be A Thing.

[RevEng]

Yeah, it would be good if you don't need to reinvent the wheel. Unless you're a wheel enthusiast, in which case disregard my note.

 

We landed on LZ4 because it's a pretty good middle-ground result, in both packing and performance. In my case, I also needed the encoder to be open and cross platform, which helped with the decision.

[+Bruce-Robert Pocock]

1 minute ago, RevEng said:

it would be good if you don't need to reinvent the wheel

I always say, most folks who re-invent the wheel end up with an octagon.

 

 

[+Pac-Lander]

On 11/4/2022 at 5:27 AM, CommodoreCEO said:

Brilliant work so far and thanks for the detailed explanations!

 

This is exactly the sort of game the Atari 7800 needs.

I could not agree more!!! 🤘🤘🤘

[+Bruce-Robert Pocock]

We are happy to say that @Pac-Lander has graciously agreed to join our team (with myself and Zephyr Salz) to assist with writing, level design, &c.

 

We're hoping to have a little proof-of-concept demo around the new year time frame; not much more than a test level that you can explore, really, but something in relatively playable shape.

[grownup]

More adventure games is always good

[+Bruce-Robert Pocock]

Just wanted to share that @TIX has kindly joined up with us and contributed already some new art for the player-character

 

 

Here's a rough take of him running around in the game engine with a shield equipped

 

As a general update, I have what I think is a race condition happening (for you programmer types out there) between DLI-NMIs and the main thread (which is using a primitive cooperative multi-tasking mechanism) that causes display list corruption and lockups. The goal is still to get a playable walking tour of one area together around the new year, with those problems solved.

[+Bruce-Robert Pocock]

A progress (or lack thereof) update.

 

So — the project is not abandoned (far from it), just had to be tabled as other things came to the forefront of my "non work" hours.

 

Since 12/2 there's been a couple of family events, in addition to Channukah, Christmas, New Year's, and now the rather involved process of moving from Florida to Oregon tying us up, so there have not been any major updates. To be honest, I haven't even been able to keep up with AtariAge nor the ZPH shows, much less get time to troubleshoot race conditions and display list corruption, unfortunately for Phantasia's progress. (For what it's worth, we've bought a house in Portland and are going to be there full-time in just a few weeks, so there's packing and getting the Fort Lauderdale house ready to sell and … and … and … all happening at once.)

 

All that being said, it'll most likely be March before my “around the new year” prediction of a demo build is even plausible. It will happen! — eventually.

 

Sorry to disappoint anyone who might be waiting for this, but "A delayed game is eventually good, a rushed game is forever bad." (Miyamoto)

[matthughson]

This looks fantastic, and thank you for sharing the information about your tools! I love seeing behind the scenes like this. I'm using Tiled for my own NES projects, and find that combining it with some Python scripts, to massage the data into a useable form, is a great combo!

[+Bruce-Robert Pocock]

OK, I am deeply ashamed, but we still haven't made more progress. We've moved 2,600 miles and been doing a rather comprehensive overhaul on the new house — floors, painting, fixtures, appliances, siding, windows, doors — which has been a big time drain.

 

Just posting to say, “yes, it's still A Thing.” Thanks for your patience.

[MajorSetback]

How's the move and the game development coming along? Been lurking this thread since the beginning, it looks awesome so far.

[+Bruce-Robert Pocock]

Oh, my, I'd missed your question @MajorSetback

 

Progress is being made, but the move / home renovation / madness and chaös in general has taken its toll.

 

Aside: We fixed up the fixer-upper. Work was done on electrical, foundation, sewer line, new insulation, new siding, some roofing, new paint inside and out, new doors, and mostly new windows, as well as all the floors that aren't the kitchen or bathrooms. It was a lot.

 

My current "way point" goal is to get the entire opening "cut scene" sequence working, and then move on to the game play systems. I'll definitely be sharing that milestone as soon as it's been reached.

 

Some of the things that are either working, or mostly working include …

• Mixed-mode 320A/C text and graphics for the status area (and subscreen) and dialogue sections, and mixed-mode 160A/B graphics for the game play (map) section
• Multiple graphical tile sets with distinct palettes for different maps/areas.
• Map compilation from Tiled supports things like: decal graphics over the top of the basic tiles, placement of named entrance points that can be used by scripts or exits from other screens, and attaching cut scenes to visiting points on the map.
• Script compilation from Fountain format supports dialogue and simple camera and stage directions to manipulate characters. The stage directions language allows statements like Aisling walks to "Some Place". Exit Aisling. and outputs assembly-language routines.
• At least 12 or 14 completely unique, hand-crafted NPC graphics, but also over 20 distinct shapes and hundreds of distinct color combinations for general NPCs constructed as "color by number" characters
• A multi-threaded operating model that distinguishes between the main thread, non-maskable interrupt handler (NMI-DLI), running compiled scripts, and a service routine module that is isolated from the main game program and handles low-level details like reading the controllers (which I've nicknamed Stagehand).
• English text is translated for dialogue purposes into both text to be displayed word-wrapped in dialogue boxes, and also into SpeakJet phonemes for AtariVox, using a slightly more sophisticated system but based on the method used for converting dialogue for Grizzards.
• Since the player can select their gender, there is a variable-substitution system for text to handle proper pronoun selection. The same system should also handle explaining the controls to you based on the controller you have connected; e.g. on a ProLine-type controller, you'll see something like “press (LB|” where someone using a Joy2b+ would see “press (i)” and someone using a Mega7800 would see “press (A)”. Likewise, a Genesis controller without a Mega7800 would show “press (B)” and a SNES2Atari user would see “press (A)” also.
• MIDI music is partly converted into HOKEY/POKEY-compatible form, although this bit is extraordinarily buggy still and the playback routine is not yet in place.

The rough story outline is over 400 lines, and is being slowly broken down into maps and scripts. We're anticipating around 150-200 maps of multiple screens each, compressed using zx7mini compression. We don't really have an estimate on the amount of dialogue/cut scenes, but it's looking to be big enough that, post-milestone 1, I may look into going to compiling to a tokenized bytecode form (like Java, or Microsoft/Commodore PETBASIC) just to reduce the footprint of the script segments versus spelling everything out in assembly-language. We also have several musical pieces in the works, for specific locations and characters.

 

There are currently 90 bugs or enhancement tickets assigned to me on our project board, of which 27 are bugs blocking the milestone 1 introduction sequence.

 

Overall, there are about 30-40,000 lines of code in the game runtime in assembly (over 300 files), about 15,000 lines of code in the conversion tool in Common Lisp (about 20 files), and about an additional 1,000 lines of scripts in Fountain format (about 30 files).

 

Not sure what is interesting to share particularly, so perhaps the above is a dump of Too Much Information

 

[MajorSetback]

I feel your pain. My wife and I bought our first house two years ago. It too needed work. Insulation in the attic, new windows, hardwood floors refinished, built a workshop/shed, new wood fence all around, and the hits keep on coming. I couldn't imaging coding and troubleshooting a game during all of that. I barely know basic, so what you're doing sounds like a Herculean effort to me.

 

One question. Are you looking to take advantage of the new 7800 Game Drive's YM2151, COVOX, or Bugchip sound or Mega7800 six button genesis controller capability? The reason I'm asking is because it looks like the 7800 Game Drive is going to be well established due to it's availability and capability.

Edited December 5, 2023 by MajorSetback

[+Bruce-Robert Pocock]

We're aiming for an eventual cartridge release, so limiting the sound to HOKEY (POKEY), but I do aim to finish integrating Mega7800 controller support. It should work with…

• 7800/ProLine/EuroPad and compatible controllers (LB, RB)
• Genesis/MD controllers, directly connected (B, C)
• Joy2b+ controllers (i, ii, iii)
• SNES2Atari-connected controllers (A, B, X, Y, L, R, Start, Select)
• Genesis/MD three-button controllers, via Mega7800 (A, B, C, Start)
• Genesis/MD six-button controllers, via Mega7800 (A, B, C, X, Y, Z, Start)

The input code is mostly in place, but hardly tested beyond the 7800/ProLine and Joy2b+ so far.

[+Bruce-Robert Pocock]

As Phantasia creeps towards functionality, I thought maybe I'd start posting some quick blog posts about what progress is being made.

 

If anyone has questions about what something particularly means, feel free to ask, but I may not answer things that would be too spoiler-y in public.

 

This past week:

• Fixed serious scheduler problems caused by poor background tiles animation performance that actually hurt the frame rate a tonne.
• Major cleanups around the multi-threading support to ensure steady frame rate under most/all circumstances
• When a decal leaves the scene heading north, and there are more rows of the map off the top of the screen, sometimes other decals in the top zone would be overwritten with the art pointers of the sprite that had just left momentarily after it passed.
• Added some spacing above/below the dialogue lines. (Still needs some color fixes, as they're solid "white")
• corrected X positioning of decals
• moved some address constants so as to better sync things between banks
• re-sync DLI with regards to MSTAT so that we don't miss one interrupt and have the screen "permanently" jammed up (e.g. text font where the map tiles belong & vice-versa)
• correct handling of finding human heads — when putting together NPC's
• fixed bug: humans' heads face east when walking south
• Broke up ComposeCharacter into a number of smaller files for manageability; current line counts …

  ⇒ wc -l Source/Routines/ComposeCharacter.s Source/Routines/ComposeCharacter/*.s
    129 Source/Routines/ComposeCharacter.s
    145 Source/Routines/ComposeCharacter/ColorizeByte.s
     56 Source/Routines/ComposeCharacter/DecalKindSearch.s
     83 Source/Routines/ComposeCharacter/DecorateFrame.s
     31 Source/Routines/ComposeCharacter/FindAction.s
     60 Source/Routines/ComposeCharacter/FindAnimationFrame.s
     32 Source/Routines/ComposeCharacter/FindFrameRateScalar.s
    101 Source/Routines/ComposeCharacter/FlipBuffer.s
     52 Source/Routines/ComposeCharacter/GetFacings.s
     43 Source/Routines/ComposeCharacter/PaintVarColors.s
     45 Source/Routines/ComposeCharacter/PrepareWorkVars.s
     41 Source/Routines/ComposeCharacter/StencilOverUnder.s
     48 Source/Routines/ComposeCharacter/UpdateDecalArt.s
    866 total
  

• Adjusted handling of personal pronouns for the player (gendered words) — so e.g. he/him/his/&c appear with correct lengths and spacing.
• Peephole optimizations using the "undocumented" lax instruction
• Early exit could occur when a one-byte "word" was next-to-last and a one-byte "word" ended a paragraph/utterance. Any word starting on the last byte of an utterance would be omitted. This was visible when the player's name (a one-byte variable code representing a word) was followed by ending punctuation (essentially a one-byte "word" also)
• Fixed alignment of, and spacing around, the dialogue speaker's name
• First draft of a new scene for the initial demo video sequence
• Fixed handling of pauses in speech (for SpeakJet): better combining adjacent pauses, and proper elimination of trailing pauses
• Fixed handling of a final pause before an exclamation mark or question mark, e.g. ...? at the end of a sentence
• Tonnes of new SpeakJet phonetic spellings added to support the ever-growing vocabulary of the game's dialogue
  

  Reading SpeakJet.dic file …  Done, 2,402 words

   

 

Overall, about 78% of the way to the First Light Demo (which is the cut-scenes-only one), with about 30 bugs & enhancements still in the queue for it.

 

Currently:

• I'm working on the script compiler and getting the engine sufficiently stable to play through the entire demo cut scene sequence properly, and trying not to distract myself with shiny things
• @Zephyr Salz is working on some more art, and scripts, for the demo sequence

[+Karl G]

I'm quite curious what you mean by "multi-threading support" in the context of a single 8-bit processor.  🙂 

[+Bruce-Robert Pocock]

@Karl G

 

"Thread" is being used a bit vaingloriously, but the intent is pretty much the same as one might expect from 1980s multiprogramming.

 

There are 4 threads of control; at different times they either voluntarily hand off control between one another, or are pre-empted by NMI (DLI). Each has its own stack region (all within the $c0 bytes of stack available on page 1) and during context switches we swap the S pointer.

 

The four threads are

• the main thread,
• the NMI thread, which takes control during display list interrupts and may pass it off elsewhere,
• the Stagehand thread, which does not access any bankable ROM but does read the controls, update sounds, and cycle background tile animations;
• and the Fountain thread, which runs compiled (from Fountain-formatted source files) scripts for NPC interactions, cut scenes, &c.

 

For example, a cut scene script could wait for a certain number of seconds to elapse while simply spinning in place reading the clock value, and the NMI interrupts would be simultaneously driving the Stagehand thread, which updates the clock.

 

Granted, this is basically just an elaboration on the usual 6502 interrupt service routine model, but it comes into play particularly because Sally and Maria bank switch at the same time. Since each screen is made up of graphics that may be in one of several ROM banks, we have to stay pinned in the appropriate bank while the display is being rendered. Stagehand is able to run and do some useful things in that context, but the main thread may want to do bank switching, and a running cut scene will be found in an arbitrary "data" ROM bank itself, so those things cannot happen while we're "locked in" for graphics, so they must compete for time slices during VBLANK.

 

By keeping separate stacks for each one, it's a lot easier to reason about what's happening in that block of code.

 

As an additional precaution, Stagehand is compiled separately with its own variables in RAM, so its symbols &c aren't accessible (except the communications "mailbox" region) from the main game, and likewise, nothing that the main game is using is accessible to Stagehand, so they are isolated in that way as well. I use a very tiny jump table at the top of Stagehand for its "on boot" and "on frame advance" entry points, and otherwise it is an opaque binary blob to the main game.

[+Bruce-Robert Pocock]

It's a bit difficult to "catch it" doing anything interesting, but you can get a little idea of how it runs from this, perhaps.

[+Bruce-Robert Pocock]

Yesterday's progress:

• Substantially improved how smoothly background tile animations can occur — which reveals that a number of things were being asked to animate at silly-fast speeds, and only slowed down to reasonable frame rates because the animation process was not running fast enough to meet the demands. (Still manually entering the animation sequence data for background tiles; I need to work on accepting the speeds specified in Tiled.)
• Began work on animation and "physics" for particles
• Improved coöperative multithreading between scripts and main game thread so that particle updates aren't frozen while scripts run (it would look super dumb)
• Improved auto-generated labels in compiled scripts, rather than just garbage numbers, they now contain descriptive names, and dialogue/speech symbols now contain the first few words of the dialogue text to make the source code easier to read. The garbage at the end is a hash to ensure distinctive uniqueness, so e.g. two utterances with the same prelude don't get confused for one another, but two identical utterances can be merged into one if they occur in the same ROM bank.
  
• Ensured that a countdown timer used to wait for a moment in scripts is clamped to zero, so it won't ever wrap around and make you wait another 255 frames before it realizes it's done (race condition)
• Improved the Interworldly Adventuring, LLC logo screen graphics
• Better handling of different classes of boats/ships and re-organized how their graphics are arranged in ROM.
• Palette tweaks to ensure that characters stand out clearly against the background in some scenes
• Based on some info by @RevEng on his web site, taking advantage of the unreliability of accesses to the RIOT timer in order to increase the entropy of the pseudo-random number generator.
• Changed the thread labels around in the crash screen, so they now identify as M/ain, S/cript, Stage/H/and, and N/MI

Edited April 11 by Bruce-Robert Pocock
link to RevEng site

[+Bruce-Robert Pocock]

Yesterday's progress:

• Uniform naming of scripting variables for my convenience
• Tweaking background tile animation frame rate
• Skeletal support for music, sound effects, and speech
• Rough implementation of speech (AtariVox) based upon Grizzards code. (Also, discovered a few bytes of dead code in the Grizzards implementation, but harmless.)
• Set construction (map editing in Tiled, tileset editing in Gimp)
• Initial stub support for weather system
• Yet more vocabulary added to SpeakJet dictionary
• Convenience feature for Skyline Tool to reload a map when a named nav point (from a script) can't be found (so you can edit the map and resume the build process)
• Increased the stack for scripts at the expense of the main stack. (Stack sizes are now: main $40, script $40, NMI $20, and Stagehand $20.)
• Initial support for animating full-screen graphics (e.g. title card)
• "Frame earthquake" cured (vertical jitter)
• Removed some unused consts and vars
• More reasonable timeouts for prelude screens before title
• Correct loop for setting palettes for full-screen graphics (ensure that all palette entries are initialized)
• Tweak to 7800GD "live push" support. (Ensure that the newly-pushed image will detect that it needs to reload)
• Tidied up arrangement of fixed ROM bank
• Sketched out some (more) ideas for the manual
• Re-arrange text on title card to support (eventual) main startup menu
• Remove duplicated efforts to compose characters (i.e. people and such) from the main loop unnecessarily
• Don't count down alarm timers when the game is paused
• Miscellaneous small tweaks