Phantasia (Atari 7800 ARPG)

[+Bruce-Robert Pocock]

• Big merge of some feature branches for improved multi-threading (despite that race condition bug that persists) and beginnings of inventory subscreens
• tidying up player data references
• fix up UI drawing bits in graphics ROM and made a few changes to take advantage of the new layout
• added some item art and alternate art for items in different states
• Game Select button, or similar buttons on some controllers, should toggle opening the subscreen
• impossible-to-exist items now display as a particular item rather than garbage
• enhancement to allow Quest Item names to be specified
• broke the fix from last week for the upper status area without the player — unclear why, re-opened that ticket…
• macro for fixed-length text, but not using it any more
• tidy and enumerate items, identifying the graphics drawing needed for the subscreen
• labels on some anonymous branch targets (+/-) for added clarity
• general documentation in the code
• fix for DetectSaveKey
• .Mul macro now has optimized case for ×20, but not using it any more
• Tidy up front buffer/back buffer for display lists a bit
• Palette re-arrangement to make item selections nicer
• detect SaveKey/MemCard/AtariVox device
• tidy up label generation in source code generated from scripts
• remove duplicate WSYNC in NMI/DLI
• remove some unused/unloved/unwanted junk files
• ensure that CurrentBank is stable before actually changing the bank
• warn and correct if we are not in the script bank when we should be
• signal an error if ScriptYield detects we're in the wrong bank
• æsthetic improvements when reading aloud to AtariVox (on PC)
• updates to the map on the Hero Ship

unfortunately, the big bad bug — a race condition in which we try to return from a subroutine and discover that the memory bank selected has been changed from what we wanted — remains in play.

[+Bruce-Robert Pocock]

Well, this was a weird week's work, including doing a very convincing 6502 implementation of a Dijkstra concurrent locking algorithm before tossing it away for a simple spin lock, but the crazy annoying bug is gone, so is the concept of an “NMI thread,” and the problem turned out to be a race condition in which one thread would try to change its thread context (e.g. yield up control to another thread) while we were in the critical section of that same thread-context-switching on another thread — and putting a semaphore around it was just exactly what the doctor ordered. The NMI thread had to go, though, because it could deadlock, and (as it turns out) we don't want to do too much in the NMI itself, all the fun stuff happens in the other threads (main, script, and stagehand).

 

So, this week's updates…

 

In Skyline-Tool …

• Cleaned up some things in Skyline-Tool, including compiler warnings and missing functions, that sort of thing.
• Enhanced support for “best fit” or “imperfect” mode for converting full-screen/large bitmaps from PNG to 7800 — the --imperfect option can be passed to allow it to choose the best palette possible from among the 8 palettes you've provided for each 4×16 px cell.
• Fixed a bug where SpeakJet phrases were not terminated properly
• When reading from a core dump file, decode the DLL for the screen currently being displayed (front buffer) not composed (back buffer) by default
• Decode character states (ActorAction) for all defined actions, including “Panic” — this enables you to use the core analysis of the decals in the scene to find a character which is panicking because of e.g. a minor fault STUK signal — that is: When an actor can't navigate the scene, their action changes to “Panic” and they log a minor fault STUK, by watching for minor faults in the debugger, now the designer can find out which actor is in Panic mode and trace what course they weren't able to navigate.

In the game runtime:

• Shaved a couple of bytes off DebugBreaks sometimes
• Implemented simple semaphore locking around critical section of thread context switching
• Added labels for the main service routines at $8000
• Fixed the (mis)use of the NonTilesetBank value that was causing issues
• Removed some code that I'd marked as “suspicious” ages ago, and found that it actually did nothing at all
• Wait only for VBlank to start, not to end, before certain actions
• Log a minor fault if we ignored an NMI
• Whenever a minor fault occurs, flash the stats section of the screen's background to red
• Zero out DLL immediately upon starting to compose a new frame
• Fixed a few front-buffer/back-buffer confusions in the code
• Fixed some bad pointer arithmetic in a couple of places
• Corrected the make ready target, which would fail if your user account did not already have a ~/.local/share/fonts/ folder; and made a “fix-up” routine to detect machines that I'd mucked up and correct them. (The font it was trying to install ended up as ~/.local/share/fonts, as a file, not a folder, confusing everything.)
• And (and I thought I'd posted this previously, but I guess not?) — ensured that the stack is reset thoroughly when we're drawing the Break screen, as failure to do so could cause the break screen to, itself, crash.

I'm going to cauterize some other references to the NMI thread, tonight, and perhaps work also on a bug that's causing the upper stats area to be ridiculously oversized; then, back to dealing with particle systems and weather.

[+Bruce-Robert Pocock]

With the race condition sorted, I have finally been able to start cleaning up the NMI procedures themselves, and abolishing the concept of an NMI thread was definitely the right thing to do.

 

Skyline-Tool:

• Remove old NMI thread references
• Actually call FadeBrightness for fade in/out to/from black/white
• Fix for setting weather (missing a # in generated assembly)
• Set up A7800 watchpoint traps for accesses to garbage memory to ignore harmless accesses caused by using BIT to skip instructions

Phantasia:

• Removed NMI stack and other materials related to the NMI thread
• Fixed top-of-screen spacing
• Better way to determine if the player is present in the scene
• Implemented stack canaries to detect stack overflows
• Cure “too much” background color bleeding out of speech balloon's top
• Reduced use of generic/shared variables further
• Identified some unused vars to reclaim some RAM
• Improved reading/responsiveness for user controls
• Initialize all fields for the player on new game
• Cured a bug where the player's gender was … animated? … I'm sure this was a debug thing, but the number of animation frames was being written to the address currently used for player's gender, which must have been something else in the past — and it's really, really unusual for me to hard-code a specific address into the code like that
• Fixed a tense-of-verb error in the test script for pronominals (he is vs he was e.g.)
• SubWordInto macro is basically an impl of x ← a - b to complement SubWord which is a ← a - b (where a, b, and x are 16-bit words)
• Use a constant for the BIOS address / upper threshold of built-in RAM
• Started populating animation sequences for some of the NPC characters whose frames were already drawn
• trimmed the number of rows of the map visible on the screen at a time, such that it doesn't just roll off the bottom of the screen. (There will now be a small black margin at the bottom of the screen.)
• optimized the code for plotting dialogue lines a bit (We have to word-wrap on the fly, because sentences can contain variables, e.g. the player's name, that can expand to varying lengths)
• ensured that the Stagehand routine ("OS") does not pass control to a routine in a different memory bank while Maria is drawing from one bank.

At the same time, @Zephyr Salz's working on some maps and script changes

 

This week-end, or next week, we're expecting to

• tidy up some background color issues related to the DLI/scheduling fixes
• finish up a few scripts for opening section cut scenes
• actually map out the animation sequences for all the NPCs that have been drawn already (assign frames to sequences and sequences to character + action), including “generic humans” and main characters (who have specific, bespoke art frames)
• implement functions for a few new stage directions
• fix up the weather system to actually work (for rain, at least)

[+Bruce-Robert Pocock]

Ah, what a long week-end — ! We did make a good deal of progress, though.

• fix for centering the camera on the right point in some circumstances
• ensure that DLIs occur the same before, during, and after stats updates
• pinhole optimizations
• wait out last row (of map tiles) better
• animation sequences for generic humans in robes
• fixed palette register assignments for speech balloons
• actually seed random numbers properly
• rainfall working (part of the effect works, at least, there are some more details to come)
• made dialogue balloon slightly narrower
• moved scrolling code to banked ROM to free up wired (last bank) space
• began support for palettes shifts for fade in/out &c
• sorted out palettes across most all of the artwork to be in sync with each region
• fixed stats updates (including place name display for cut scenes)
• Tidy up display list interrupts around narration text

Also a few bug fixes in Skyline-Tool, mostly related to the palette shuffling details.

 

We still are working on some scripts and scripting commands even, and a few NPC sequences may not be assigned yet, but we've made great strides in both.

[+Bruce-Robert Pocock]

Week-end update …

 

Phantasia runtime improvements:

• Navigation for characters is now via a Course object, of which there is currently only a very naïve implementation, but we have plans to implement a couple of alternative classes for various character types in future. This was actually a pretty big lift and worked out some parts of the Object-Oriented system that had not been exercised before, so we fixed some edge cases there.
• New lighting features (partially implemented). We're taking the palette for each tile set, and using relatively high-quality color code running on the PC at compile-time, pre-calculating various tinted versions (dark, light, &c) of each color. These alternate palettes are available immediately for use via scripts; e.g. a script can say “It is night.” and the scene will be dimmed slightly with a small loss of saturation.
• Work in progress is to take advantage of the CoLu colorspace of the 7800's native palette to enable fades between the normal palette, the tinted (e.g. dark) palette, and the extreme solid colors (e.g. “Fade to black.” will fade from the normal palette, to the dark palette, on its way to fading to black entirely.) There are some weird bugs in this area still being resolved, and I probably won't be working on them today. Bugs include it flickering like candlelight as it argues with itself whether to make the scene 1 unit lighter or darker every few frames, or just plain turning the scene weirdly-mis-colored…

Night

Just a mess

• Added a scripting command to bring up the title screen
• Added minimal implementation of Game Over — you lose/you win. (Very minimal)
• Tidied up some DLI's related to narration
• Ensure that scripts yield to the main thread when they're not busy
• Added a bunch of arbitrary words to the SpeakJet dictionary, which now clocks in at 2.434 words in an American accent (For referenece, a fourth-grade vocabulary is around 4,000 words, but we have any number of imaginary or unusual words and names in there, so that isn't quite a fair comparison.)

Skyline-Tool improvements:

• Fixed some UI type things: clearer output for errors and reporting of successes
• The UI now decodes Course objects
• Upgraded to the latest McCLIM toolkit with improved TrueType font support for nicer output, but incidentally all the font sizes are (for now) slightly larger. This also fixed an annoying bug where small gaps appeared in some "fat bits" type pixel displays in the GUI. It did create a bug where some of the table displays look a little “off” from the GUI, but they look fine in a terminal (see the “Instance of Character” section in the screenshots for an example, or the palette explanation below the “Show Decal” example.)
• 
• Tweaked the YACC grammar for stage directions to avoid some complaints it was presenting.
• Began to implement a “prepare scene” stanza for scripts to avoid “pop in“ of characters. Essentially, the author will write something like “We open on: … ” in the script, and the system will suppress page flips (between the back buffer and front buffer display lists) until the author has enumerated what the scene looks like, e.g. where any characters might be standing.
• The names of the 12 “fixed” colors (four upper palettes that are available from any tile set) are now taken from Project.json, so they can be renamed for other projects' use in future. (This is part of the slow walk out of making Skyline Tool more generally useful to others.)
• Various bug fixes and tweaks.
• Fixed the make install target (Linux®-only) to add the icon to the Activities Overview applications library — somewhere along the line, the icon got lost, although it is very Phantasia-specific right now and we'll want to improve upon that.

 

 

 

[+Bruce-Robert Pocock]

Another week, a little more progress.

 

Skyline-Tool:

• Force-apply lighting (palette) changes when needed immediately
• Commands for waiting for the scene to be ready before the first page flip
• Support for multiple bodies within a block of NPC graphics, including generic humans

Runtime:

• Runtime support for lighting changes (fixes)
• Deferred page flip support in the runtime as well
• Decoding new animation sequence "body" discriminator — animation sequences are now based on a Kind, a Body number, the Facing direction, and the Action, with fallbacks.
• Tidied up some unnecessary code
• Lightning test
• New text for the "Interworldly" author card
• Began working on the "real" title card to replace the current placeholder

 

And, what's this?

[+Bruce-Robert Pocock]

This week:

• Fix up body matching rules
• Move boats from the main loop
• Shipboard maps and detailed tile graphics for shipboard
• Sleep and death frames for NPCs
• Player sleep frames conform to NPC style
• Mess hall / galley of the "hero ship"
• Unused tiles marked / freed up in shipboard tile set
• NPC stats updates
• Relabel some boards with "final" names, e.g. "the open seas" and "ANV Sentiment"
• NPC art updates for special (main) characters
• Tidy up some memory layouts on startup
• Renamed "Phantom Soldiers" as "Sentinels"
• Skeletal data structures for Jump Point A* navigation (Harabor/Grastien)
• Sweat/strain glyph
• Numerous script edits
• Better frame flip when needed
• Prepare/ready block — stop front/back buffer flipping during critical sections
• Handle boats more nicely
• Fixes to waiting for an Actor to "Settle" (quiesce)
• Adjust camera placement
• fixed ExitCharacter
• Missing # on a literal 0 used in bankswitching! (happened to work as the value at $0000 = $00 usually)
• Ensure Y = 0 reset when building display list list
• Title Card replacement background graphics piece
• Sync up detection of dialogue on screen with updates to DLL and DLI count
• Fix up sky color in title card
• Show the name of the detected controller on the title card — and advise the player to press pause to re-attempt controller detection. Detection is flaky for most controller types, for now, but at least you'll know if it's doing it wrong. The button indicated to start the game is also affected, but you can't e.g. tell if you were detected as one of the thee kinds of Sega gamepads or the SNES if the game says press (b) to start, so the full name will "rock around" eventually.

in Skyline-Tool:

• Support for asynchronous boat movements
• Use Ultralisp (software repository)
• Fix output alignment of some tables in the debugger GUI
• Yield when boats move
• Wait for fade out
• Make guardrails optional for A7800, as they seem to cause the emulator to crash far more often when enabled
• Add decode all objects to the GUI menu

 

What's that, you say? You want to see the new title card? All in good time … but here's a little preview …

 

Edited June 17 by Bruce-Robert Pocock
image broke, replaced

[+Bruce-Robert Pocock]

Controller testing update:

✓ ProLine (& compatible)   detected             both triggers work
✓ Joy2b+ controllers       detected             buttons i, ii, iii work
~ Sega (Genesis) directly  detected as Joy2b+   buttons B, C work¹
✗ Sega 3b via Mega7800     detected as ProLine  does not work
✗ Sega 6b via Mega7800     detected as ProLine  does not work
~ SNES via SNES2Atari      detected             partially working, but buggy²
~ One-button controller    detected as Joy2b+   Fire works³

^1 — Genesis/MegaDrive 3/6 button controllers work natively, but are mis-detected as Joy2b+, so the system labels the buttons as (i) and (ii) rather than (B) and (C), and there is no (iii) button as it would expect.

^2 — SNES controllers are reading some buttons, but not others; (B) at least is not working.

^3 — One-button controllers are not actually allowed and I mean to complain about one if detected, but it currently claims to be a Joy2b+ where only button (i) works.

 

Not my highest priority, right now, to ensure that these are all working, but it's definitely on the roadmap overall. Note that I intentionally safeguard by making you press the second button (RB, ii, C, B, whatever) to get past the title card, to ensure that you have a working controller with at least the necessary two buttons.

 

(more updates on weekly progress coming later)

[+Bruce-Robert Pocock]

This week in Aclypt …

• New title screen art "phase one" completed
• Show controller type on title screen (previously mentioned)
• Re-detect controller when Pause is pressed (on title screen, or anywhere else)
• New animation of a special introductory screen for ZPH
• Switched to using MuseScore files as the canonical file format to check in to the repository (although we convert them to MIDI before using them, much like how we canonically store Gimp XCF files but convert them to PNG before using them)
• Aliases for equipping shields from scripts
• Corrections to credits
• Screenplay work
• Corrected text on title screen
• Implemented Hurt and Kill methods for Characters, so all Characters (including Player, NPCs, and Enemies) can take damage and die. (More work needed to properly handle things like armor)
• Special Corpse class, to ensure that NPCs, once killed, properly remain dead.
• Fixes to some maps for the intro demo
• Ambient lighting for rain, particularly at night
• Animation of sails on sailboats when they move
• Additional tile art
• Improvements to controller auto-detection
• Build chain changes to omit bank $3e for 7800GD's purposes
• Added graphics for emphasizing picking up/putting down items
• Implemented Exit method for all Actors (including Characters)
• Fixed a crash when we asked a Boat to Think (which is not valid)
• Fixes for some timing issues with the intro full-screen art scenes
• Initial implementation of the Main Menu support, supporting starting a new game (or at least the intro demo sequence) and credits, but with an unconnected continue option displayed for the future

Currently working on a branch with many of the title screen changes to eliminate some truly obnoxious frame flashing happening on the title screen.

[+Karl G]

11 hours ago, Bruce-Robert Pocock said:

Fixed a crash when we asked a Boat to Think (which is not valid)

You can lead a boat to water, but you can't make it think! 

[+Bruce-Robert Pocock]

Well, it's been a bit of a week.

• Lots of script work for the reveal trailer
• Tidied up the Break (crash) screen, including revealing additional details: We now display the code of the last minor fault that has occurred, along with the counter of the total number of minor faults (if any) that have occurred, next to the main code for the break. (This shows a break code sthd and the last minor fault was nfac, and it was the only minor fault, as there have been only $0001 since startup.) Also updated the email address (and abbreviated "bank" and "thread" to make room) and fixed a bug where the stack dump would not show the last 2 bytes of the stack ($01fe and $01ff) sometimes (which you actually see in this screenshot, from before that fix).
  
  
• Switched the NMI vector to be into a jump table. Since the high byte of all NMI routines is now the same, there's no race conditions possible around updating it, which simplifies a few things.
• Lots of other work around race conditions with the NMI (DLI) routines right around when dialogue is being added to, or removed from, the display. A new system for indicating whether dialogue is intended to be displayed or is displayed on the front or back buffers is mostly staving off weird NMI-out-of-sync race conditions that was causing (similar to the other NMI problem, but slightly different)
• Lots of vocabulary work for SpeakJet … broke 2,500 words in the dictionary (with very few being made-up words)
  
• Title screen tweaks
• Sails on boats now adapt to reflect being full or slack, tied to the boat's scripted movement
• Build (through automation, from the sources) a class hierarchy diagram upon demand to which developers & designers can refer
  
  
• Cleaned up some unused code
• Basic Main Menu functionality for the title screen, but with continue basically non-functional (and start skips right into the opening cutscene, since "roll your character" is not yet implemented)
  
• Basic framework for SaveKey/HSC detection and save game slot "in use" to be detected (not actually functional yet)
• Registered HSC codes (for one save game) and reserved SaveKey slots (for two save games)
• Actually calculate exact ROM usage of assets and asset loader code, to pack things into ROM asset banks as tightly as possible.
• Improve instrument selection code for Hokey music (be more tolerant of weird stuff in MIDI files)
• Provide a Restart to render Hokey music using "piano" (type 10) if the MIDI instrument is not known or not supported — although the compiler there is still very badly broken
• Fixed some build chain issues around bank $3e, and also around music compilation
• Only regenerate PNG files individually if their own respective XCF was changed (previously, we regenerated all PNGs from all XCFs if any were changed)
• Expanded SpeakJet.dic syntax to allow sending numeric parameters, e.g. \Bend \012 to send the Bend command with 12 as a parameter
• Explicitly add character info to any memory bank that might need it, but only those and no more. (This was previously kinda ad hoc and sometimes did not work right.)
• Ensure that all functionality available through the Skyline-Tool GUI is also available in Slime or a Terminal.
• NPCs can now "pick up" an object from the scene.

[+Karl G]

But how long until we get to see the first demo? 

[+Bruce-Robert Pocock]

5 minutes ago, Karl G said:

But how long until we get to see the first demo? 

We're working about daily towards a goal of a demo trailer that we'll be sharing via @ZeroPage Homebrew "soonish." That's our first real milestone goal, and we've packed into it a tonne of infrastructure and special effects, so it's actually foreshadowing a lot of the technology for the full game. (As you might imagine, being as we're two years in!) It's essentially a series of cut scenes from early in the game and a bit of the tutorial area content without player control.

 

That "First Light" demo is mostly waiting on stabilizing the interrupt routines as close to 100% as practical and will most likely be silent, or at best, mostly silent.

 

My back of envelope goal is to wrap that up by the end of July, at the latest.

 

The following goal is to have "Chapter I" working, which will be the initial scenes (some of which will be spoiled slightly by the intro trailer) all the way up to entering the first mini-"dungeon," and (possibly) might include that first dungeon as well. That's the first "real" demo (alpha build) and I'm really hoping to have it in decent shape by the start of September.

 

PRGE got scheduled this year on the same week-end as our wedding, so despite our inability to attend, if that demo is ready by early September I'm hopeful that Albert will put a demo up there, but we'll release it here (forum) as well, naturally. That does mean throughout September we're gonna be focused on big family wedding stuff and taking a break from Atari dev.

 

Subsequently, things start to speed up. All other things being equal, there will be decreasing amounts of 6502 code to write and increasing amounts of maps to build and dialogue to write, which should increase our pace a great deal. Right now, every feature is something new — for example, the intro needs a specific character emote, next, and there is currently no system for emotes, so adding the first one may take a whole evening's dev time, but once it's working, it might only take 15 minutes to add another. We also have been slogging through timing-related crashers, and I'm hopeful that there are behind us. Post alpha, the biggest code needs will probably be for boss fights …

[+ZeroPage Homebrew]

On 6/28/2024 at 11:24 AM, Bruce-Robert Pocock said:

We're working about daily towards a goal of a demo trailer that we'll be sharing via @ZeroPage Homebrew "soonish."

 

Looking forward to it!

 

- James

[Maztr_0n]

this is really neat

 

[+Bruce-Robert Pocock]

Latest updates …

• NPCs can now have "emote" icons appear over their heads (and so forth)
• Combined and re-organized data to simplify the map loading routines, and also achieve much more favorable compression rates on the per-map data. Now, all of the map data is written out (for NTSC or PAL, differences due to timing of animations in frame units) as a singular binary and compressed in a single block, yielding around 45-91% reductions in ROM footprints — the current ROM budget usage went from 56% to 45% with no actual changes to the data. (Don't get too excited about that 56% — that's no indication of the completeness of the game; a good portion of that space is filled with placeholder maps, most of which are basically empty rooms.)
• Refactored the way that "sprites" are added to the display lists to allow Z-ordering, and automatically write "short" (4-byte, versus "extended" 5-byte) headers into the display list whenever possible. Due to alternation between 160A and 160B sprites being possible, we may actually have to write more extended headers than before, depending on the Z ordering.
• Cured crashes due to the DLI getting out of sync with the dialogue section split-screen code
• Animations for background tiles are now derived directly from the TSX (Tiled sprite sheet) files; previously, I had been hand-transcribing the tile sequences into the source code. Animations for scenery-type sprites are being parsed, but not copied into the runtime, yet.

Been a busy week in the real world, so that's actually about all that's moved forward.

[+Bruce-Robert Pocock]

Since last update … we've been working on scripts for the unveiling of the game, and miscellaneous bug fixes. In addition, the inventory items are mostly completed and the routines for drawing the stats area and inventory subscreen were reworked and fine-tuned, although a few odd bugs remain. This included rewriting the inventory items' drawing routines to accommodate drawing segments from different parts of the sprite sheet, which allowed us to pack more visually distinct states in, such as indicating whether you have arrows remaining by changing the bow graphic (as well as for other items).

——

A special thanks to @vhzc who assisted with the redesign of the hook and rope item

 

[+Bruce-Robert Pocock]

Our first mini-milestone is complete, and we've packaged up a short video clip which shows off some of what's working, and also (warts and all) shows off some of the drawing glitches that my latest code introduced. The video clip also features (at the end) an MP3 version of one of the original songs written for the game.

 

 

There are some unhappy interactions between the double-buffering drawing, the sprite/display list updating code, and the weather system adding rain drops which cause intermittent garbage draws and weird flicker on the bottom halves of boats, specifically.

 

New features since I last posted

• scripting command improvements around placing boats at sea
• player can no longer walk around freely during cutscenes
• "sprites" entering from off-camera were not being added to the display list when they reached visible areas
• player's decal-ID was not reset when clearing the scene, causing corruption
• characters now (palette) flash when injured
• dialogue was not always dismissing when button i is pressed
• missing margins around narration boxes
• various script improvements
• additional item graphics improvements

[+Bruce-Robert Pocock]

The "Extremely Early Edition" trailer revealed on @ZeroPage Homebrew tonight is now at https://PhantasiaGame.com/

 

Yes, glitches and all 😓

[+Bruce-Robert Pocock]

Wedding planning has derailed this a bit — we will not have a demo at PRGE, but in a couple of weeks we'll resume work in earnest. (They decided to reschedule PRGE earlier to co-incide with our wedding, which has been problematic.)