Video tools and player for the TI-99/4A

[Eric Lafortune]

I've released a set of tools to create and play optimized videos:

• Command-line tools to prepare and merge animations, music, and speech in a video.
• A video player to play the result on the TI-99/4A.

Some features:

• Animation: Import still frames or zip files with sequences of frames in common image formats (GIF, PNG, PBM, BMP), at a full 256x192 pixels in all 15 colors, at 25 or 30 fps.
• Music: Extract the sound commands from VGM files. Cut sound files.
• Speech: Import binary LPC files, from the speech dictionary, BlueWizard, python_wizard, and Praat. Cut speech files. Tweak the files in a more convenient text format. Tune vocals based on synchronized music.
• Resulting video: Merge any number of the above media at freely chosen time frames in a single optimized video.
• Video player: Assemble the player as an application on a cartridge with bank-switched ROM (up to 32 MB).

You can find the tools and documentation as an open source project on Github. The tools are written in Java. The player is written in TMS9900 assembly.

 

The project sprouted from creating my Bad Apple demo, also a project on Github. Its build file provides an elaborate example applying the tools, but you can also simply create a cartridge to play a silent animation, or just a chiptune, or just some speech.

 

The video player is optimized to efficiently update the differences between subsequent frames, so it works best on stylized animations such as Bad Apple. If you want to play general video with sampled sound, you should look at @Tursi's amazing TI VidConvert.

[Nick99]

This is really nice, thanks!

A question comes to my mind:

How difficult would it be to make the animation tool to use the full capacity of the V9938 VDP?

It would be really exciting to have tools like this for the Geneve or for the ones that have an 80-column card to the TI-99.

[+9640News]

38 minutes ago, Nick99 said:

This is really nice, thanks!

A question comes to my mind:

How difficult would it be to make the animation tool to use the full capacity of the V9938 VDP?

It would be really exciting to have tools like this for the Geneve or for the ones that have an 80-column card to the TI-99.

I agree it would be nice.  I went to Github and looked the code over to see if there was any 9938 support but did not see anything.

[Eric Lafortune]

4 minutes ago, Nick99 said:

How difficult would it be to make the animation tool to use the full capacity of the V9938 VDP?

The video format and the video player essentially stream data bytes to the video processor (and wait for a vsync at specified points). So you'd only need to change setting up the VDP registers and clearing some VDP memory in the player.

 

The main change would be the compression code. Right now it analyzes subsequent video frames to find minimal changes in the screen image table, the pattern definition table, and the color definition table. It then encodes these changes as a series of fragments with a VDP address, a byte count, and the consecutive bytes that can be streamed. It works well with shadow-art animations like Bad Apple because there are many completely black or white characters of 8x8 pixels, for example. You can then reuse a single black resp. white character, and write it as a single byte. As an added benefit, you can get away without double-buffering if the timing works out well enough. In the bitmap modes of the V9938, you don't have the indirection of the screen image table; it's just a table of pixels. That's a lot simpler and straightforward to handle, but also less efficient, because you can't reuse patterns. Higher resolutions and more colors increase the required bandwidth further. For Bad Apple, it's a pity there's no compact monochrome mode.

 

In short: technically quite easy, but I'm not sure the performance would be good enough.

[Asmusr]

I would be very interested in seeing an example with color, if you have any. 

[Asmusr]

If you don't have any ready examples with color, maybe this could be used?

cube.zip

[Eric Lafortune]

1 hour ago, Asmusr said:

If you don't have any ready examples with color, maybe this could be used?

cube.zip 183.51 kB · 2 downloads

My Bad Apple demo has a small postcredits section in color, but it's intentionally simple. For the cube animation, I first cleaned up the zip file so the frames are in the right numeric order. Then: create the video, assemble the player, and create an RPK file:

java -cp tools/out/videotools.jar ComposeVideo cube_clean.zip player/data/video.tms
xas99.py --register-symbols --binary --output romc.bin player/src/player.asm
zip --junk-paths cube.rpk player/layout.xml romc.bin

The video has some artefacts: some due to color clashes of the bitmap mode, and probably some due to not meeting the vsync (avg 550 bytes/vsync, max 1456 bytes/vsync). Color is almost twice as hard as monochrome...

cube.rpk

Edited August 3, 2022 by Eric Lafortune

[Tursi]

That looks really good! Nice and smooth!

 

[Tursi]

I attempted to abuse the toolchain for a video that I knew was too much, but I wanted to see how it coped. Unfortunately, it failed.

 

First, I might ask if you publish the jar as well as source - it was a right pain to build on my system which is not normally set up for java.  I started with the Windows version which was working great, but hand-typing the find results was painful. So I tried to use the WSL Ubuntu I already had, but the java version there was too old. Ultimately I compiled on Windows, and build the jar in Linux. I will give Java credit for that.

 

Anyway, my massive, complex, 6000 frame video failed:

 

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 256 out of bounds for length 256
        at display.DisplayFrameOptimizer.optimizeSection(DisplayFrameOptimizer.java:159)
        at display.DisplayFrameOptimizer.optimizeDisplay(DisplayFrameOptimizer.java:42)
        at display.DisplayDeltaOutputStream.writeDisplayDelta1(DisplayDeltaOutputStream.java:63)
        at ComposeVideo.main(ComposeVideo.java:229)

 

 

 

[Eric Lafortune]

7 minutes ago, Tursi said:

I attempted to abuse the toolchain for a video that I knew was too much, but I wanted to see how it coped. Unfortunately, it failed.

Thanks! It looks like too many character definitions in a frame are different, and the display optimizer crashes on it. That was sloppy of me. I'll fix it in the coming days. This may become a rough flickering video, but it's an interesting experiment.

 

The compiled jar (for Java 14+) is one of the release binaries. Github's link to the releases is hard to spot on the main page, but it's also mentioned in the "Downloading" section of the README.

[+OLD CS1]

5 hours ago, Eric Lafortune said:

The video has some artefacts: some due to color clashes of the bitmap mode, and probably some due to not meeting the vsync (avg 550 bytes/vsync, max 1456 bytes/vsync). Color is almost twice as hard as monochrome...

Please excuse the outburst: holy shit.  I would love to see real-time rendering this fast, but this is no less amazing.

[Tursi]

4 hours ago, Eric Lafortune said:

Thanks! It looks like too many character definitions in a frame are different, and the display optimizer crashes on it. That was sloppy of me. I'll fix it in the coming days. This may become a rough flickering video, but it's an interesting experiment.

 

The compiled jar (for Java 14+) is one of the release binaries. Github's link to the releases is hard to spot on the main page, but it's also mentioned in the "Downloading" section of the README.

Ah, guess that's on me. I hate the Github 'releases' feature. Didn't even think to look! 

 

[Tursi]

For what it's worth, I hacked up the code to continue in the error case (or rather, to abort the loop). That gave me a finished video, but there are clearly other artifacts involved with running out of characters. Still, much more worked than I expected to.

 

Then I hacked up a version of the code that started the diff search from the middle segment instead of the top one, worked from the center outwards, and honored a bandwidth maximum to keep the frame rate up. That sort of worked, but it still had a lot of glitchyness (just focused in the center now instead of across the image). I wasn't able to get this limited version to keep the previous background on the characters it changed though, which I was hoping for. I was pretty much guessing at a lot of it, so there's no question I either introduced new bugs or encouraged old ones to show up.  

 

Still, lots of potential in this diff processing approach.

Anyway, I'm sure you already know how you want to fix it, but it was fun to play with. Look forward to seeing the update!

 

[Eric Lafortune]

I've now released version 2.0 of the Video Tools. This version adds:

• A brand new tool ConvertWavToLpc to convert audio files to speech files, based on an accurate simulation of the TMS5200/5220 speech processors. I'll introduce it in a separate thread.
• A tool ConvertLpcToWav to convert LPC speech files to audio files, based on the same simulation of the speech processors.
• A bug fix and enhancements for processing animations with large differences between frames. The performance of such converted animations also degrades more gracefully.

As always, you can find the tools, documentation, and source code on Github. Notably, you can find compiled binaries (for Java 14) in the release section.