Imvtogif

[intvnut]

Good news: I had some time on an airplane this evening, and figured out the issue. The audio subsystem was originally designed the way I had described it. Everything pushes audio to the mixer at 48000 samples per simulated second. If you're running rate control > 1.0, that translates to 48000 * rate_ctl samples per second. Because the audio device won't keep up, the mixer ends up dropping audio accordingly to match the skewed notion of real time.

 

At some point, I had gone up into the PSG emulation and changed the rules: If rate control > 1.0, the PSG would actually scale all of the pitches downward, and produce audio as if the CPU's clock sped up. The net result is that it's only producing the exact right number of samples, and there's nothing for the mixer to drop. Tones still sound "right", though, rather than sped up like a chipmunk. So, the audio pipeline leaving the PSG represents the sped-up rate, and only produces 48000 samples per second. The mixer then only drops samples if it absolutely needs to—because something stole too much CPU from jzIntv and it couldn't get audio out fast enough, and now it has to catch up.

 

The Intellivoice, however, didn't get updated alongside the PSG. It was still producing audio at normal rate, expecting something downstream to drop audio frames to make the rate control > 1.0 work. The audio pipeline leaving the Intellivoice wanted to produce 48000 * rate_ctl samples per second. And, that just led to all sorts of weirdness.

 

When I hooked in the AVI encoder, I had forgotten about the PSG "upgrades" and thought the audio pipeline coming into the mixer was still producing 48000 * rate_ctl samples per second. So what it did was drop (1 - 1/rate_ctl) of the audio frames coming in. That made the PSG stuff go way too fast, but the Intellivoice probably sounded OK.

 

What I've done now is apply the same "upgrade" to the Intellivoice. When rate_ctl > 1.0, the Intellivoice with drop blocks of samples it generates internally, so the audio pipeline leaving it is also at 48000 samples/second. It drops blocks of samples, so that voices still sound the right pitch. Everyone just speaks faster.

 

(Note: 48000 samples/second is the default sampling rate in jzIntv, but you can change it with the -a flag.)

 

In the AVI encoder, I taught it about the fact that the audio pipe coming in has its own time scaling relative to the video stream, and adjusted how all that works.

 

And finally, there were several places in the audio code that assumed NTSC; I've now plumbed through NTSC/PAL switching to all the places I spotted while I was in there.

 

With all these changes, I believe I have this fully fixed now. If you play jzintv with --rate-control=2, for example, and record an AVI, you'll get a 2x speed AVI with all the audio and video still in sync. The --rate-control flag sets the default --avirate value, but you can still override the AVI rate with --avirate=<#>.

 

Things might sound a little funky if you set --avirate to a smaller value than --rate-control, as the AVI encoder will repeat blocks of samples to make up for the samples it didn't get from the rest of the audio pipeline. But honestly, I'm not too concerned about that use case.

 

Here's a clip of WSMLB, with --rate-control=2, to show how it's all working now.

 

 

I'll post again once I have download links for all the platforms. Unfortunately, I left my Windows laptop in California, and I'm in Texas right now. However, I believe I left a Windows build environment set up on my wife's dual-boot box, and so I'll check it out tomorrow.

wsmlb_rate_2.zip

Edited June 18, 2016 by intvnut

[tacrec]

Great, thank you so much for all your hard work!

[tacrec]

Testing it now with Super Pro Football, with ratecontrol=2 and avirate=2, the whole thing crashes after around a minute while recording. Works fine without ratecontrol=2.

[intvnut]

Interesting. What (if anything) did jzIntv print out when it crashed?

 

I'm curious if the game crashed, or if jzIntv itself crashed. (The latter usually results in a dialog from your operating system saying "jzintv.exe needed to close" or some silly thing like that.)

Edited July 29, 2016 by intvnut

[tacrec]

jzintv itself crashed with a windows error box to close it.

[intvnut]

jzintv itself crashed with a windows error box to close it.

 

OK, that's really bad. You said, Super Pro Football, as in the INTV release? What settings did you use, if you don't mind me asking?

Edited July 29, 2016 by intvnut

[intvnut]

Hmm... I've let it run for several minutes now with computer players, avirate=2, ratecontrol=2, recording a large AVI. No crash. This is on my Mac.

 

I'll have to try again with a Windows box later.

 

EDIT: I let it get through halftime (with a 0 - 100 score; apparently one team was waaaay better than the other), and no crash on the Mac. 160M+ AVI file, though. LOL.

Edited July 29, 2016 by intvnut

[tacrec]

Here's my command line:

(path)\jzintv --avirate=2 --ratecontrol=2 --gfx-palette=(path)\colors_smsh.txt SPFOOTBL.BIN

[intvnut]

Here's my command line:

(path)\jzintv --avirate=2 --ratecontrol=2 --gfx-palette=(path)\colors_smsh.txt SPFOOTBL.BIN

 

I mean, in the game. Any settings other than "Computer Players"? I don't know if the skill level of the computer players should matter.

[tacrec]

Oh, I enter 2 and 9 for both players (computer and super-pro level)

[tacrec]

Any update on this?

[intvnut]

So far, I haven't been able to reproduce the crash. I'll try again. I do most of my work from a Mac, so testing on my Windows laptop is a bit... sporadic.

[tacrec]

I loaded DOS first, then ran jzintv, and got this info when it crashed:

Video overrun! Closing Avi. Something's null or flush out of range in deflate

Assertion Failed

File: avi/avi.c, line 839

Expression zstream->avail_in == 0

[intvnut]

I'm letting it run now in the background on my Windows laptop to see if I can reproduce it. So far, I haven't been able to, which is a little frustrating.

 

That particular error implies that a compressed frame was too large somehow, or that too much compressed video got queued up while waiting for audio.

[intvnut]

Whatever the issue was, I seemed to have fixed it. I did verify that the version on the website crashes on Windows, but my current development version does not.

 

I recorded an entire game of SP Football on the Windows laptop with the current dev version. Here it is on YouTube.

 

 

I should have an updated binary on the website in a day or so.

[intvnut]

Updated jzIntv executables posted at http://spatula-city.org/~im14u2c/intv/for Windows, Mac and Linux.

 

The archives also include the 'doc' folder, which has minor updates to some programming docs and jzIntv configuration.

[tacrec]

New version works great, thank you!

[intvnut]

Thanks again to nanochess getting the ball rolling! I'm glad it's finally all working properly, including time scaling and all that!

[+nanochess]

Thanks again to nanochess getting the ball rolling! I'm glad it's finally all working properly, including time scaling and all that!

 

You're welcome. I thought a proper AVI recording was missing from your great jzintv emulator I'm glad to have contributed.

[+DZ-Jay]

Updated jzIntv executables posted at http://spatula-city.org/~im14u2c/intv/for Windows, Mac and Linux.

 

The archives also include the 'doc' folder, which has minor updates to some programming docs and jzIntv configuration.

 

Could you post a change-list? That way people can decide whether they should (or want to) upgrade. The last time I updated my tools, it broke my builds and took a long while to notice and figure it out.

 

By the way, thanks for the AVI enhancement, it is something that will prove very useful during development. Currently, my workflow includes a rather heavy screen recording application to snap the action while play-testing in order to analyse it later. Although very versatile and easy to use, the tool taxes my resources and requires more attention than I care to give it most of the time.

 

-dZ.

Edited August 25, 2016 by DZ-Jay

[intvnut]

Could you post a change-list? That way people can decide whether they should (or want to) upgrade. The last time I updated my tools, it broke my builds and took a long while to notice and figure it out.

 

Hmm. I try very hard to not break things that have been working for a long time*. I use these tools also. Do you remember what broke in your build?

 

Bug / regression reports are always welcome. I think this is the first I heard of your breakage due to a tool upgrade.

 

Most of the recent changes have all been around the AVI stuff, minor bug fixes, and documentation tweaks that were discussed either here or on INTVPROG. Notable recent changes in the last year:

• Clarified PSG docs.
• Force a screen refresh every 32 frames even if the screen is completely static. (Fixes window going all-white when dragging between monitors on some systems.)
• Increase MACRO expansion buffer to 1MB
• Fix longstanding "MACRO M" bug, where macro preprocessor would break if the macro's name was a substring of the word "MACRO".
• Fix rate control on Intellivoice for --rate-control > 1
• Fix audio in AVI for --rate-control > 1
• Add flag for alternate palette
• Add --avirate rate-scaling
• Handle reset properly for AVI recording
• Force full STIC rendering pipeline active when recording AVI, regardless of frame-drop requests from rate-control.
• Fix audio/video sync in AVI recordings
• Minor documentation tweak in joystick.txt
• Fix 4 cycle window where STIC registers inadvertently (and incorrectly) become inaccessible when display is disabled
• Add flags to dis1600 to declare ranges as data, and to assign labels to addresses
• Add support for PAGEd RAM, ROM and WOM in both [mapping] and [memattr] sections of .CFG files; support for paged RAM in as1600 and jzIntv.
• Add more documentation on ORG for paged memory.
• Fix PAL emulation speed.
• Handle WOM sections correctly in assembler and other tools
• Split memory mappings at 4K boundaries in .CFG files for readability

 

 

________________

* Touching anything in the MACRO preprocessor and/or its interaction with RPT and IF/ENDI is frought with peril, of course.

Edited August 25, 2016 by intvnut

[+DZ-Jay]

 

Hmm. I try very hard to not break things that have been working for a long time*. I use these tools also. Do you remember what broke in your build?

 

Bug / regression reports are always welcome. I think this is the first I heard of your breakage due to a tool upgrade.

 

Most of the recent changes have all been around the AVI stuff, minor bug fixes, and documentation tweaks that were discussed either here or on INTVPROG. Notable recent changes in the last year:

• Clarified PSG docs.
• Force a screen refresh every 32 frames even if the screen is completely static. (Fixes window going all-white when dragging between monitors on some systems.)
• Increase MACRO expansion buffer to 1MB
• Fix longstanding "MACRO M" bug, where macro preprocessor would break if the macro's name was a substring of the word "MACRO".
• Fix rate control on Intellivoice for --rate-control > 1
• Fix audio in AVI for --rate-control > 1
• Add flag for alternate palette
• Add --avirate rate-scaling
• Handle reset properly for AVI recording
• Force full STIC rendering pipeline active when recording AVI, regardless of frame-drop requests from rate-control.
• Fix audio/video sync in AVI recordings
• Minor documentation tweak in joystick.txt
• Fix 4 cycle window where STIC registers inadvertently (and incorrectly) become inaccessible when display is disabled
• Add flags to dis1600 to declare ranges as data, and to assign labels to addresses
• Add support for PAGEd RAM, ROM and WOM in both [mapping] and [memattr] sections of .CFG files; support for paged RAM in as1600 and jzIntv.
• Add more documentation on ORG for paged memory.
• Fix PAL emulation speed.
• Handle WOM sections correctly in assembler and other tools
• Split memory mappings at 4K boundaries in .CFG files for readability

 

 

________________

* Touching anything in the MACRO preprocessor and/or its interaction with RPT and IF/ENDI is frought with peril, of course.

 

Oh, you are very aware of this breakage; there was a discussion about it on the INTVProg list. It was a change several years ago, but I didn't upgrade until 2015, so mostly missed it until it was too late. The change was regarding the behaviour of "NOT" from boolean to bitwise in the assembler's pre-processor, which arguably was broken from the start but was established behaviour nonetheless. You eventually reverted it, but it caused some headaches to some people, and I at least couldn't figure it out since it wasn't a recent change and my older code inexplicably just wouldn't build. Admittedly, some like Arnauld had already moved on by working around the issue with a macro (%val% XOR $FF), which I implemented as well just in case.

 

Anyway, don't worry about it too much, it wasn't intended to be an accusation. I'm just overly cautious when I upgrade anything (I'm still running iOS 6 on my iPhone and refuse move away from Mac OS X Mountain Lion, and I stopped upgrading iPhone apps when some started introducing ads and creepy features in later builds -- even after I paid for them!)).

 

A change-list is always useful, if at the very least to be aware of enhancements and bug fixes and know what to expect. That list up there is certainly a whopper!

 

-dZ.

Edited August 25, 2016 by DZ-Jay

[intvnut]

Oh, you are very aware of this breakage; there was a discussion about it on the INTVProg list. It was a change several years ago, but I didn't upgrade until 2015, so mostly missed it until it was too late. It was regarding the behaviour of "NOT" from boolean to bitwise in the assembler's pre-processor, which arguably was broken from the start but was established behaviour nonetheless. You eventually reverted it, but it caused some headaches to some people, and I at least couldn't figure it out since it wasn't a recent change and my older code inexplicably just wouldn't build. Admittedly, some like Arnauld had already moved on by working around the issue with a macro (%val% XOR $FF).

*whew*

 

I was worried I had broken something recently. Yes, the NOT change even broke my code, which is especially egg-on-face inducing. And actually, the change was in the other direction as I recall. NOT was supposed to be boolean, but I had implemented it as bitwise way back in prehistory. That prevented statements of the form IF NOT DEFINED FOO from working, but it made statements like MVII #NOT flag, R0 put the bitwise inverse of 'flag' in R0. I didn't anyone was using the bitwise version when I changed the assembler to match the docs.

 

Now I tread a lot more carefully on such things. That's why, for example, I really don't want to touch the MACRO / RPT / IF / ENDI interaction, no matter how goofy and convoluted it is. Yes, it's broken, but it works well enough... Who knows what'll break if I muck around in there. That particular bit of code resembles the last panel in this comic:

 

[+nanochess]

Finally I had time to test the new jzintv package, as1600 fails strangely in my Mac with OS X 10.9.5 (path names changed to hide the guilty one )

 

 

 
dyld: lazy symbol binding failed: Symbol not found: __ZNSt8__detail15_List_node_base7_M_hookEPS0_
  Referenced from: /Users/fhrfhusr/Documents/kposkposks/../jzintv/bin/as1600
  Expected in: /usr/lib/libstdc++.6.dylib

dyld: Symbol not found: __ZNSt8__detail15_List_node_base7_M_hookEPS0_
  Referenced from: /Users/fhrfhusr/Documents/kposkposks/../jzintv/bin/as1600
  Expected in: /usr/lib/libstdc++.6.dylib

Trace/BPT trap: 5

[intvnut]

 

Finally I had time to test the new jzintv package, as1600 fails strangely in my Mac with OS X 10.9.5 (path names changed to hide the guilty one )

 
dyld: lazy symbol binding failed: Symbol not found: __ZNSt8__detail15_List_node_base7_M_hookEPS0_
  Referenced from: /Users/fhrfhusr/Documents/kposkposks/../jzintv/bin/as1600
  Expected in: /usr/lib/libstdc++.6.dylib

dyld: Symbol not found: __ZNSt8__detail15_List_node_base7_M_hookEPS0_
  Referenced from: /Users/fhrfhusr/Documents/kposkposks/../jzintv/bin/as1600
  Expected in: /usr/lib/libstdc++.6.dylib

Trace/BPT trap: 5

 

Urf. I'll have a look at it. Demangling that, it's not finding std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*), which is part of the C++ std::list<> implementation. That's used inside the assembler's macro engine. (The macro engine is in C++; the rest of the assembler is in a mix of K&R C that over time has been upgraded toward ANSI C.)

 

Unfortunately, Macs don't really support static linking, otherwise I'd statically link in as much as I could.

Edited August 26, 2016 by intvnut