Turgen System 8.8.x Adventures

[rra]

I'm looking forward to this dual track update.  I wonder why dual track cassettes with audio wasn't used more when the Atari computers were going strong in the 1980's.  It was boring and listening to the screeches of the loading file wasn't pleasant.  It would have been much nicer to listen to music, or game instructions, or even commercials for other games by the same company.

[baktra]

9 minutes ago, rra said:

I'm looking forward to this dual track update.  I wonder why dual track cassettes with audio wasn't used more when the Atari computers were going strong in the 1980's.  It was boring and listening to the screeches of the loading file wasn't pleasant.  It would have been much nicer to listen to music, or game instructions, or even commercials for other games by the same company.

It would be interesting to speculate on that... I will share my speculations.

 

In the early times (1979-1982), dual track might have been a good idea. The software was still small in size and releasing software on tape with audio track was appealing and also novel.

 

Later, as the software grew in size, the serious flaw of the Atari tape loading system became obvious. It was simply too slow! While 600 bps might have been good enough for computers with 16 KB of  RAM, loading a 48 KB game was thoroughly unpleasant (and boring) experience.

 

The competing C64 and ZX Spectrum had slow basic speeds (300 bps, 1200 bps) indeed. Increasing speed to 3000 bps was matter of writing a faster loader, because the limitation was given by the software.

 

On the other hand, Atari data recorders were limited by hardware with theoretical maximum around 1400 bps. In practice, however, you were lucky if you got to 900 bps, which was rather abysmal than stellar.

 

Since 1983, the disk drives were more and more affordable (at least in the USA) and many games were sold on cartridges, so the position of the cassettes on the marked had inevitably shifted to low-cost. And low-cost implicates cutting expenses. Mastering of a dual track tape was more expensive than just data track.

 

Later there was no hope for the Atari tapes anyway.

Since 1985, Atari focused on 16-bit machines with higher profit margin, so the 8-bit line was continued to be milked, not to be invested in. And fixing the Atari tapes would require two major investments: 1. A redesigned data recorder that would support at least 2400 bps, 2. Major changes in Atari OS ROM to support the increased speed. No way. Cheaper disk drives and bank-switching cartridges were the solution.

 

On the other side of the hill (e.g. Czechoslovakia of 1987, Poland), disk drives were ridiculously expensive for private users, so we had to take our soldering irons and "updated" our data recorders to 3000 bps, basically by "borrowing" the loading system from ZX Spectrum and calling it "TURBO whatever". Something unthinkable (and unnecessary) for a typical customer from the West.

 

 

 

[rra]

You make some really good points that are all true.  I couldn't wait to get rid of the cassette recorder and go to a disk drive. Another reason why the audio track may not have been used is the unreliable loading of the cassettes.  An experience Atari user could listen to the screeches of the cassette loads and hear when the cassette load was about to fail.  After this happened, the cassette would just keep playing but the program  load was halted in an error.  If there was an audio track playing, you might not be paying attention to an aborted load and instead enjoying music, and not able to figure out that your program didn't load.

[baktra]

10 hours ago, rra said:

You make some really good points that are all true.  I couldn't wait to get rid of the cassette recorder and go to a disk drive. Another reason why the audio track may not have been used is the unreliable loading of the cassettes.  An experience Atari user could listen to the screeches of the cassette loads and hear when the cassette load was about to fail.  After this happened, the cassette would just keep playing but the program  load was halted in an error.  If there was an audio track playing, you might not be paying attention to an aborted load and instead enjoying music, and not able to figure out that your program didn't load.

So, to summarize the issues for the tape loading system:

 

1. Slow transfer speed, hardware preventing from getting too much faster

2. Not reliable as expected. Firstly given by the nature of the tapes, secondly the infamous and never fixed bug in the Atari OS which screwed baud rate calculation sparingly.

3. When an error occurs while booting from tape, motor stays on. Many custom loaders fixed that, though.

4. The C: SIO device doesn't have support for file names. Well done.

5. The XC12 and derivative models (XCA-12, CA-12 etc.) were nasty and cheap. A true implementation of cutting costs.

 

The way to hell was paved with good intentions, though

1. 600 bps was OK in 1979, dual track was a great idea. The FSK modulation theoretically more resistant to errors.

2. The data recorder was looking (almost) as a regular SIO device to the system, the data recorder was yielding serial signal. Not bad for the time.

3. Reading/writing didn't make the system fully busy.

4. Data recorder models 410, 1010, XC11 and XL12 were of an acceptable quality.

 

 

 

 

[stepho]

There's no technical reason why the 2nd track could not have been used for music or adverts (although adverts would make me more and more reluctant to load it or any other title from that company).
I suspect it is more to do with Ralph Baer having a patent on it. He mentions this in his book "Videogames: In the Beginning".

[baktra]

Back to the topic of dual track.

• Executing SOX command to convert from whatever supported input format to 16-bit WAV works
• Executing SOX command to downmix the audio track to mono works

More fun is remaining:

• Trimming/padding the audio track to the desired length + applying fade-in and out.
• Creating the final stereo WAVE file.

As it is usual for free software, support for MP3 will require extra installation steps. I cannot do much here, just provide good instructions on what to download and install. Platform specific, of course. But let us assume that someone longing for dual track tapes will be willing to go through all that.

 

It is all still in the POC stage, no cancellation, running the commands blocks the UI.

This will be addressed later with proper SwingWorker implementation.

 

[baktra]

If you are a brave soul willing to early test the dual track creation, now you have an opportunity.

Instructions are for Windows, but can be adopted for GNU/Linux or others.

 

1. Download and install TURGEN 9.0.0

2. Download https://sourceforge.net/p/turgen/turgen-code/ci/dev-9.0.1/tree/dist/turgen.jar?format=raw and replace the existing turgen.jar

3. Download and install SOX for Windows from https://sourceforge.net/projects/sox/files/sox/14.4.2/

4. Set the PATH environment variable to point to the directory where you have installed SOX

 

1. Run TURGEN

2. Create a playlist item (Standard Plugin is recommended)

3. Select the playlist item

4. Select Create dual track... from the Tools menu

5. Try it yourself. 

 

Notes:

1. MP3 music is not supported by that version of SOX. Try OGG, FLAC or WAV instead.

 

How the dual track creation works.

In essence, TURGEN composes SOX syntax and runs SOX

 

1. Generate data track and determine its duration

2. Check if a potential audio track would fit, if not, terminate.

3. Run SOX. Convert the audio track to WAV file 16-bit, desired sampling rate

4. Run SOX. Downmix the audio track to MONO

5. Run SOX. Trim the audio track to a length that would fit.

6. Run SOX. Pad the audio track from the left and right with silence according to the offset setting

7. Run SOX. Merge the data track and audio track.

 

There is total of 5 SOX executions, so some space is needed for temporary WAV files.

In the future, I will try to make this more efficient.

[rra]

I tried to create a Dual Track file and it gave me an error when trying to merge the data track and audio track.   Based on the error warning that sox gives, I suspect that the problem happens during the Step 5 trimming.  I tried both a long (45 seconds) audio track and a short (10 seconds) audio track. 

 

Creating dual track for E:\O.xex ...
Data track successfully created
sox "C:\Users\A\Downloads\O.wav" -r 44100 -b 16 "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_1266039327374587149.wav"
sox "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_1266039327374587149.wav" "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_25525165704252945817.wav" remix 1,2
sox WARN dither: dither clipped 1 samples; decrease volume?
sox "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_25525165704252945817.wav" "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_37212243353882762484.wav" trim 0s 12171164s
sox WARN trim: End position is after expected end of audio.
sox WARN trim: Last 1 position(s) not reached.
sox "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_37212243353882762484.wav" "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_41791124987945110286.wav" pad 1543500s 220500s
sox -M "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_audio_41791124987945110286.wav" "C:\Users\A\AppData\Local\Temp\turgen_dualtrack_datatrack87669457972418247.wav" "C:\Users\A\Desktop\dual track\O" 
sox FAIL formats: can't determine type of `C:\Users\A\Desktop\dual track\O'
ERROR: Unable to merge the audio and data tracks
java.lang.Exception: Command terminated with return code :2

[baktra]

Thanks for trying...

 

On the error...

If this is what you specified as output file: C:\Users\A\Desktop\dual track\O

then it will not work, because without file extension, SOX has no idea about the resulting file format as t specifies in the error message.

Try to specify C:\Users\A\Desktop\dual track\OutDual.wav as output file instead.

 

It would be useful, if TURGEN checks presence of the file extension.

 

As for the warnings...

sox WARN trim - this is probably because the audio track is shorter than the data track. Should be harmless.

sox WARN dither: dither clipped 1 samples; decrease volume - mixing stereo to mono resulted in clipping. I might be able to set SOX to avoid that while downmixing.

 

[rra]

Thanks for explaining the error!  By specifying .wav as the output extension, I was able to generate the dual track .wav file correctly.

 

Now if I want to use dual track cassette loading with Altirra, can I use .wav2cas or Turgen to convert the dual track .wav file to create a dual track .cas file?  Does this even work on any emulators?  Or can I only use the dual track .wav file on an actual cassette using a real Atari data recorder?

 

One other request, it might be a good feature to add the option to repeat the audio track as many times necessary to fill the entire loading time.   Sometimes you might not want to do this (probably with spoken words), but sometimes you might want to do this (with audio).

[baktra]

#1

There is no such thing like dual track .CAS files. The .CAS files are only for the data.

However, the Altirra emulator happily accepts stereo WAV files as dual track tapes, so you can try with Altirra. The atari800-a8cas works too.

 

#2

Yes, it could be useful indeed.

 

If you re-download the jar file using the very same link, it will have mono downmix parameters adjusted to prevent clipping.

Edited May 6, 2022 by baktra

[rra]

OK, thanks.  I loaded the dual track .wav file in Altirra and it worked just fine!

 

A couple comments:

 

The program loaded in two stages.  The first stage is only about 5 seconds - probably a cassette bootloader.  The audio track doesn't play during this stage.  When the second stage starts, the audio track starts playing.   Not a problem, just an observation.

 

I wasn't sure if you could still hear the noises of the program data loading simultaneously along with the music in the audio track.  The answer is "yes, you can hear both the music and screeching noises of the data loading."  I'm sure there is probably a way to turn the data sounds off by writing some value to some memory address.  Is that a feature that Turgen could support in the cassette bootloader? (Maybe it already does support that).

 

 

[baktra]

37 minutes ago, rra said:

OK, thanks.  I loaded the dual track .wav file in Altirra and it worked just fine!

 

A couple comments:

 

The program loaded in two stages.  The first stage is only about 5 seconds - probably a cassette bootloader.  The audio track doesn't play during this stage.  When the second stage starts, the audio track starts playing.   Not a problem, just an observation.

 

I wasn't sure if you could still hear the noises of the program data loading simultaneously along with the music in the audio track.  The answer is "yes, you can hear both the music and screeching noises of the data loading."  I'm sure there is probably a way to turn the data sounds off by writing some value to some memory address.  Is that a feature that Turgen could support in the cassette bootloader? (Maybe it already does support that).

 

 

#1

That's why the Offset box allows you to determine when the audio track starts. How many seconds from the beginning. Depends on your preference.

 

#2

Normally, what you actually hear is the screeching (the data) and the beeping (added by the Atari OS).

 

You can switch the beeping off by placing a value to the SOUNDR memory location.

If you create your playlist item using the Standard plugin and choose the TSCBL or LBE loader, you have the "Silent I/O" option. This option disables the beeping. Of course, the beeping is disabled only after the TSCBL is booted (or LBE loads its first long block). That's where you want the audio track to start in most cases.

 

As for the screeching. Theoretically, you shouldn't hear it, because it is in the right channel, not connected to the AUDIO IN pin.

In reality, you somehow hear it anyway because of the crosstalk. Altirra doesn't seem to have any control for crosstalk level, atari800-a8cas allows you to set it up.

 

Good news is that SoX can easily determine length of the audio track and report it back to TURGEN, so with minimum effort, I will be able to repeat the audio track when it is too short and also prevent the warning about trimming beyond the end of file. Long live SoX.

 

 

 

 

 

 

[rra]

Thanks so much for explaining how this works.  I'm glad you have figured out a way to repeat the audio track - that would be a really nice option.  I'm looking forward to your next updates!

[baktra]

If you re-download the .jar, you will have ability to automatically repeat short audio tracks. TURGEN will repeat a short track as many times as needed. Selecting Fade in and out is recommended in this case. I've also added few enhancements - duration of the tracks is displayed in the processing log etc.

https://sourceforge.net/p/turgen/turgen-code/ci/dev-9.0.1/tree/dist/turgen.jar?format=raw

 

There are few convenience items remaining:

1. Detect SOX absence and stop processing immediately, saying "SOX not installed/available"

2. Update product documentation

[rra]

@baktra Great work!  I just created a dual track .wav file with repeating .mp3 and it works great.   For use on an emulator, the long 12 second blank header seems like a really long time to wait.   

 

Is there a parameter that can set to shorten the length of the blank header?

 

Also, what are the settings that can be set for the Silence List: (S) (P) (1.0)?

[rra]

Sorry - I just found the explanation of the Silence List parameters in the documentation.  I also see that the minimum duration for the blank header is 15 seconds. 

[baktra]

9 hours ago, rra said:

Sorry - I just found the explanation of the Silence List parameters in the documentation.  I also see that the minimum duration for the blank header is 15 seconds. 

Grate, glad that you found the information in the documentation.

• 15 seconds is somehow above the real minimum, but it is minimum set by TURGEN.
• The same with silence list. The Standard tape records require around 2.5 seconds of silence (in fact a neutral signal) as minimum after INIT segment, so if the silence list is missing or values are lower than this minimum, TURGEN has an invisible caring hand which fixes that.

Why is that? Some users in the past shot themselves in the foot by keeping these too short. So we have a million minus 2 ways of shooting oneself in his foot when using TURGEN ?

 

[baktra]

Let me take a small detour to a different topic.

 

Something I realized when trying WAVE files containing Polish Turbo Blizzard with the Altirra emulator. In Preferences>Wave Output, I had to set the waveform to SQUARE instead of AUTO to make it work.

 

The current implementation of the AUTO setting sets the pulse waveform to be square if the pulse is narrower than certain duration, so the waveform is decided on individual pulse level. God, how this is wrong. You can finish with a signal where the pilot tone pulses are harmonic, while pulses for binary 0 are square.

 

The first impulse was to remove the AUTO setting completely, but I still believe in its usefulness. So I've decided to change the implementation.

Now, the decision is made on the playlist item level. Whenever a plugin generates instructions for the signal generator, it also returns a clue which indicates whether the waveform should be automatically changed to square (if AUTO is set).

 

I made it work for several plugins where it makes sense (Super Turbo, Omicron Turbo, Turbo Blizzard). These work simply, when transfer speed is above 4200 bps, the clue is set and used.

 

Implementing this for the Tape Image Plugin is a specialty, the plugin will need to perform analysis of the pwm chunks and if there is a regular pulse narrower than certain threshold, the clue will be set, probably for all chunks since the last pwms chunk.

The tape side creator will also require this special approach, as it will need to set the clue in the middle of the generated instructions too.

It seems a new SignalGenerator instruction (Set waveform clue) will solve the problem.

 

Raise your hand who understands all I've just written ?

In the end, the AUTO setting will just work much better than before.

 

Edited May 10, 2022 by baktra

[baktra]

Have fun with the dual track: https://sourceforge.net/projects/turgen/files/turgen/turgen_9.0.1/

Blog Entry: https://sourceforge.net/p/turgen/blog/2022/05/turgen-901---dual-track/ 

Just note that it is an initial implementation, so there can be some rough edges to be polished in the future releases.

[baktra]

While you can be playing with dual track, I am finalizing the Waveform: AUTO setting enhancement. 

 

To give you some background information.

 

For WAVE or AUDIO outputs, you can select the output waveform. You can choose the theoretically perfect square waveform (which will never make it intact to the tape), practical harmonic pulses (very reliable), or three options between these two.

However, there is also the "AUTO" option (which means I, dear user, do not want to bother). The idea behind AUTO was that for wide pulses, the waveform will be harmonic, while for narrow pulses, the waveform will be square. It is because too narrow harmonic pulses do not pass through the recording chain.

 

The decision was made on pulse level, which yielded a mix of harmonic and rectangular pulses in one turbo block (Turbo Blizzard, Turbo ROM and Super Turbo suffered from this). 

Result? With AUTO setting, the WAVE files with Turbo Blizzard and Turbo ROM were useless with Altirra, for example.

 

The work-in-progress enhancement makes the decision on turbo block level, which results in cleaner, more usable signal.

 

For those who might be interested in the internals:

1. The conventional plugins will set a flag in the SETUP signal generator instruction, which was extended by one byte for auxiliary flags. The flag is set when the plugin determines that the baud rate is > 4000 bps. 

2. The Tape Image plugin will set a flag in the PWMS signal generator instruction, which has been extended by one aux flags byte too. The plugin is doing a pre-scan of turbo chunks to determine the baud rate.

 

When the signal generator is about to create pulses, it will execute the following logic.

1. If waveform is not set to auto, use the configured waveform. Done.

2. If waveform is set to auto, and the flag indicates square waveform, use square. Otherwise use harmonic.

 

The solution is clean and works out of the box also with the Create Tape Sides and Create Dual Track functions, and allows multiple waveforms in one tape side.

The Pilot Tone Test function is a special case and I will see what I can do there.

Edited May 12, 2022 by baktra

[baktra]

What shall we do with Apple M1 users? Is it feasible to get SoX using homebrew?

If it turns out to be a problem, there is always possibility to switch from SoX to ffmpeg, or support even both. 

 

 

[baktra]

On 5/12/2022 at 9:49 PM, baktra said:

What shall we do with Apple M1 users? Is it feasible to get SoX using homebrew?

If it turns out to be a problem, there is always possibility to switch from SoX to ffmpeg, or support even both. 

 

 

We will  make them sit in front of Windows desktop, we will make them sit in front  Windows desktop ,early in the morning.

[baktra]

Release 9.0.2 is a small update that ultimately solves the problem with automatic selection of the waveform.

Especially Turbo Blizzard and Turbo ROM users will benefit (or suffer less).

 

Under the covers, I've made a substantial cleanup of the turgen.Utils class. Utility classes are sometimes a necessary evil for object oriented programmers, a place for code that is needed everywhere but belongs nowhere. Now it holds only what it should. Everything that is at least plugin or function specific is now part of that function or plugin.

[baktra]

I have a few ideas on what to do next. Don't take the comments too seriously.

 

1. Third LBE loader that would allow to re-read a failed block. Is this really needed, or is the PMG loader good enough?

 

2. Enhance GENCAS command line interface to output not only .cas, but also .wav. Cannot beat a8cas in the decoding area? Beat it in signal generation then.

 

3. Redesign menu bar, so it has more standard menus and items. The current menu is still weird. Definitely an UX challenge. Anyone with UX experience willing to help?

 

4. ExpressLoading for the Lower Silesian Turbo plugin. Cloners of Turbo 2000 always prosper.

 

5. Dual track for multiple playlist items. More unicorns?

Edited May 25, 2022 by baktra