AVF metadata format for SIDE3, etc

[flashjazzcat]

I'm revamping the SIDE3's PDM player and adding video playback support, and since this will require a redesigned file format for video files (there being no need for padding bytes in the audio/video data; I'll write a converter for this and provide instructions for encoding videos from scratch), I thought it might be an opportune moment to add a file header with metadata rather than just persist with binary blobs and inferring all information from the filename alone. I'm thinking PAL/NTSC encoding, bitrate, extended title information, genre, etc. There are no particular alignment issues, so we can use as much or as little space as required for the metadata header.

 

If people have no suggestions, I'll go ahead and design it to my own specifications, but I thought I'd just throw it out there for discussion.

[Beeblebrox]

Nice one. Meta data wise, data on video file duration is something I assume is going into the metadata? Would be cool if, like the online Fujiconvert, there was an online converter. Not sure if that is a plan. 

 

Either way, thanks for undertaking this - I know it's a big coding job. 

[flashjazzcat]

13 minutes ago, Beeblebrox said:

Meta data wise, data on video file duration is something I assume is going into the metadata?

I figure run-time (and thus a time position indicator) can be cooked up from filesize divided by bitrate, but yes: if it's common to include runtime data in such headers, why not.

15 minutes ago, Beeblebrox said:

Would be cool if, like the online Fujiconvert, there was an online converter.

My coding skills beyond 6502 assembler extend to C and pretty much no further, so perhaps someone else can take my converter and web-ify it when the time comes.

16 minutes ago, Beeblebrox said:

Either way, thanks for undertaking this - I know it's a big coding job. 

Been wanting to attack this task for some years, but lack of time, etc.

[+CharlieChaplin]

@jhusak worked on an improved AVF player and a new file format a few years ago. Do not know for sure, if he also implemented a header or not. Find attached what I have available (think I downloaded it from atarionline.pl)...

 

The new file format will (however) only work with his new player and he wanted to further improve it - but nothing else was released until today. Don't know if he stopped this project or if its "simply" lack of time. Afair, the new player(s) can also play the old AVF format (but the old player cannot play the newer format).

 

Maybe you can take a short look.

 

P.S.: - AVFPlay is for playback with Pokey sound, all other AVF players are for playback with Covox sound (AVFPlay2 uses Covox at D2xx, AVFPlay5 uses Covox at D5xx, AVFPlay6 uses covox at D6xx and AVFPlay7 uses Covox at D7xx). No matter if Pokey or Covox, sound in AVF is mono only. Short textfile included that explains the extra keys (Start, Select, Option, ESC) in use.

 

 

 

new_AVF_players.zip new_AVF_Files.zip

[+CharlieChaplin]

On my big wishlist would be a completely new AVF format using Gr. 10 (luma) + Gr. 11 (col) and the HIP/RIP  GTIA-shifting trick, so we end up with 160x200 pixels resolution (thanks to GTIA-shifting) and 8 lum x 16 col = 128 colours. What, only 128 colours, not 256 ? Yes, but we have 4 times higher resolution, since Gr. 9+11 has only 80x100 pixels resolution.  phaeron said HIP (greyscale only with 160x200 pixels) would be doable, so I guess (or better wish) this Gr. 10+11 would also be doable and someone would finally do it...  

 

HIP/RIP with GTIA-shifting: (160x200 or up to 160x240 pixels resolution)

Gr.8

Gr.10

Gr.8

Gr.9

Gr.8

Gr.10

Gr.8

Gr.9

etc.

 

my Gr. 10+11 wish with GTIA-shifting (160x200 or up to 160x240 pixels):

Gr.8

Gr.10

Gr.8

Gr.11

Gr.8

Gr.10

Gr.8

Gr.11

etc.

 

Gr. 10 gives us 8 lumas, Gr. 11 gives us 16 colours + the overall/base luma (which could be used to set the darkness or brightness of the picture). Gr. 10 is required for the GTIA-shifting (higher resolution), which afaik happens when switching from Gr. 8 to Gr. 10...

 

 

 

[Mathy]

Hello Jon

 

Don't all these AVF players load data in a fixed block size?  (512kB, 1MB or 2MB?)  If that is true, you could use one such block (the first one?) for the header.  That should be more than enough.

 

One/first block would, if it's not skipped while playing, only be displayed for 1/50th or 1/60th second.  That's shorter than the blink of an eye.

 

Sincerely

 

Mathy

 

[flashjazzcat]

15 hours ago, CharlieChaplin said:

Maybe you can take a short look.

Thanks so much.

14 hours ago, CharlieChaplin said:

On my big wishlist would be a completely new AVF format using Gr. 10 (luma) + Gr. 11 (col) and the HIP/RIP  GTIA-shifting trick, so we end up with 160x200 pixels resolution (thanks to GTIA-shifting) and 8 lum x 16 col = 128 colours. What, only 128 colours, not 256 ? Yes, but we have 4 times higher resolution, since Gr. 9+11 has only 80x100 pixels resolution.  phaeron said HIP (greyscale only with 160x200 pixels) would be doable, so I guess (or better wish) this Gr. 10+11 would also be doable and someone would finally do it... 

It's going to be encoding of the source material for anything but the current 80x96 pixel format which will probably prove most challenging for me, but there is no reason whatsoever why different software graphics modes can't be used in accordance with the playback format description in the file header.

18 minutes ago, Mathy said:

Don't all these AVF players load data in a fixed block size?  (512kB, 1MB or 2MB?)  If that is true, you could use one such block (the first one?) for the header.  That should be more than enough.

 

One/first block would, if it's not skipped while playing, only be displayed for 1/50th or 1/60th second.  That's shorter than the blink of an eye.

Yes - in this case, we'll be pulling in a whole cluster at a time (so multiples of 512 bytes), but data alignment doesn't really matter since video and audio data will be buffered ahead. Header will never be mistaken for AV data, and we can have a header size byte in the first sector which just tells us the size of the metadata area and thus how much data to skip when commencing buffering and playback.

[+CharlieChaplin]

Jakub Husak's AVF players on his Github page:

https://github.com/jhusak/avgcart_avfplay

(just so you know.)

[+MrFish]

KODI (open-source home-theatre software) uses NFO (info) files with simple XML for movie metadata. I'm not suggesting use of separate files or XML; but... you might want to use some of the standard fields they use (or at least get some ideas for what might be useful), since these are based on some more-universal standards for movie metadata.

 

Here's their standard movie template.

 

If you want explanations for any of the tags (and more movie tags that are accepted by KODI), you can see them here: KODI - NFO Files / Movies

 

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<movie>
    <title></title>
    <originaltitle></originaltitle>
    <userrating>0</userrating>
    <plot></plot>
    <mpaa></mpaa>
    <uniqueid type="" default="true"></uniqueid> <!-- add a value to type="" eg imdb, tmdb, home, sport, docu, see sample below -->
    <genre></genre>
    <tag></tag>
    <country></country>
    <set>
        <name></name>
        <overview></overview>
    </set>
    <credits></credits>
    <director></director>
    <premiered></premiered> <!-- yyyy-mm-dd -->
    <studio></studio>
    <actor>
        <name></name>
        <role></role>
        <order></order>
    </actor>
</movie>

 

[w1k]

cant wait for testing:)

[Mathy]

Hello Jon

 

On 5/9/2024 at 2:37 PM, flashjazzcat said:

Header will never be mistaken for AV data, and we can have a header size byte in the first sector which just tells us the size of the metadata area and thus how much data to skip when commencing buffering and playback.

 

Sorry for replying this late, but ...  Will the header be the only difference between the old AVF file format and the new one?

 

Sincerely

 

Mathy

 

 

 

[flashjazzcat]

10 hours ago, Mathy said:

Will the header be the only difference between the old AVF file format and the new one?

No: I'm proposing a new format for SIDE3, since it makes no sense to pad frames of video and audio out to seventeen 512 byte sectors when everything can be packed into sixteen sectors and thus have each frame lined up on an 8K boundary. It shouldn't be very difficult to create a conversion tool for existing AVF files, and of course a modified muxer for those wanting to create their own videos from scratch.

 

Much depends on whether DMA operations can be interleaved with the display kernel such that 500KB/s of data can be buffered in SRAM, which I'll find out shortly when I write some test code to pull data out of an extremely large file. I already think there'll be a few unobtrusive requirements:

• 16KB clusters or larger; this will allow one cluster read of at least two video frames on alternate read cycles, and the reading of FAT linkage on the other cycle
• Files on volumes with 8K or shorter clusters will need to be contiguous (unfragmented) on disk

[+CharlieChaplin]

Please make sure that the new format also works with AVG and Sub cart.

(Someone once gave you an AVG cart, afair.)

 

Think there are already 200+ (or 300+) AVF videos in PAL format available, most of them converted by Poison.

Whereas there are only 10, 20 or 30 AVF videos in NTSC format available by various Atarians.

 

Therefore a conversion tool that converts old AVF format videos into new AVF format videos would be very handy, since no-one would like to convert so many videos manually.

 

[phaeron]

15 hours ago, flashjazzcat said:

No: I'm proposing a new format for SIDE3, since it makes no sense to pad frames of video and audio out to seventeen 512 byte sectors when everything can be packed into sixteen sectors and thus have each frame lined up on an 8K boundary. It shouldn't be very difficult to create a conversion tool for existing AVF files, and of course a modified muxer for those wanting to create their own videos from scratch.

 

Much depends on whether DMA operations can be interleaved with the display kernel such that 500KB/s of data can be buffered in SRAM, which I'll find out shortly when I write some test code to pull data out of an extremely large file. I already think there'll be a few unobtrusive requirements:

• 16KB clusters or larger; this will allow one cluster read of at least two video frames on alternate read cycles, and the reading of FAT linkage on the other cycle
• Files on volumes with 8K or shorter clusters will need to be contiguous (unfragmented) on disk

IIRC, SIDE 3's DMA engine can read from SD at 1 cycle/byte and copy memory at 2 cycles/byte. This means that at best, there is only enough memory bandwidth for one copy of the data at most in NTSC. I don't believe the DMA engine is affected by CPU/ANTIC access contention, but at 29,868 cycles per frame and ~8000 bytes/frame there is only enough time for three full buffer accesses. Probably the more critical issue is window bank availability; a 16-sector format ensures 4K/8K alignment and thus needs only one of the cart windows for the video data, freeing up the other for display list and audio data. Zero-copy playback also frees up DMA engine time to copy the audio data directly into the IRQ handlers, saving CPU time.

 

Since there is 2MB of RAM, pre-buffering the FAT chain is a possibility, at the cost of a short delay at the start and periodically. 1MB of buffer gives 256K clusters, or 128K frames. That's 36 minutes in NTSC.

 

The original player was forced to use a full-screen kernel due to ANTIC needing almost all of the cycles during the display region. With SIDE 3 it only needs about half, so an IRQ-based display is a possibility -- this would make the mainline code a lot easier to write and avoid the need to align the SD reads to frame boundaries. I did this in the past with a 4-channel MOD player to avoid needing to do really complex threading of the music engine that was running in mainline. It was very tight and only ran about 3 instructions of the mainline code per scanline as well as needing the screen off, but it was also mixing four channels in the IRQ handler, which is way more than you'd need to do here. You could probably also get away with not dedicating the Y register to the IRQ handler as I had to do.

 

[flashjazzcat]

7 hours ago, phaeron said:

IIRC, SIDE 3's DMA engine can read from SD at 1 cycle/byte and copy memory at 2 cycles/byte. This means that at best, there is only enough memory bandwidth for one copy of the data at most in NTSC. I don't believe the DMA engine is affected by CPU/ANTIC access contention, but at 29,868 cycles per frame and ~8000 bytes/frame there is only enough time for three full buffer accesses. Probably the more critical issue is window bank availability; a 16-sector format ensures 4K/8K alignment and thus needs only one of the cart windows for the video data, freeing up the other for display list and audio data. Zero-copy playback also frees up DMA engine time to copy the audio data directly into the IRQ handlers, saving CPU time.

That's absolutely correct, and my assumptions are the same regarding CPU/ANTIC access contention.

7 hours ago, phaeron said:

Since there is 2MB of RAM, pre-buffering the FAT chain is a possibility, at the cost of a short delay at the start and periodically. 1MB of buffer gives 256K clusters, or 128K frames. That's 36 minutes in NTSC.

Candle suggested this the other day, but although I pre-map cluster numbers in the PDM player, I'd like to try and avoid a potentially lengthy delay at the start of playback while the entire FAT chain is mapped (and any futher delays during playback if extra mapping is required). Of course, if my plan doesn't work, that would be the fallback approach. At least every cluster link read (using DMA) only takes 512 cycles plus the setup and data-ready overhead, and if the code filling the buffer doesn't have to run synchronously with the player pulling data out of the other end, it might keep up. Once 16K and bigger clusters are working, it would be a toss-up whether 8K or 4K would work with asynchronous buffering, but we could try it and see.

7 hours ago, phaeron said:

The original player was forced to use a full-screen kernel due to ANTIC needing almost all of the cycles during the display region. With SIDE 3 it only needs about half, so an IRQ-based display is a possibility -- this would make the mainline code a lot easier to write and avoid the need to align the SD reads to frame boundaries. I did this in the past with a 4-channel MOD player to avoid needing to do really complex threading of the music engine that was running in mainline. It was very tight and only ran about 3 instructions of the mainline code per scanline as well as needing the screen off, but it was also mixing four channels in the IRQ handler, which is way more than you'd need to do here. You could probably also get away with not dedicating the Y register to the IRQ handler as I had to do.

That's encouraging, since interleaving reads with a display kernel is obviously challenging and inflexible. It would also be nice to avoid cycle counting to ensure compatibility with accelerators (since the player code will be in main RAM, one SRAM window being mapped to the video frame buffer and the other to the FAT buffer since that's also filled by DMA).

 

No doubt it will seem like a dumb question once you've answered it, but is there any specific advantage to running the display off IRQs instead of NMIs? I'd be interested to see the MOD player (I will take any good reference examples I can get).

 

PS: Regarding IRQ v NMI: I'm guessing it's to enable the interrupt later on the line (since we're only doing one register change and playing a byte of audio) and thus avoid a costly STA WSYNC.

Edited May 26 by flashjazzcat

[flashjazzcat]

11 hours ago, CharlieChaplin said:

Please make sure that the new format also works with AVG and Sub cart.

(Someone once gave you an AVG cart, afair.)

Yes they did (in the happy, innocent times, LOL). Whether AVG/Sub cart, etc, are capable of playing any AVF file format I design will be up to the person maintaining those cart designs, however. It's essential that video plus audio for one frame occupies 8KB (sixteen sectors, not seventeen) owing to the fact SIDE3 will be paging one 8K SRAM window per frame.

11 hours ago, CharlieChaplin said:

Therefore a conversion tool that converts old AVF format videos into new AVF format videos would be very handy, since no-one would like to convert so many videos manually.

It goes without saying. I'll probably end up using converted versions of existing AVF files for testing when developing the SIDE3 player.

[flashjazzcat]

So I found this:

And an interesting discussion on AtariArea about the benefits of using POKEY IRQs instead of NMIs. Good stuff.

[phaeron]

11 hours ago, flashjazzcat said:

No doubt it will seem like a dumb question once you've answered it, but is there any specific advantage to running the display off IRQs instead of NMIs? I'd be interested to see the MOD player (I will take any good reference examples I can get).

 

PS: Regarding IRQ v NMI: I'm guessing it's to enable the interrupt later on the line (since we're only doing one register change and playing a byte of audio) and thus avoid a costly STA WSYNC.

You've already found the player source, but there's a more important advantage beyond the timing: the IRQ can run for all 262/312 scanlines. It needs to, because it's driving audio as well as the display mode switching. With a DLI there's no capability to continue writing audio during vertical blank.

 

WSYNC is optional but does improve sound quality due to the more consistent write timing. For 4-bit audio, it's theoretically possible to avoid this need by double-buffering the audio with a pair of exactly opposing channels -- these can then be scheduled so that the IRQ handler is always changing the volume of the inactive channel. I haven't tried this to check for crossover noise, however. PDM requires two channel writes and AFAIK there is no way to double buffer the low channel, the best you can do is buffer the high channel to improve lo/hi sync.

 

[flashjazzcat]

13 hours ago, phaeron said:

You've already found the player source, but there's a more important advantage beyond the timing: the IRQ can run for all 262/312 scanlines. It needs to, because it's driving audio as well as the display mode switching. With a DLI there's no capability to continue writing audio during vertical blank.

Good point; this means the mainline code can continue to handle file reading during the vertical blank.

13 hours ago, phaeron said:

WSYNC is optional but does improve sound quality due to the more consistent write timing. For 4-bit audio, it's theoretically possible to avoid this need by double-buffering the audio with a pair of exactly opposing channels -- these can then be scheduled so that the IRQ handler is always changing the volume of the inactive channel. I haven't tried this to check for crossover noise, however. PDM requires two channel writes and AFAIK there is no way to double buffer the low channel, the best you can do is buffer the high channel to improve lo/hi sync.

I hope to back-port the DMA file reader to the PDM player, but since my knowledge of audio processing is entirely rudimentary, input from those with expert understanding will be most welcome when it comes to fine-tuning things.