Announcing "AGSP Enhance" v1.0 for the MyArcade Atari Gamestation Pro

[Handy DG]

2 hours ago, tjrtech said:

Hey, Handy DG, a few questions:

1. When you say that the original games.ini, when updated by AGSP Enhance, "doesn't work" when loaded up by the AGSP, what do you mean exactly?   I assume the AGSP runs, loads, but with no new games.

2. After using the file contents copy "hack" extra step does that new games.ini work from that point on when adding new games, updating games via AGSP Enhance?  (wondering if this is truly an original games.ini issue)

Thanks,

TJR

1.  Yes, the AGSP boots normally, but is essentially still in factory default mode because the games.db database will not update.  Also, please note that the "update_db" flag in games.ini is still set to "1", so it seems that the AGSP does not even recognize the file.

 

2. Yes, the games.ini will work after that point if modifications are made manually in a text editor, but it will not be recognized if any modifications are made using AGSP Enhance.

[tjrtech]

On 4/26/2024 at 1:10 PM, Handy DG said:

1.  Yes, the AGSP boots normally, but is essentially still in factory default mode because the games.db database will not update.  Also, please note that the "update_db" flag in games.ini is still set to "1", so it seems that the AGSP does not even recognize the file.

 

2. Yes, the games.ini will work after that point if modifications are made manually in a text editor, but it will not be recognized if any modifications are made using AGSP Enhance.

I understood everything you said, but have no way of figuring that out without a Windows PC to test on. 

I wonder if simply opening the version of the games.ini file that AGSP Enhance puts on the SD card with a text editor, changing one thing within (add a character, remove it, save) if that alone would make the AGSP "recognize" the file and process it?

[Handy DG]

29 minutes ago, tjrtech said:

I wonder if simply opening the version of the games.ini file that AGSP Enhance puts on the SD card with a text editor, changing one thing within (add a character, remove it, save) if that alone would make the AGSP "recognize" the file and process it?

Unfortunately not. I have tried that several times without success.  As @Torq surmised, there seems to be some low level attribute to the original games.ini file that the AGSP recognizes. 

[+Torq]

3 hours ago, Handy DG said:

Unfortunately not. I have tried that several times without success.  As @Torq surmised, there seems to be some low level attribute to the original games.ini file that the AGSP recognizes. 

 

I'm fairly convinced the updates to the games.ini file are occurring on an SD card with LOTS of files on it, and are pushing the re-written copy of games.ini beyond the number of FAT entries the AGSP will read.

 

I do NOT think it is the CONTENT of the file that "AGSP Enhance" is writing that is a problem.

 

Here's a way to further test my supposition:

1. Use AGSP to update your "native" games.ini file.
2. Make a copy/backup of your AGSP SD Card, with the updated games.ini file included.
3. Format ("Erase" in Disk Utility on macOS) the SD Card.
4. Copy JUST your updated/backed-up "AGSP Enhance" games.ini file to /data/games.ini (ensuring the update flag is set).
5. Then copy the OTHER files/folders back to the SD Care making sure to exclude games.ini.

If that works, the problem is what I think it is.  And if so, it may NOT occur in situations in which there aren't LOTS of files (don't know the precise number) on the SD card in question.

 

[Spanner]

This just needs a GUI that installs Python for you and sets everthing up, nice work, well done...  I just tested it on windows and scan and found my games.ini file on my SD card OK.

Have you got the link to it on Github, I can not find it...?

Edited May 6 by Spanner

[Spanner]

I am looking for something to use with PCUAE when I add it to AGSP, has anyone tried to see if they can use the USB-C Power port in the back as a OTG USB Drive/stick yet, like on the Atari2600 Plus they both use the same specs, if you can then it might be better then using the SD card, because its stuck on exFAT.
What about the 2 USB-C ports at the front are they for power only so power the joysticks only...?

 

I looked at the main game binary yesterday and it uses `/lib/ld-linux-armhf.so.3` that all the other minis use so its a armv7 so like THEC64 Mini/Atari2600 Plus, THEC64, THEA500, THEVIC20, Atari Flashback all use the same libs, if you look at the game binary with HxD the first lib you will see is `ld-linux-armhf.so.3`all the others use it because it comes from originally THEC64 Mini that RGL copied to each machine they made.
The games GUI menu might be able to be run on another system, if you add all its dependences it needs.

Edited May 6 by Spanner

[Spanner]

On 4/14/2024 at 4:32 PM, tjrtech said:

AGSP Enhance is a utility that makes it easier to add games to the MyArcade Atari Gamestation Pro (AGSP) whether those games are for the Atari 2600, 5200, 7800, Arcade, Nintendo Gameboy, Sega Genesis, etc.
 

All you need to use AGSP Enhance is access to game ROMs (for the aforementioned consoles and arcade) as well as image files for things like box art, arcade cabinets, screen shots and banner titles. Yes, that all sounds daunting but it really isn't.  Some links to resources and helpful tips are included below.

Much of the curation (finding ROMs and image files) has been done for you. However, rather than load up your AGSP micro SD card with games you do not want, the final steps of curation have been left to you, and they include:
 

1. Editing a "curation list" in order to define which games you wish to import to your AGSP.

 

2. Adding need game roms and images to the appropriate Games subfolder(s) of AGSP Enhance for each game you wish to import. For more on this refer to the "How To Curate Games" section below.

3. Executing agsp_enhance to import your curated games - this is where the magic happens. See "The Magic" section below for details.

AGSP Enhance includes 600 curated ROMs that show up within the existing AGSP menus.  You will need to have added the v1.3c Community Firmware to your AGSP.

Attached is a PDF file with further instructions along with a ZIP file which is the complete v1.0 AGSP Enhance utility:

README.pdfagsp_enhance_v1.0.zip

Error when you run it at start, it still works:
 

C:\Games\agsp_enhance>py agsp_enhance.py enhance
C:\Games\agsp_enhance\agsp_enhance.py:155: SyntaxWarning: invalid escape sequence '\ '
  print("     /\   / ____|/ ____|  __ \             | |")
C:\Games\agsp_enhance\agsp_enhance.py:156: SyntaxWarning: invalid escape sequence '\ '
  print("    /  \ | |  __| (___ | |__) |   ___ _ __ | |__   __ _ _ __   ___ ___")
C:\Games\agsp_enhance\agsp_enhance.py:157: SyntaxWarning: invalid escape sequence '\ '
  print("   / /\ \| | |_ |\___ \|  ___/   / _ \ '_ \| '_ \ / _` | '_ \ / __/ _ \\")
C:\Games\agsp_enhance\agsp_enhance.py:158: SyntaxWarning: invalid escape sequence '\ '
  print("  / ____ \ |__| |____) | |      |  __/ | | | | | | (_| | | | | (_|  __/")
C:\Games\agsp_enhance\agsp_enhance.py:159: SyntaxWarning: invalid escape sequence '\_'
  print(" /_/    \_\_____|_____/|_|       \___|_| |_|_| |_|\__,_|_| |_|\___\___|\n")

 

[tjrtech]

On 4/28/2024 at 7:18 PM, Torq said:

 

I'm fairly convinced the updates to the games.ini file are occurring on an SD card with LOTS of files on it, and are pushing the re-written copy of games.ini beyond the number of FAT entries the AGSP will read.

 

I do NOT think it is the CONTENT of the file that "AGSP Enhance" is writing that is a problem.

 

Here's a way to further test my supposition:

1. Use AGSP to update your "native" games.ini file.
2. Make a copy/backup of your AGSP SD Card, with the updated games.ini file included.
3. Format ("Erase" in Disk Utility on macOS) the SD Card.
4. Copy JUST your updated/backed-up "AGSP Enhance" games.ini file to /data/games.ini (ensuring the update flag is set).
5. Then copy the OTHER files/folders back to the SD Care making sure to exclude games.ini.

If that works, the problem is what I think it is.  And if so, it may NOT occur in situations in which there aren't LOTS of files (don't know the precise number) on the SD card in question.

 

If it is a games.ini file size issue then that could be understandable... but then, if so, why is it working for me on MacOS?  I am using the same Curated List, with pretty much everything that can be imported imported, so about 600 additional INI entries and it all works for me.

Oh, and my SD card was original formatted as ExFAT... so there is that!

Edited May 6 by tjrtech

[tjrtech]

59 minutes ago, Spanner said:

Error when you run it at start, it still works:
 

C:\Games\agsp_enhance>py agsp_enhance.py enhance
C:\Games\agsp_enhance\agsp_enhance.py:155: SyntaxWarning: invalid escape sequence '\ '
  print("     /\   / ____|/ ____|  __ \             | |")
C:\Games\agsp_enhance\agsp_enhance.py:156: SyntaxWarning: invalid escape sequence '\ '
  print("    /  \ | |  __| (___ | |__) |   ___ _ __ | |__   __ _ _ __   ___ ___")
C:\Games\agsp_enhance\agsp_enhance.py:157: SyntaxWarning: invalid escape sequence '\ '
  print("   / /\ \| | |_ |\___ \|  ___/   / _ \ '_ \| '_ \ / _` | '_ \ / __/ _ \\")
C:\Games\agsp_enhance\agsp_enhance.py:158: SyntaxWarning: invalid escape sequence '\ '
  print("  / ____ \ |__| |____) | |      |  __/ | | | | | | (_| | | | | (_|  __/")
C:\Games\agsp_enhance\agsp_enhance.py:159: SyntaxWarning: invalid escape sequence '\_'
  print(" /_/    \_\_____|_____/|_|       \___|_| |_|_| |_|\__,_|_| |_|\___\___|\n")

 

Others have seen that invalid escape sequence and running under Python3 seems to fix that.

[+Torq]

24 minutes ago, tjrtech said:

If it is a games.ini file size issue then that could be understandable... but then, if so, why is it working for me on MacOS?  I am using the same Curated List, with pretty much everything that can be imported imported, so about 600 additional INI entries and it all works for me.

Oh, and my SD card was original formatted as ExFAT... so there is that!

I'm not saying it's about the size of the games.ini file size, nor the number of entries in it.

 

It's about how the AGSP reads ExFAT (linearly, in FAT order ... literally the order files were placed on the media), and how ExFat file replacement works across different OSs (create the new FAT entry, before removing the old one, so the new file's FAT entry is later in the list).  If the AGSP has a limit to the number of files it will read, and there are that many (or more) files on the card prior to editing games.ini, the AGSP may simply NOT SEE the new games.ini file at all, as it is now past the number of files it will read.

 

Doing the writes on a different OS means a potentially different way of updating the FAT (Windows might do the create/delete, where macOS does direct-replace, and how Python uses the OS file handling routines can also differ) and can affect where in the FAT the games.ini file shows up.

 

Anyway ... it's just a theory.  I don't have a Windows machine I can test with.  I'm not running AGSPEnhance.  And if you're not familiar with the low-level workings and structures of ExFAT and FATs in general, as well as how different OSs handle them (and/or how the different Python runtimes invoke the OS to make file changes), its not going to make any sense.  

 

And, as such, that just means I'm cluttering up the thread by trying to help with something you may not even be able to address (or may be an entirely different issue) ... so ... I'll stop.  Hope you figure it out!

[Spanner]

It sounds like you talking about how a hard drive works, well FAT is part of it, files that are copied to the hard drive could be at the beginning or middle or the end of the hard drive, that why you can defrag the drive, well older ones so it loads programs quicker cause it moves them to the front the more important programs and one part of a program can be at the beginning and another part of the program at the end for the hard drive so split so takes longer to load it, thats why old drives are slower plus the speed they reads at...
Most Minis use FAT32 not exFAT, first of its kind...  but FAT32 has a 4GB limit of the amount of data it can read in one file.
So is the game binary that reading the SD card mounting it in exFAT, I wounder if it could be hacked to mount it with FAT32, you can use HxD to look at it or ghidra, it finding the right part in the binary then try changing it.

The game binary is the GUI.

 

game

Edited May 6 by Spanner

[tjrtech]

Folks, I have sent to a friend a 64GB microSD card all set, update_db=1, all that, and ready to go.  I will let you all know how it works for him.

TJR

[tjrtech]

BTW, I have sent out SD cards with the 600 games, sdCard in exFat, games.ini set to update_db=1, and it worked flawlessly for someone who had only loaded up their AGSP with the 1.30c firmware.  And, the've made copies of the sd card contents to a folder, and copied to other exFat microSD cards with no issues loading onto other AGSP consoles.

[DFELD2005]

On 4/25/2024 at 4:54 PM, tjrtech said:

I really don't think it has anything to do with the files curated.  I suspect there is an EOF character or something special in the factory version of games.ini that is not in a version of that file that is saved by an editor (or in my case a Python program running Win/Mac).  Simply editing that file, in it's original contents and saving via a text editor I bet would avoid the issue.

I have had other Windows users use the AGSP Enhance utility successfully, and I will reach out to them to see if, by chance, they "hand edited" their games.ini before using the utility.

YES. The factory version of games.ini has text lines ending with a LR character (MacOS style)

        The AGSP Enhance software, running under Windows, creates a games.ini with text lines ending with CR LR characters (Windows style)

There is a free program (Notepad++ for Windows) that can convert line endings in simple text files.

 

[tjrtech]

On 6/20/2024 at 8:37 AM, DFELD2005 said:

YES. The factory version of games.ini has text lines ending with a LR character (MacOS style)

        The AGSP Enhance software, running under Windows, creates a games.ini with text lines ending with CR LR characters (Windows style)

There is a free program (Notepad++ for Windows) that can convert line endings in simple text files.

 

Thanks.  Yeah, that is what I figured it had to be.  So that is the work around.   The default end-of-line for both OS are supposed to be CRLF (carriage return then line feed).  Weird that the original file only has LF.

 

[pmcx]

I am having an           issue getting the sd card to read.   I am on the 1.30c firmware and I formatted the sd card using exfat.

I extracted out all the files and tried to run them.  It gave me an error because it couldn’t find the sd card.  I inserted the sd card and copied all of the files / directories into the data folder of the sd  card. 

I try and run the script and it says it cant find the games.ini in the directory lost+found.

I create a directory called lost+found and execute the script.  I get the below message.

 

I followed the instructions in the pdf.  I got all the files and ran the python script.

I got these results

Good news - games.ini files found in microSD card path E:\.

Copying games.ini file locally.

Successfully refreshed 'protected list.csv' from 'protected list.xlsx'.

Successfully refreshed 'curated list.csv' from 'curated list.xlsx'.

 

Enhancement complete:

        # games processed...........675

        # games complete............600

        # games ready for import......0

        # games imported..............0

        # files copied................0

 

 

The games.ini file is updated in my c:\directory.     

 

After the latest messages I open the file in I open the file in notepad++

At this point I open the ini file and convert end line breaks from lf to cr  by selecting Edit-EOL Conversion – Macintosh (CR).

I save and copy the I then copy the games.ini file over to the sd card.  

I cant get it to pick up the games.

I have tried using the original games.ini and pasting the new contents in.  Setting the values DB=0 and DB=1.  I have tried multiple ways of editing then EOL converting the file.

 

I am not sure what I am doing wrong at this point.

 

 

 

 

[tjrtech]

On 7/2/2024 at 9:26 AM, pmcx said:

I am having an           issue getting the sd card to read.   I am on the 1.30c firmware and I formatted the sd card using exfat.

I extracted out all the files and tried to run them.  It gave me an error because it couldn’t find the sd card.  I inserted the sd card and copied all of the files / directories into the data folder of the sd  card. 

I try and run the script and it says it cant find the games.ini in the directory lost+found.

I create a directory called lost+found and execute the script.  I get the below message.

 

I followed the instructions in the pdf.  I got all the files and ran the python script.

I got these results

Good news - games.ini files found in microSD card path E:\.

Copying games.ini file locally.

Successfully refreshed 'protected list.csv' from 'protected list.xlsx'.

Successfully refreshed 'curated list.csv' from 'curated list.xlsx'.

 

Enhancement complete:

        # games processed...........675

        # games complete............600

        # games ready for import......0

        # games imported..............0

        # files copied................0

 

 

The games.ini file is updated in my c:\directory.     

 

After the latest messages I open the file in I open the file in notepad++

At this point I open the ini file and convert end line breaks from lf to cr  by selecting Edit-EOL Conversion – Macintosh (CR).

I save and copy the I then copy the games.ini file over to the sd card.  

I cant get it to pick up the games.

I have tried using the original games.ini and pasting the new contents in.  Setting the values DB=0 and DB=1.  I have tried multiple ways of editing then EOL converting the file.

 

I am not sure what I am doing wrong at this point.

 

 

 

 

It's as if the script didn't find any games to import.  Did the games.ini file you edited seem to have the extra games?

TJR

[tjrtech]

Just now, tjrtech said:

I

On 7/2/2024 at 9:26 AM, pmcx said:

I am having an           issue getting the sd card to read.   I am on the 1.30c firmware and I formatted the sd card using exfat.

I extracted out all the files and tried to run them.  It gave me an error because it couldn’t find the sd card.  I inserted the sd card and copied all of the files / directories into the data folder of the sd  card. 

I try and run the script and it says it cant find the games.ini in the directory lost+found.

I create a directory called lost+found and execute the script.  I get the below message.

 

I followed the instructions in the pdf.  I got all the files and ran the python script.

I got these results

Good news - games.ini files found in microSD card path E:\.

Copying games.ini file locally.

Successfully refreshed 'protected list.csv' from 'protected list.xlsx'.

Successfully refreshed 'curated list.csv' from 'curated list.xlsx'.

 

Enhancement complete:

        # games processed...........675

        # games complete............600

        # games ready for import......0

        # games imported..............0

        # files copied................0

 

 

The games.ini file is updated in my c:\directory.     

 

After the latest messages I open the file in I open the file in notepad++

At this point I open the ini file and convert end line breaks from lf to cr  by selecting Edit-EOL Conversion – Macintosh (CR).

I save and copy the I then copy the games.ini file over to the sd card.  

I cant get it to pick up the games.

I have tried using the original games.ini and pasting the new contents in.  Setting the values DB=0 and DB=1.  I have tried multiple ways of editing then EOL converting the file.

 

I am not sure what I am doing wrong at this point.

 

 

 

 

You should not have had to copy anything to data, or to have created the lost+found folder.  I suspect you didn't start with an SD card that was booted in the 1.30c firmwared system.  You need that.

[pmcx]

I appreciate the comments and info on this forum but I just cant get this to work.  I have about 20 hours or more into this.   I can see the games in the ini file 

I can see games being copied but they just dont show up in the AGSP interface.  

I have messed around trying to change the line feeds in notepad and still have no luck. 

Does anyone have an IMG file I could just burn to a SD card?

 

 

Good news - games.ini files found in microSD card path E:\.
Copying games.ini file locally.

Successfully refreshed 'protected list.csv' from 'protected list.xlsx'.

Successfully refreshed 'curated list.csv' from 'curated list.xlsx'.
4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(s) copied...4 file(

[Tundrorock]

Was able to get it working, had some issues, as others reported. I'm on Windows 10.
Here's what worked for me.

1. Python install link installed Python but when I ran the command it didn't work. When I ran Python3 I was prompted for Windows Store install of Python 3. I let it do it's thing despite already installing it prior via the link.  This worked and got me past this issue.

2. Got errors in the script - did the pipenv manual commands as stated in the pdf (this is not a bug, simply a note as this IS in the readme).

3. Got more errors, had to get my ORIGINAL 1.30c games ini file back on to the SD card as I had fiddled with it (others reported this prior). Reran and it worked.

4. The GSP did not recognize the file (or possibly the update setting)

4a. I took the original 1.30c games.ini file and deleted all text (not ctrl-A, but manually highlighting all text, I left the spaces/special characters? at the bottom of the file).

4b. I then highlighted all text (same method as above) in the AGSPenhanced generated version and pasted it in to the original that I had cleared.

All was good at that point, the AGP recognized the file.
I know I'm repeating others, but thought it would be good to get a detailed checklist summary.

 

Thanks for the great work on the script and also the custom firmware!