Geneve OS development discussion

[+OLD CS1]

"Ignored"  How rude!

[+mizapf]

Yes, you could save on "Ignored". Like in the Unix world, no news is good news, so any kind of message is a warning or an error.

 

But this is a great and long-awaited enhancement. Can't tell you how many times I misspelled the directory name when using CD.

 

(I'm currently watching my Farscape Bluray box, so as a shorter exclamation I'd suggest "Frell", "Yotz", or "Dren". )

 

[+InsaneMultitasker]

Well, there go my plans for a 'cranky' version of the OS error messaging alerts.

[+OLD CS1]

8 minutes ago, InsaneMultitasker said:

Well, there go my plans for a 'cranky' version of the OS error messaging alerts.

Long time ago in my C++ class, I was docked points for an error message which read "RTFM!  You must enter an integer."  After seeing the deduction, I shook my fists into the air and said that programs do not need to be friendly, just usable, and dashed out the door cackling and yelling "You'll see!  You'll see!"

 

Now my fraking Windows machine wants to be my fraking friend with split personalities.  It also shows "We're getting things ready for you!"  Who the frak is "we"?!  Must be the gremlins in the code sending all my usage telemetry to Redmond.

 

 

 

[+9640News]

@dhe

 

I just looked at github.  I think you have an easy way to download the files without using the git command.  In the below snip of the github.com/beerymiller/mdos URL, I selected the green "Code" button.  Last option in that drop down menu is "Download ZIP".  I had a file that was downloaded.  Simply copy the contents in the "MDOS-main" folder to a folder on your TIPI drive that has a native files directory, and you should be good to go.

 

[+dhe]

@mizapf

 

I've made some progress on getting MDOS, download and getting MDOS to build.

 

 TIPI is setup with 3.17, NATIVE_TEXT_DIR=MDOS - PI.CONFIG.

 

  I can list files now as dv80.

 

  make hdos,f  <- hard lock up!

 

  I'm running mame 254 from whtech.

 

  In order to try to debug what is causing the lock up, after windows I added -oslog

 

  I further reduced the command line I start with to:

      mame geneve -oslog

 

  I still am not getting an error.log file.

 

  It is possible mame 254 doesn't have error.log enabled?

 

 

 

[+mizapf]

error.log is created by the option "-log". The "-oslog" outputs the same content to the console.

[+dhe]

My problem spans -- mdos / genasm / tipi / mame, so I don't think I can say MAME issue.

 

But here are the messages repeated over and over again in the log.

 

[:peb:slot6:tipi] Writing to DSR space: 4f9a <- 20
[:peb:slot6:tipi] Writing to DSR space: 4f9b <- 01
[:peb:slot6:tipi] Writing to DSR space: 4f98 <- cf
[:peb:slot6:tipi] Writing to DSR space: 4f99 <- 7e
[:peb:slot6:tipi] Writing to DSR space: 4f96 <- cf
[:peb:slot6:tipi] Writing to DSR space: 4f97 <- 7c
[:maincpu] Undefined opcode 0000 at logical address cf7c, will trigger MID

 

  @mizapf   Do these errors mean anything to you?

 

 

[+mizapf]

geneve or genmod?

[+dhe]

The full line I'm using is:

 

:: LDOM 07.08.2023 - updated -flop1, and tipi address
mame genmod -window -log -bios 1.00 -peb:slot3 horizon -peb:slot5 tirs232 -peb:slot6 tipi -conn rpi.192.168.2.224 -peb:slot8 hfdc -peb:slot8:hfdc:h1 generic -peb:slot8:hfdc:h2 generic -peb:slot8:hfdc:h3 generic -peb:slot8:hfdc:f3 525dd -peb:slot8:hfdc:f4 525dd -hard1 genos7boot.HD -hard2 Bootdisk2.HD -hard3 Bootdisk3.HD -flop1 DM1000SRC.dsk -flop2 DSDD2.hfe -flop3 DSDD3.dsk -flop4 DSDD4.dsk -serl1 socket.localhost:10000

 

I have been able to get Clint's DM (as updated by InsaneMultitasker) to lockup, by going up and down directories.

 

The lockup only occurs on K: (tipi), not E: (hfdc).

 

Hard lock still happens when running make, error log result:

 ladmin@DESKTOP-MC8GIGS:/mnt/d/mame$ tail error.log
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:maincpu] Undefined opcode 0003 at logical address 0002, will trigger MID
[:peb:slot6:tipi] Stopping TIPI

 

Hardlock,

   -rwxrwxrwx 1 ladmin ladmin 586M Jul  8 17:23 error.log

 

This is under windows 10, I just hop in to bash because ... tail.

 

I learned that mame doesn't seem to write log entries as it goes, because I was attempting to use the growing of the file size as a monitor for if geneve/mame was still running. error.log was 0 sized, until I closed mame, and then it wrote the entire 586MB at once! 😃

 

I'm not sure how to continue debugging.

   I'm able to read and write to K: and K:\MDOS

   This is a real PI running 3.17

     Emulation Mode is Set via the tipi web interface.

 

    Here is the config.pi.

     

Anyone else here able to successfully MAKE Geneve OS on your end entirely on the tipi with MAME 254?

 

Thanks!

 

 

[+mizapf]

I did not try on the Genmod yet. Could you try it on a Geneve setup?
 

Edit: I just tried it; the Genmod works with the tipi as expected; boots and shows the directory. So it occurs only when you use DM?

[+9640News]

@dhe

 

I would use Fred's GDM2K versus Clint Pulley's DM.  I am not sure how many directories and files you have on the TIPI, but if you get over 127 of each, you may be encountering buffer issues potentially corrupting code depending where the buffers were set.  

 

I do not presently have MAME setup under genmod, and my real PI configuration needs some work at the moment, otherwise I would test.

 

The other thing you could do is to run MEMTEST I released to see if you are flagging any memory issues to also rule out some issues.  I think the latest MEMTEST was released with the last MDOS release.

 

I think the MID trigger is causing the issue with the lockups, but whether that is code being overwritten, corrupt memory, or another issue is difficult to identify.  I would also ssh into the TIPI and see what the tipi.log file has to say to see if it is reporting any errors.

 

Beery

 

 

[+mizapf]

1 hour ago, dhe said:

I learned that mame doesn't seem to write log entries as it goes, because I was attempting to use the growing of the file size as a monitor for if geneve/mame was still running. error.log was 0 sized, until I closed mame, and then it wrote the entire 586MB at once! 😃

This is not MAME's fault, perhaps Windows buffering? When I start MAME in Linux in one terminal and do a tail -f error.log in the other, the output shows up as soon as it is produced.

 

Please send me the DM files you are using.

[+InsaneMultitasker]

I wouldn't be too quick to blame DM; the issue noted with DM may be a symptom of the problem (hard lock) seen when trying to build Geneve OS.

 

If "TIPI" is a common denominator for the issues with DM and the OS build process, I would suggest that you copy the files to the HFDC device.  If there is a file/directory limitation, you'll find that immediately during the copy.  If the OS builds from HFDC, then you can dig deeper into the TIPI setup and why the OS won't build from the TIPI.  You could have a corrupt GenProg utility, maybe a file in the native directory is being accessed incorrectly, etc. Isolation is important, IMHO.

 

It would also be wise to use CRCOS to verify whether SYSTEM-SYS has been corrupted.

 

@mizapf might know if the DSR space error messages are issues or warnings.  The only real DSR spaces reside on the external bus, as the Geneve OS uses RAM at 0x4000-0x5FFF for its DSR.  So it would depend on the reason for the writes and what is executing at the time. 

 

That's my $0.02 coming late to the conversation.

[+InsaneMultitasker]

18 hours ago, 9640News said:

I think the MID trigger is causing the issue with the lockups, but whether that is code being overwritten, corrupt memory, or another issue is difficult to identify

@9640News I see that Dan is setting "MDOS" as his native text directory.  What happens to the files that are not DV80 like the PROGRAM files in the root of the MDOS folder?   Will TIPI present these files properly?  What about the DF80 object files?  Edit: files are presented as expected during copy but there may be an issue with some file types created within the folder. See later post regarding DF80 example. 

[+InsaneMultitasker]

@9640News I attempted to MAKE MDOS from the TIPI using the files on Github.  The MAKE program crashes and displays a block of gibberish. ASM is unable to close object files for anything it assembles.

 

When I copy from the native folder to a non-native folder, level 2 works as expected.  With this make process, the startblock 48 is very suspicious to me and may be a clue as to why the operation is failing. Interestingly, 48*256=12288 + 128 = 12416 bytes.  The file is only 11 records (11 text lines) so why is the start block so high?  (Edit: need to validate this with a fresh copy) 

 

unit: 0, blocks: 1, filename: HVERSIONS, startblock 48
2023-07-08 20:14:31,107 LevelTwo    : INFO     getFileBytes reading lines from native file
2023-07-08 20:14:31,108 LevelTwo    : INFO     found 11 records
2023-07-08 20:14:31,109 LevelTwo    : ERROR    request exceeds file size: 384, start: 12416, end: 12672

 

 

I'm not sure what to make of the inability to open the catalog file.

This happened during the assembly process. Maybe it is indicative of what Dan is experiencing with DM. 

As noted in a later post: If the list filename parameter is empty, ASM parses the current directory and attempts to open it as OUTPUT, D/V 80.  As we all know, the catalog file can't be opened in OUTPUT mode so the Geneve DSR returns an error and ASM continues on its way.  It is only thanks to the excellent logging in the TIPI that we see this oddity.  ASM is protected by CRC traps and pitfalls so for now, this is noted for awareness. 

 

2023-07-09 12:22:46,337 TipiService : INFO     Request completed.
2023-07-09 12:22:46,345 TipiDisk    : INFO     Opcode 0 Open - TIPI.MDOS.CYA.
2023-07-09 12:22:46,346 Pab         : INFO     opcode: Open, fileType: Sequential, mode: Output, dataType: Display, recordType: Variable, recordLength: 80, recordN                          umber: 0
2023-07-09 12:22:46,347 TipiDisk    : ERROR    responding with error: 2
Traceback (most recent call last):
  File "/home/tipi/tipi/services/TipiDisk.py", line 93, in handleOpen
    cat_file = CatalogFile.load(unix_name, pab, devname)
  File "/home/tipi/tipi/services/ti_files/CatalogFile.py", line 45, in load
    raise Exception("bad record type")
Exception: bad record type
2023-07-09 12:22:46,348 TipiDisk    : ERROR    failed to open dir - TIPI.MDOS.CYA.
Traceback (most recent call last):
  File "/home/tipi/tipi/services/TipiDisk.py", line 93, in handleOpen
    cat_file = CatalogFile.load(unix_name, pab, devname)
  File "/home/tipi/tipi/services/ti_files/CatalogFile.py", line 45, in load
    raise Exception("bad record type")
Exception: bad record type

 

[+dhe]

 

Last night, I also tried upgrading to 256 of mame.

  I didn't expect that to change anything and it didn't.

Also last night I was mentally broke, so I decided to sleep on it, and I came up with the same idea of copying things from tipi to hfdc and trying the make from there.

 

First I imported gdm2k-16-04-2023 into the hd image.

 

My first run at copying from tipi to hfdc I did with genmod, only made it cli/getcats before lock up.

 

I then followed @mizapf advice and switched over to geneve (instead of genmod) in the mame start up.

  This time I made it to scsi2/sectors_s before a hard lockup.

 

As you can see in the attached photo, it appears gdm2k locks up when reading from tipi (no problems where found afterwards in either tipi or daemon logs).

 

I can also say, -oslog explodes with undefined opcode errors when doing file i/o off the tipi, but doesn't when copy or deleting files from the hfdc.

 

 

 

 

 

[:maincpu] Undefined opcode 0000 at logical address 67b4, will trigger MID
[:maincpu] Undefined opcode 0000 at logical address 67b4, will trigger MID
[:maincpu] Undefined opcode 0000 at logical address 67b4, will trigger MID
[:maincpu] Undefined opcode 0000 at logical address 67b4, will trigger MID

 

 

[+mizapf]

Yes, it seems that using Genmod is not the origin of the problem.

 

As for the write accesses - remember that you can always look into the source code at github: https://github.com/mamedev/mame/blob/master/src/devices/bus/ti99/peb/tipi.cpp:

 

In setaddress_dbin, the m_dsr flag was set because the write access was indeed to the DSR space (>174xxx) and not to the ports, while the card was selected. Hence, the write method complains about that fact.

 

You could use the debugger (-debug). There you can set a watchpoint which stops execution when the condition is true.

 

wpset a000,4,w

 

will stop when you write to A000-A003 (w=write). Type "go" to continue (or hit F5).

 

Note that you have to put a prefix before the address: You need to watch out for 174f9a (with Genmod) if it complains about a write access to 4f9a; for the Geneve it is 074f9a if I recall correctly. (Edit: right so)

 

When it stops you can see the portion of code that caused the access in the disassembler window.

 

Edit: So you could set a watchpoint this way for the Genmod: wpset 174f9a,1,w.

[+dhe]

Follow up:

  Two indicators of where the problem might be:

     1) Subsequent lockups always resulted during 0R - which is to say - initial reading of the file from tipi.

     2) I decided to try to finish the data migration by doing the remaining two directories using c(opy) each file, instead of doing them by directories.

          Doing the method I was able to copy the 25 remaining file.

       This tells me the fail is happening most frequently when the tipi is being access frequently, and I think on a file open, not the amount of data being moved.

 

   After manually copying the rest of the files, I was able to build mdos from the hfdc.

 

 

 

[+dhe]

ISSUE #2 with GeneveOS Make.

 

Next up was to test this with "real" geneve.

 

ssh'ed in to pi.

  cd to tipi_disk

  git {mdos files}

 

On Geneve:

modified pi.config with native_text_dir = mdos.

cd mdos

make

 

I started the compile - it didn't lock up, but didn't seem happy.

 

 

[+InsaneMultitasker]

Seems we are at a similar point, @dhe

 

If I copy an object file (DF80) from my ramdisk to the TIPI native folder, I can assemble the associated source file without an error.  I noticed this only because I was able to successfully "MAKE GPL" after I copied all source and object files from my ramdisk to the TIPI native_text folder.  

 

I compared the files that I copied from the ramdisk versus those I downloaded from github.  The source files matched 100%.  The object files are a different story. 

 

See screenshot.  The file on the left was created on the ramdisk, copied to TIPI, and over-written successfully (during assembly) without errors.  The one on the right was created as a new file by the assembler and generates the same error you saw during your attempt.

 

If I am understanding correctly, the newly-created object file (DIS/FIX 80) does not contain a well defined TIFILES header.  When I catalog the folder, the file shows up as DV80 file, presumably because TIPI thinks it is a raw text file?  It would be good for someone to validate my findings and then post in the support forum for @jedimatt42 to review.    

 

 

(I haven't tracked down why the catalog file is seemingly being opened as "OUTPUT, DISPLAY, VARIABLE" on occasion. This might be related to the ASM bug that we found a year or two ago)

 

[+9640News]

@jedimatt42, @dhe, @InsaneMultitasker

 

I did some testing on my own as well.  It looks like the Native Files directory is having newly created DIS/FIX 80 files being thought of as DIS/VAR 80 files at least when you do a directory listing.  The tipi.log file shows the DIS/FIX 80 file is being opened as a native file when I think it should be opened with the TIFILES header..  

 

I assembled a source file in the Native folder with a destination object file on the HRD, no errors.  With a destination folder to a non-existent file in the Native folder, I get the error. 

 

If I already have a destination object file that is DIS/FIX 80 and assemble a file with a destination file of that name, then no errors.

 

If I take TIFILES headers of those files to the TIPI, with no object destination files, then things assemble properly.

 

Looking at the tipi.log file below, it looks like it is opening the DIS/FIX 80 file as a native file.

 

 

 

 

tipi.log

[+InsaneMultitasker]

29 minutes ago, 9640News said:

I did some testing on my own as well. 

I sent you a PM with some additional file type tests to confirm whether we are seeing the same results.  For example, attempting to write a new Int/Fix 100 file generated an error and required me to restart the TIPI.  If I am reading the wiki correctly, only DV80 should be read/written as 'native'. 

[+9640News]

2 minutes ago, InsaneMultitasker said:

I sent you a PM with some additional file type tests to confirm whether we are seeing the same results.  For example, attempting to write a new Int/Fix 100 file generated an error and required me to restart the TIPI.  If I am reading the wiki correctly, only DV80 should be read/written as 'native'. 

I believe that was the intent.  I know earlier when I was doing some testing with @jedimatt42's updates, program image files were excluded.  When I looked at the tipi.log file, it showed the new DIS/FIX 80 file being written as a native file and a directory listing then showing it as a DIS/VAR 80 file.

 

I was going to say I did not see the tipi lock up, but it did.  I had to ssh into the PI and do a sudo reboot now where my PI monitor is displaying a "Stop job is running" message with it going in a loop from 1 to 9.  It looks like some of the stop jobs have about 5 minute timeouts if I am understanding the display correctly.

[+InsaneMultitasker]

@9640News Unrelated to the native_text issue...

I can confirm that ASM is at fault for incorrectly opening the catalog file.  If the list filename parameter is empty, ASM parses the current directory and attempts to open it as OUTPUT, D/V 80.  As we all know, the catalog file can't be opened in OUTPUT mode so the Geneve DSR returns an error and ASM continues on its way.  It is only thanks to the excellent logging in the TIPI that we see this oddity.  ASM is protected by CRC traps and pitfalls so for now, this is noted for awareness.  We can only guess as to whether this was an oversight or intentionally ignored.