TIPI Usage and Support

[Duewester]

Working with the TiPi 3.10. When the Pi is halted, my FG-99 stuff just freezes up the unit. Never noticed it doing that before. 

[+jedimatt42]

2 hours ago, Duewester said:

Working with the TiPi 3.10. When the Pi is halted, my FG-99 stuff just freezes up the unit. Never noticed it doing that before. 

FG-99 stuff is ridiculously vague. 

 

Try again.

[+arcadeshopper]

14 hours ago, Duewester said:

Working with the TiPi 3.10. When the Pi is halted, my FG-99 stuff just freezes up the unit. Never noticed it doing that before. 

logically if the tipi is halted and you make an access, it will hang waiting for the pi to respond..   reboot the pi and it will start when the pi responds as the service is enabled 

[+jedimatt42]

Running TI-invaders from the FinalGROM99 menu should not hang with TIPI connected while the PI is off.

 

Running Tutankham might, but not because of TIPI.

 

Running the TIPI halt from ForceCommand and then issuing an FG99 command to switch cartridges might hang depending on if you have the clock bar running. 

 

Running original TI Extended BASIC will hang if the PI is off. This has been the case since the beginning of TIPI. As XB tries to access disk immediately.

 

In the future please provide more specific detail in requests or in this case a statement of observation.

 

[Duewester]

Sorry for the vague comment.

I had tried several versions of extended basic to no avail. I was trying to do a Size to verify the 32k part of my setup was working. I had also tried different SD card sizes and speeds. Each time, I could ssh into the tipi PRIOR to doing a Call tipi. After the call froze the unit, I couldn't ssh into the pi.

I was trying to create a Tipi3 account on the cards and had convinced myself that the cards were at issue. Eventually I created a Tipi account (no 3) on one of the cards and got everything working. 

When I Halted the Pi to shut down for the evening, I did a few things from Basic with no issues and then went into the FG for TI Extended basic and things locked up.

I was tired and frustrated so, I turned off the Ti, saw on my desktop PC that there was a response to another post in the forums, so I checked in here and made my comment without giving it much thought and signed off my desktop.

Usually when I Halt out of the Tipi, I shut down and turn off the lights so I never set up the situation where I was trying to use the PC post tipi. Now I know better.

My apologies for the abuse of the forum.

[+9640News]

15 hours ago, Duewester said:

My apologies for the abuse of the forum.

I don't think it was abuse of the forum.  Having been on the development side of things, developers need very specific details to find and/or be able to reproduce the symptoms.  One can't say this software does not work, rather, when I do A), then B), then C), I get this result and it only happens when it is after 8 pm..... <grin>.

 

I just had someone report something to me privately this past week that was looking at something I wrote and have not released yet.  He saw something I have never seen.  He is running his system differently.  We are still working through some details.

 

[+InsaneMultitasker]

On 3/19/2022 at 1:20 PM, jedimatt42 said:

Update 2.34 - 2022-03-19

 

Fix the rename operation to set the new name in the TIFILES header ( it is now really a load, modify, write, delete )

Fix bounds checking on lvl2 direct-input and direct-output to respect the declared limit in the header.

 

 

Hi @jedimatt42,

 

@9640News and I wanted to run something by you related to the level 2 boundary checking fix for direct input and direct output. We seem to be running into a similar situation where we are able to exceed the declared file limit by one block (sector).

 

To confirm what we are seeing, I wrote a short test program that reads a file one block at a time starting with block 0. My example file (TIP-BAD) is a PROGRAM image 1070 bytes in length (4 blocks * 256 bytes/block + 46 bytes in the fifth block) which should present a maximum of 5 blocks via direct input. Here is the pertinent information thanks to TI99Dir: 

  

 

TIPI allows the test program to read a non-existent block 5 and generates an error on block 6. 

We are expecting the error to occur when trying to read block 5.  (Edit: to clarify, the log and input routine are base 0, starting at block 0).

 

The TIPI log reports a file size of 1664 bytes which seems to be the sum of (6 blocks * 256 bytes/block) + 128 TIFILES header bytes. This might suggest that the TIFILES header length of 128 and an extra block are being added to the file size for the bounds check routine.

FYI, the file type e.g., PROGRAM, display variable, etc. did not matter - I was able to read one block beyond the true size in all test cases so far.  And for reference, I am using the 3.x branch.   Thank you. 

 

[+jedimatt42]

@InsaneMultitasker & @9640News

 

I thought that was a year in the past. I guess I should have followed up to make sure it was passing your test cases back then. I'll be asking to make sure we close the loop this time.

 

I thought block 0, is the metadata block for direct-input, then blocks 1-5 are the 5 actual 256 byte blocks used in the file. so, failing on read of block 6 seems appropriate. I'll start by clarifying the logging, making sure the metadata read is distinctly indicated from the file data.

 

But that aside, I must have still been logically off by one somewhere... counting relative to the TIFILES size instead of the other limit, maybe. 

The file size in the TIPI logging from your example does look like it is 1, 256 byte block too large.

 

I'll have to look into it. 

 

I don't have any 4A software written that ignores the metadata read in block 0, and tries to read beyond the boundary. So I guess I'll have to write a test case. I can probably just temporarily break something in ForceCommand to read until error instead of read until the expected count.

 

 

[+InsaneMultitasker]

On 4/29/2023 at 8:15 PM, jedimatt42 said:

I'll be asking to make sure we close the loop this time.

Sounds good.  I did not close the loop on my/our end either, as non-hobby items have been the priority for me.

 

On 4/29/2023 at 8:15 PM, jedimatt42 said:

I thought block 0, is the metadata block for direct-input, then blocks 1-5 are the 5 actual 256 byte blocks used in the file. so, failing on read of block 6 seems appropriate. I'll start by clarifying the logging, making sure the metadata read is distinctly indicated from the file data.

Thank you. I can see where clarifying the logging will help, as I didn't notice that there were two entries for 'block 0'

I altered my test program to read only the additional info block, and the log shows that activity as unit 0 blocks 0 start block 0. 

I then ran the complete test.  The file data follows the format "blocks 1, startblock x".  x starts at 0.  The spoiler contains the log. 

 

Spoiler

LOAD TEST PROGRAM: 

2023-04-30 20:44:23,301 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:23,303 TipiDisk    : INFO     LOAD image size 1280
2023-04-30 20:44:23,479 TipiService : INFO     Request completed.

2023-04-30 20:44:23,804 LevelTwo    : INFO     set path request
2023-04-30 20:44:23,807 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:23,808 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS

2023-04-30 20:44:23,808 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.

 

Get metadata / information block for the file: 

2023-04-30 20:44:23,809 LevelTwo    : INFO     direct input
2023-04-30 20:44:23,813 LevelTwo    : INFO     unit: 0, blocks: 0, filename: TIP, startblock 0
2023-04-30 20:44:23,813 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP

2023-04-30 20:44:23,973 LevelTwo    : INFO     set path request
2023-04-30 20:44:23,977 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:23,977 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:23,978 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.

 

PERFORM DIRECT INPUT OF THE FILE :

Reads one too many blocks.  File is contained within blocks 0-4 (total of 5 blocks)

2023-04-30 20:44:23,979 LevelTwo    : INFO     direct input
2023-04-30 20:44:23,982 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 0
2023-04-30 20:44:23,983 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,123 LevelTwo    : INFO     set path request
2023-04-30 20:44:24,127 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:24,127 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:24,128 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.
2023-04-30 20:44:24,129 LevelTwo    : INFO     direct input
2023-04-30 20:44:24,133 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 1
2023-04-30 20:44:24,133 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,273 LevelTwo    : INFO     set path request
2023-04-30 20:44:24,277 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:24,277 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:24,278 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.
2023-04-30 20:44:24,280 LevelTwo    : INFO     direct input
2023-04-30 20:44:24,283 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 2
2023-04-30 20:44:24,284 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,424 LevelTwo    : INFO     set path request
2023-04-30 20:44:24,428 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:24,429 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:24,429 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.
2023-04-30 20:44:24,431 LevelTwo    : INFO     direct input
2023-04-30 20:44:24,434 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 3
2023-04-30 20:44:24,435 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,575 LevelTwo    : INFO     set path request
2023-04-30 20:44:24,579 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:24,579 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:24,580 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.
2023-04-30 20:44:24,581 LevelTwo    : INFO     direct input
2023-04-30 20:44:24,585 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 4
2023-04-30 20:44:24,585 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,726 LevelTwo    : INFO     set path request
2023-04-30 20:44:24,729 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:24,730 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:24,730 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.
2023-04-30 20:44:24,732 LevelTwo    : INFO     direct input
2023-04-30 20:44:24,735 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 5
2023-04-30 20:44:24,736 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,842 LevelTwo    : INFO     set path request
2023-04-30 20:44:24,846 LevelTwo    : INFO     unit: 0, path: TIPI.2023.TIPS.
2023-04-30 20:44:24,846 tinames.tinames: INFO     TIPI.2023.TIPS. -> /home/tipi/tipi_disk/2023/TIPS
2023-04-30 20:44:24,847 LevelTwo    : INFO     set unit 0 path to TIPI.2023.TIPS.
2023-04-30 20:44:24,849 LevelTwo    : INFO     direct input
2023-04-30 20:44:24,852 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 6
2023-04-30 20:44:24,853 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-04-30 20:44:24,853 LevelTwo    : ERROR    request exceeds file size: t: 1664, s: 1664, e: 1920

 

 

 

 

[+jedimatt42]

23 hours ago, InsaneMultitasker said:

 request exceeds file size: t: 1664, s: 1664, e: 1920

 

so, what's fun is that a 5 block TIFILES file should be ( you said this already I think... ) 128 + (5 * 256) = 1408 bytes. 

The t: 1664 in the message above is the calculated total size of the file.. I have no idea what bug took me through a maze to calculate it as:  ((((len(bytes)-128)/256)+1)*256)+128... when it should have just been len(bytes), no calculation, just measurement.

Nothing like a good old +1 looking you in the eye and saying, "You're nearly 50 kid, and you still write off by one errors..." 

 

Needs a little more testing though, before I push out a patch.

 

Note: on the 4A, DM2K and ForceCommand and other direct-input access based software, programs are respecting the FDR sector count, so we are not seeing this big manifest. It is a bug in TIPI, but it hasn't been corrupting your TIFILES.

 

[+InsaneMultitasker]

1 hour ago, jedimatt42 said:

so, what's fun is that a 5 block TIFILES file should be ( you said this already I think... ) 128 + (5 * 256) = 1408 bytes. 

I think so, provided the reported and/or calculated total size is meant to represent real file size (blocks*256)+128 versus the /4a-side file size (blocks*256). The 128 bytes in the log entry is what caught my attention when I tried to compare block size totals.  After the update is released, I'll run my tests on direct input/output.  Are there any other operations that might be affected or need a quick test, or is this isolated to direct input/output?  Thanks for isolating the cause!

[+jedimatt42]

Update 3.11 - May 1st, 2023

 

- Fix bounds checking for direct-input. 

 

This should error at the correct point if the direct-input request attempts to exceed the bounds of the TIFILES file.

[+InsaneMultitasker]

Update 3.11 resolved the boundary condition error in my test program.  @9640News and I will test a bit more, but so far so good! 

[+InsaneMultitasker]

I re-checked routine >14 and added a test for >15; both are adhering to the same boundary condition.  No further concerns from my perspective.  @9640News please confirm at your convenience. 

 

Log report (two different files; one is an output file)

2023-05-02 22:27:04,700 LevelTwo    : INFO     direct output
2023-05-02 22:27:04,704 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 5
2023-05-02 22:27:04,705 tinames.tinames: INFO     TIPI.TIP.TIP -> /home/tipi/tipi_disk/TIP/TIP
2023-05-02 22:27:04,706 LevelTwo    : INFO     tifiles.getSectors: 5
2023-05-02 22:27:04,707 LevelTwo    : ERROR    request exceeds file size: t: 1408, s: 1408, e: 1664
 

2023-05-02 22:29:19,893 LevelTwo    : INFO     direct input
2023-05-02 22:29:19,896 LevelTwo    : INFO     unit: 0, blocks: 1, filename: TIP, startblock 5
2023-05-02 22:29:19,897 tinames.tinames: INFO     TIPI.2023.TIPS.TIP -> /home/tipi/tipi_disk/2023/TIPS/TIP
2023-05-02 22:29:19,898 LevelTwo    : ERROR    request exceeds file size: 1408, start: 1408, end: 1664
 

[+9640News]

@InsaneMultitasker,

 

I will try and do some testing this evening if I decide to forego some catch-up sleep.  Our friends/visitors are leaving this morning after their excursion to the Creation Museum on Monday and to the Ark yesterday.  Sunday evening I served up pecan smoked rat rolls, Monday they took us out for Mexican, and last night I served up venison and squirrel with some Fireball chasers.  They were not up for ground hog.  I think they should have some stories to tell their friends in Memphis when they get back home.

 

I should mention if it has not been mentioned elsewhere, Matt's update will allow the use of the COPY command from the MDOS prompt to/from the TIPI with MDOS 7.40 and above, but not with MDOS 7.30.  Insanemultitasker and myself are in the last stages of preparing for the next release of MDOS 7.40.

 

Beery

[+jedimatt42]

Update 3.12 - May 3rd, 2023

 

More support for native files. Native files, is the term I use for files on the linux filesystem that are not encased in a TIFILES container ( have no header that represents the spinny-disk FDR )

TIPI has for a while, recognized some file extensions to be text, and so treated them like D/V 80 files, with some limitations. If a native file didn't have a known text file type extension, then it would treat it like a D/F 128 file, per the ancient tradition.

 

Now direct-input is supported for native D/V 80 files, and D/F 128 files. direct-output is supported for native D/V 80 files. 

The current state of things is oriented toward being able to copy these implied file types from TIPI storage to other 4A media like your ramdisks and such. 

 

If you have a directory specified in the PI.CONFIG NATIVE_TEXT_DIRS setting, then you should be able to copy native text files in and out of TIPI as though they are D/V 80 files. 

 

I also fixed rename for native files as well. 

Edited May 5, 2023 by jedimatt42
clarity

[+9640News]

Thanks Matt.

[+dhe]

 

I made a backup of my existing tipi - in a geneve.

 

I downloaded the backup to my local drive:

  

 

I installed a tipi with the latest dsr and 3.1 image in the geneve.

 

I used PI.CONFIG, connected to the network, and did an update from 3.1 to 3.12.

 

I then accessed the webgui, uploaded my backup.

 

When I try to restore, I get:

  

 

The backup that I uploaded doesn't show up.

 

 

[+arcadeshopper]

a crdownload file isnt a valid backup its a partial file that wasn't completed

 

Greg

[+dhe]

So what would be the path forward to move files from old tipi to new tipi?

[+arcadeshopper]

20 minutes ago, dhe said:

So what would be the path forward to move files from old tipi to new tipi?

finish the backup before trying to restore?

[+9640News]

@dhe

 

On my Windows PC, I have a drive letter mapped to the TIPI.  I just highlight the TIPI and PDF folders and right click for COPY.

 

Then, I go to another drive, and create a folder with the date (20230505Geneve1), and paste the files into that folder.  Once the files are copied and the copying window disappears, then I am satisfied a copy was successfully made.  I am typically copying > 1GB of files, so it does take some time.

 

I keep several "2023xxxxGeneve1" folders on my destination drive so I can go back in time as needed.  I also have a 4TB drive I occasionally plug in that backs up several drives on my Windows system for a secondary backup.  When that backup is complete, I disconnect the drive to prevent the potential risk a ransomware virus could hit the computer.

 

 

[+jedimatt42]

There is a bug in the web-ui for the 2.x line, that shows the link to download the backup before it is done creating the backup archive... It should have had a label saying it was still processing, but the UX wasn't good... I have improved this in 3.x, but that doesn't help you much.  

 

If you still have the 2.x SD image you can try to download the backup again. If you don't then the backup archive you have is likely a truncated tar file. 

 

My restore code can be found here, https://github.com/jedimatt42/tipi/blob/af30fd0cd42a29d0053baf0394014f411fd8533a/setup/restore.sh#L33

 

Basically the tar is extracted into the /home/tipi directory with everything set to be owned by the tipi user. If you only have a partial tar file, you might be able to extract some of it by: cat backup.tar | tar -xvf - -C /home/tipi --owner=tipi

 

I would expect the last file listed as it extracts to be corrupt. It is hard to predict how much will be lost.

[+dhe]

I have separate cards with separate tipis and storage devices.

 Should I just rsync /home/tipi from one old device to new device?

 

 or hook up old device, click download backup and wait say 10 minutes?

[+dhe]

From what I recall, on old device.

 

Clicking backup - immediately returns a screen showing a generated filename, and multiples if backup done multiple times.

 

When I click, download backup, it happens immediately, I don't thing there is any other buttons to be pressed.... So I don't think I can wait...