TIPI Usage and Support

[+jedimatt42]

29 minutes ago, dhe said:

From what I recall, on old device.

 

Clicking backup - immediately returns a screen showing a generated filename, and multiples if backup done multiple times.

 

When I click, download backup, it happens immediately, I don't thing there is any other buttons to be pressed.... So I don't think I can wait...

There was a little hint above the list of files, that shows 'Creating archive...' and then it'll refresh once a second, if the status is complete the 'Creating archive...' will go away... If it does not show the 'Creating archive' then it should be safe to download the backup file.

 

You can verify the downloaded file. It'll be called a .tar.gz, but your browser has probably uncompressed it ( another bug I fixed in 3.x ) so it is at that point just a .tar with a bad name. You can check the validity with a tar friendly tool like 7zip, or tar. 

In 3.3? I think, I fixed the download headers so browsers should stop un-gzipping the backups on download. And I fixed the restore to tolerate that a thing named .tar.gz might be just a tar, so that you can restore a good backup from 2.x to 3.x. 

 

 

 

 

[+dhe]

 

So I copied (scp) the backup file over to the ubuntu box..

05/05/2023  10:30 AM         9,820,160 tipibk.tar.gz
05/05/2023  10:30 AM         9,820,160 Unconfirmed 332369.crdownload

 

dhe@phish2:[/tmp/unarc]$file tipi.tar
tipi.tar: POSIX tar archive (GNU)

 

dhe@phish2:[/tmp/unarc]$tar -xvf tipi.tar

{large snip}

tipi_disk/MDOS65/RAVEFORM
tipi_disk/MDOS65/DU2M
tipi_disk/MDOS65/!UPDATE600
tipi_disk/MDOS65/MDOSDOCS
tipi_disk/USB
tipi.config
tipi.uuid
tmp_wpa_supplicant.conf
 

I changed the name to tipibk.tar.gz, click upload.

  It says it's uploading.

 

Clicking restore shows no options and doesn't do anything....

 

Please let me know what to look at.

 

Thanks!

[+dhe]

FWIW - when first installed (ie... I imaged the MicroSD card) - it was version 3.1.

When web interfaced started on 127.0.0.1.

 

But, I used TSTAT to upgrade to 3.12 - upon restart - The WebGUI was available.

 

I also upgrade the DSR version to the latest on the TIPI board - as sent to me by @arcadeshopper

 

So everything 'should' be minty and up2date!

[+9640News]

@jedimatt42

 

I believe I have found a bug with the new NATIVE_TEXT_DIRS command.

 

I had it set to a directory called MDOS.  

 

I was copying files from another directory on the TIPI to a directory called MDOS740DIS with the MDOS COPY command.  All files being copied are copied as native files. It looks like the string compare treats directories MDOS and MDOS740DIS the same .

 

I was able to create a directory called TEST.  I copied a program image file to it when the NATIVE_TEXT_DIRS was still defined as MDOS.  File copies fine.  I delete file, change the NATIVE_TEXT-DIRS command to TEST.  I rename the directory from TEST to TEST1.  File copies a DIS/VAR 80 file, 1 sector.

 

When the file copies bad, it looks to have no contents.  I have used both GDM2K and the MDOS COPY command with the same results.

 

 

[+jedimatt42]

1 hour ago, dhe said:

 

So I copied (scp) the backup file over to the ubuntu box..

05/05/2023  10:30 AM         9,820,160 tipibk.tar.gz
05/05/2023  10:30 AM         9,820,160 Unconfirmed 332369.crdownload

 

dhe@phish2:[/tmp/unarc]$file tipi.tar
tipi.tar: POSIX tar archive (GNU)

 

dhe@phish2:[/tmp/unarc]$tar -xvf tipi.tar

{large snip}

tipi_disk/MDOS65/RAVEFORM
tipi_disk/MDOS65/DU2M
tipi_disk/MDOS65/!UPDATE600
tipi_disk/MDOS65/MDOSDOCS
tipi_disk/USB
tipi.config
tipi.uuid
tmp_wpa_supplicant.conf
 

I changed the name to tipibk.tar.gz, click upload.

  It says it's uploading.

 

Clicking restore shows no options and doesn't do anything....

 

Please let me know what to look at.

 

Thanks!

ok, normally the backups show up in a list, and have a checkbox next to them you can select one, and then press restore. But since your upload was renamed first, you found another bug in my backup system... It only looks for backups that match this glob pattern: /home/tipi/tipi-backup*.tar.gz

 

( I suspect the upload succeeded )

so, you can ssh into the PI, and then rename /home/tipi/tipibk.tar.gz to /home/tipi/tipi-backup-one.tar.gz 

then refresh the page in the web-ui and it should list the backup.

then you should be able to check the box next to it, and then press 'Restore Backup' 

 

( Sorry this has been a hassle for you )

[+jedimatt42]

20 minutes ago, 9640News said:

@jedimatt42

 

I believe I have found a bug with the new NATIVE_TEXT_DIRS command.

 

I had it set to a directory called MDOS.  

 

I was copying files from another directory on the TIPI to a directory called MDOS740DIS with the MDOS COPY command.  All files being copied are copied as native files. It looks like the string compare treats directories MDOS and MDOS740DIS the same .

 

I was able to create a directory called TEST.  I copied a program image file to it when the NATIVE_TEXT_DIRS was still defined as MDOS.  File copies fine.  I delete file, change the NATIVE_TEXT-DIRS command to TEST.  I rename the directory from TEST to TEST1.  File copies a DIS/VAR 80 file, 1 sector.

 

When the file copies bad, it looks to have no contents.  I have used both GDM2K and the MDOS COPY command with the same results.

 

 

Ok, I see what I did wrong... with the implied subdirs logic. I'll work up a fix. 

[+9640News]

Just now, jedimatt42 said:

Ok, I see what I did wrong... with the implied subdirs logic. I'll work up a fix. 

Thanks.  Hopefully, it is pretty simple.

[+jedimatt42]

Update 3.13 - May 5th, 2023

 

- Fix Beery's report of NATIVE_TEXT_DIRS applying to siblings that start with the same letters as an entry in the NATIVE_TEXT_DIRS list.

[+jedimatt42]

43 minutes ago, jedimatt42 said:

ok, normally the backups show up in a list, and have a checkbox next to them you can select one, and then press restore. But since your upload was renamed first, you found another bug in my backup system... It only looks for backups that match this glob pattern: /home/tipi/tipi-backup*.tar.gz

 

( I suspect the upload succeeded )

so, you can ssh into the PI, and then rename /home/tipi/tipibk.tar.gz to /home/tipi/tipi-backup-one.tar.gz 

then refresh the page in the web-ui and it should list the backup.

then you should be able to check the box next to it, and then press 'Restore Backup' 

 

( Sorry this has been a hassle for you )

I should be able to fix it so that whatever you upload from this page is considered a backup, regardless of what you named it. 

[+InsaneMultitasker]

I've trying to pull some data using what to me should be a simple JSON query.  I can copy/paste the full result into the JMES tutorial and query it there, but I am having no success with XB.  The program halts with "IO Error 37 in 110", the PRINT #1 statement. The log shows 'illegal read mode' but the program never reaches the LINPUT.  If I comment out the PRINT statement, LINPUT generates an error as well.   Is this another gap in my JSON knowledge?

 

If I change the OPEN statement to "INPUT" mode and skip the PRINT, then LINPUT works and A$ contains all elements.  Short program and log in the spoiler.  Thank you.

Spoiler

1 !SAVE DSK8.LONGURL 
99 GOSUB 300
100 OPEN #1:"URI1.2023-05-04.json",DISPLAY ,VARIABLE 254
110 PRINT #1:"id"
120 LINPUT #1:A$ :: PRINT A$
200 CLOSE #1
299 END
300 OPEN #1:"PI.CONFIG",OUTPUT :: PRINT #1:"URI1=?J.HTTPS://www.nytimes.com/svc/wordle/v2" :: CLOSE #1 
301 RETURN

=-=-=-=-=-=-=-=-=-=-=-=-=-

If I change to INPUT mode and comment out line 110, A$ contains all elements: 

{"id":2121,"solution":"guppy","print_date":"2023-05-04","days_since_launch":684,"editor":"Tracy Bennett"}

 

-=-=-=-=-=-=-=-=-=-=-=-=-=
2023-05-05 22:56:37,157 Pab         : INFO     opcode: Close, fileType: Sequential, mode: Update, dataType: Display, recordType: Variable, recordLength: 254, recordNumber: 0
2023-05-05 22:56:37,158 CurlFile    : INFO     close devname - PI.?J.HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 22:56:37,234 Pab         : INFO     opcode: Open, fileType: Sequential, mode: Update, dataType: Display, recordType: Variable, recordLength: 254, recordNumber: 0
2023-05-05 22:56:37,235 CurlFile    : INFO     open devname - PI.?J.HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 22:56:37,236 CurlFile    : INFO     url: HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 22:56:37,236 CurlFile    : INFO     cmd: /usr/bin/wget --user-agent=ti994a-Tipi/3.13 -O /tmp/CF 'HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json'
2023-05-05 22:56:37,491 ti_files.NativeFile: INFO     loading as native file: HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 22:56:37,492 ti_files.NativeFile: INFO     loaded 1 lines
2023-05-05 22:56:37,493 ti_files.NativeFile: INFO     records: [bytearray(b'{"id":2121,"solution":"guppy","print_date":"2023-05-04","days_since_launch":684,"editor":"Tracy Bennett"}')]
2023-05-05 22:56:37,531 Pab         : INFO     opcode: Write, fileType: Sequential, mode: Update, dataType: Display, recordType: Variable, recordLength: 254, recordNumber: 0
2023-05-05 22:56:37,532 CurlFile    : INFO     write devname - PI.?J.HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 22:56:37,532 CurlFile    : ERROR    illegal read mode for PI.?J.HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json

 

=-=-=-=-=-

Log when I use INPUT mode and simply LINPUT A$: 

2023-05-05 23:09:11,541 ti_files.NativeFile: INFO     loading as native file: HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 23:09:11,543 ti_files.NativeFile: INFO     loaded 1 lines
2023-05-05 23:09:11,544 ti_files.NativeFile: INFO     records: [bytearray(b'{"id":2121,"solution":"guppy","print_date":"2023-05-04","days_since_launch":684,"editor":"Tracy Bennett"}')]
2023-05-05 23:09:11,574 Pab         : INFO     opcode: Read, fileType: Sequential, mode: Input, dataType: Display, recordType: Variable, recordLength: 254, recordNumber: 0
2023-05-05 23:09:11,575 CurlFile    : INFO     read devname - PI.?J.HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
2023-05-05 23:09:11,595 CurlFile    : INFO     record bytes: 105
2023-05-05 23:09:11,799 Pab         : INFO     opcode: Close, fileType: Sequential, mode: Input, dataType: Display, recordType: Variable, recordLength: 254, recordNumber: 0
2023-05-05 23:09:11,799 CurlFile    : INFO     close devname - PI.?J.HTTPS://www.nytimes.com/svc/wordle/v2/2023-05-04.json
 

 

 

 

[+jedimatt42]

@InsaneMultitasker your expectations around JSON are fine. But I broke something in the detection of the ?J flag when I extended handling of the NATIVE_TEXT_DIRS... 

 

One difference in how you are using this though... I would expect the ?J to be used with the device name... so URI1.?J.stuff   and not placing ?J in the beginning of the URI1 value. 

 

So, right now, the ?J is having no effect, and you are getting the old read-only non-json behavior that you would have seen before I implemented ?J... 

 

I have a fix, after more testing, will release in the morning.

[+dhe]

@jedimatt42 closer...

 

 I followed the instructions in

12 hours ago, jedimatt42 said:

rename /home/tipi/tipibk.tar.gz to /home/tipi/tipi-backup-one.tar.gz 

 

Here is what the web software is telling me....

 

[+jedimatt42]

1 hour ago, dhe said:

@jedimatt42 closer...

 

 I followed the instructions in

 

Here is what the web software is telling me....

 

 

Ok, another fragile decision I made. it expects the file name to have a more specific format in another place ( probably cause I expected the os file timestamp to be unreliable when copied from machine to machine ) 

 

Try renaming it to have the time stamp in the name, here is a good backup file name: tipi-backup-2023-04-01T114801-0700.tar.gz

 

[+dhe]

@jedimatt42

 

Victory Lap Time!

 

 

Many Thanks!

[+dhe]

Now that we have a restore, what's the first function?

Why a backup of course!

 

It looks like gzip basically cuts the size in half as the file uploaded was inconveniently gunzipped by my browser. 😃

 

[+jedimatt42]

1 hour ago, dhe said:

Now that we have a restore, what's the first function?

Why a backup of course!

 

It looks like gzip basically cuts the size in half as the file uploaded was inconveniently gunzipped by my browser. 😃

 

If you download the new backup now from 3.x it should set the response headers correctly to instruct the browser to leave it be. 

[+jedimatt42]

Update 3.14 - May 6th 2023

 

- Fix broken PROTECT bit after recent native file handling changes ( you cannot protect a native file, but any TIFILES file should support it )

- Fix broken JSON file handling with URI drive aliases. The ".?" modifier needs to stay with the device name you are using.. URI1.?J.request.json, or PI.?J.HTTPS://something.crazy.hostedby.com/request.json, it should not be in the value of the URI alias mapping.

- Make restore more tolerant of files being renamed, nolonger crash the webui if they don't have the timestamp and perfect name... they still have to be named ____.tar.gz 

- Add some archive validation before extraction when the restore button is hit.

- Add restore completion status to the web-ui

[+InsaneMultitasker]

@jedimatt42 - I updated from 3.13 to 3.14, moved the modifier from the alias mapping "URIx=value" to the open statement, e.g., OPEN #1:"URI1.?J.request.json", and was able to successfully retrieve information via JSON query.    

[+9640News]

On 5/3/2023 at 10:14 PM, jedimatt42 said:

Update 3.12 - May 3rd, 2023

 

More support for native files. Native files, is the term I use for files on the linux filesystem that are not encased in a TIFILES container ( have no header that represents the spinny-disk FDR )

TIPI has for a while, recognized some file extensions to be text, and so treated them like D/V 80 files, with some limitations. If a native file didn't have a known text file type extension, then it would treat it like a D/F 128 file, per the ancient tradition.

 

Now direct-input is supported for native D/V 80 files, and D/F 128 files. direct-output is supported for native D/V 80 files. 

The current state of things is oriented toward being able to copy these implied file types from TIPI storage to other 4A media like your ramdisks and such. 

 

If you have a directory specified in the PI.CONFIG NATIVE_TEXT_DIRS setting, then you should be able to copy native text files in and out of TIPI as though they are D/V 80 files. 

 

I also fixed rename for native files as well. 

Matt,

 

I need some clarification based on this post. 

 

If I have a program image file, and I copy to a NATIVE_TEXT_DIRS folder, what should or should not happen.  Since it is not a D/V 80 file, should it copy over as a PROGRAM image file, or should it attempt to convert the file?  In this case for the contents of the file, I only get about 1 sector's worth of content.

 

 

[dgrissom]

3 hours ago, jedimatt42 said:

Update 3.14 - May 6th 2023

 

- Fix broken PROTECT bit after recent native file handling changes ( you cannot protect a native file, but any TIFILES file should support it )

- Fix broken JSON file handling with URI drive aliases. The ".?" modifier needs to stay with the device name you are using.. URI1.?J.request.json, or PI.?J.HTTPS://something.crazy.hostedby.com/request.json, it should not be in the value of the URI alias mapping.

- Make restore more tolerant of files being renamed, nolonger crash the webui if they don't have the timestamp and perfect name... they still have to be named ____.tar.gz 

- Add some archive validation before extraction when the restore button is hit.

- Add restore completion status to the web-ui

So you've finally updated the PI on the tiPI to PI!

May the blessing of PI be on this version!  😉

 

[+jedimatt42]

52 minutes ago, 9640News said:

Matt,

 

I need some clarification based on this post. 

 

If I have a program image file, and I copy to a NATIVE_TEXT_DIRS folder, what should or should not happen.  Since it is not a D/V 80 file, should it copy over as a PROGRAM image file, or should it attempt to convert the file?  In this case for the contents of the file, I only get about 1 sector's worth of content.

 

 

Oh, I tested SAVE to a NATIVE_TEXT_DIRS entry, but not copy... there is something wrong for sure.. 

 

The goal is only DISPLAY VARIABLE 80 files should get written as native text files on direct-output. 

[+jedimatt42]

20 minutes ago, jedimatt42 said:

Oh, I tested SAVE to a NATIVE_TEXT_DIRS entry, but not copy... there is something wrong for sure.. 

 

The goal is only DISPLAY VARIABLE 80 files should get written as native text files on direct-output. 

Ok, oddly enouugh, this is what that off-by-one error I recently fixed in direct-input was about.  

[+dhe]

1 minute ago, jedimatt42 said:

off-by-one error

 

Same as a fence post error?

[+jedimatt42]

Update 3.15 - May 6th 2023 ( round two / cause all good pies are quickly consumed. )

 

- Fix direct-input limit comparisons, again. Corrects the issue I can observe in ForceCommand with copying a PROGAM image into a directory marked as NATIVE_TEXT_DIRS... ( or anywhere for that matter )

[+jedimatt42]

25 minutes ago, dhe said:

 

Same as a fence post error?

Now that I read the comp-sci definitions, I guess it wasn't really an off by one, but a failure to round up. Originally, (earlier this week), I had code that was always rounding up to the next block size, even if there was no usage in that last block. I think it is now properly conditional.