Whtech site and wget/rsync

[FSword7]

Folks,

 

I tried to get some from whtech site with wget command.  wget can't download some files due to 404 errors in /Cassettes/rawWAVfiles folder.  Does anyone able get all wav files through wget command?

 

Someone told me that wget is not effective for mirroring because it does not support delete option.  He recommends rsync with --delete option is the most effective way for mirroring.  Rsync replaces old ftp protocol. 

 

I tried to rsync some from whtech but it did not response.  Does someone at whtech plan to implement rsync access on whtech site?

 

Thanks,

Tim

[gferluga]

Wget works for me:

 

localhost:~# wget "https://ftp.whtech.com/Cassettes/rawWAVfiles/3D-LABY%20(2).w
av"
--2023-12-03 18:35:17-- https://ftp.whtech.com/Cassettes/rawWAVfiles/3D-LABY (2).wav
Resolving ftp.whtech.com (ftp.whtech.com)... 148.135.75.22
Connecting to ftp.whtech.com (ftp.whtech.com)|148.135.75.22|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3986220 (3.8M) [audio/x-wav]
Saving to: '3D-LABY (2).wav'

3D-LABY (2).wav     100%[==================>]   3.80M   768KB/s    in 5.7s    

2023-12-03 18:35:23 (680 KB/s) - '3D-LABY (2).wav' saved [3986220/3986220]

localhost:~# 

 

[FSword7]

Ok, I executed it again, but I did not see 404 errors anymore. 

 

My wget command was: wget -m -np https://ftp.whtech.com/Cassettes/

 

[FSword7]

Well, when I tried execute 'wget -m -R "index.htm*" -nv https://ftp.whtech.com/',  I can only see some 404 errors but got all 60 GB stuff.

 

http://ftp.whtech.com/Cassettes/rawWAVfiles/2171¯D.wav:
2023-12-03 18:23:55 ERROR 404: Not Found.
2023-12-03 18:23:55 URL:http://ftp.whtech.com/Cassettes/rawWAVfiles/2171%c2%afD.
wav [955180/955180] -> "ftp.whtech.com/Cassettes/rawWAVfiles/2171¯D.wav" [1]
http://ftp.whtech.com/Cassettes/rawWAVfiles/2180¯D.wav:
2023-12-03 18:23:55 ERROR 404: Not Found.
2023-12-03 18:23:55 URL:http://ftp.whtech.com/Cassettes/rawWAVfiles/2180%c2%afD.
wav [5539628/5539628] -> "ftp.whtech.com/Cassettes/rawWAVfiles/2180¯D.wav" [1]

 

It showed 404 errors but successfully downloaded them. I think that filenames contain Unicode characters.  I checked downloaded files and see them. 

 

NTFS and ext4 supports Unicode characters in filenames. 

 

I recommend rsync over wget so that whtech need to enable rsync access.  With wget, I have to make a backup and download entire site each time because wget do not support delete option.  Rsync do support delete option so that you can update it each time to save file data transfers a lot and much faster. 

 

Tim