HN Explorer

Ask HN: Moving buku files from one remote server to another?

1 points by gaoshan ↗ HN
I'm about to move many GB (about 11GB) of files from one unix based host to another.

Here is how: 1. SSH into old. 2. tar and gzip everything (to save on bandwidth and time in transfer). 3. SSH into new. 4. use scp to move this tar/gzipped file 5. gunzip the file 6. Profit (well, be done anyway)

Can anyone tell me if there is a better (more bandwidth efficient, faster, safer, smarter, etc.) way than this?

5 comments

[ 3.1 ms ] story [ 23.4 ms ] thread
evdawg
rsync... it will basically do everything you just listed for you, while making sure everything gets over from server to server OK with multiple checks. If your transfer fails halfway through, you can just pick up where you left off.

I would not recommend gzipping 11GB of files into one archive... it will take hours to compress/unzip.

Good luck!

gaoshan
Doh! I know rsync... perfect. Thanks for the pointer.
gaoshan
In case anyone ever references this post here is the command I used:

  rsync -avz --progress --exclude-from=exclude   username@some.location.com:/files/to/copy/ /place/to/put/them/
The exclude file is a plain list of several items to be ignored and they are just listed by name 1 per line. If you want to do a "dry run" on what will happen add a -n to the arguments ( -navz instead of -avz).