10 years ago I found there was little benefit for async i/o to a disk, even though it great for networking.
With SSD it coukd he different if you have a very long q depth, but he shows charts for copies over nfs where async i/o might find a lot of latency to hide.
The chart is showing data from an NFS copy, but the listed speeds in the blog and on github[1] are from a copy on a local SSD. That is a bit confusing though, maybe I should make it more clear. I used the network copy for ETA calculation because it was an easy way to make the transfer take longer - if the whole copy is only a few seconds long it's difficult to meaningfully compare ETA estimation accuracy between two approaches. I would like to have more performance data though.
I would love to see a comparison between scp and wcp over NFS, more specifically large single file sizes, if folks are looking to discern more performance parameters.
I would expect pretty much any plain copying program over NFS to beat scp, since scp has the overhead of encryption. On a slow disk with a fast CPU it might roughly match though.
6 comments
[ 3.6 ms ] story [ 28.3 ms ] thread10 years ago I found there was little benefit for async i/o to a disk, even though it great for networking.
With SSD it coukd he different if you have a very long q depth, but he shows charts for copies over nfs where async i/o might find a lot of latency to hide.
1: https://github.com/wheybags/wcp/#how-fast
wcp is good stuff!
0: https://github.com/facebook/wdt