HN Explorer

28 comments

[ 3.5 ms ] story [ 67.6 ms ] thread
crazy_eye
love pv, mostly use it for percentage bar on db restores:

pv -f *.sql.gz | zcat | mysql db_instance_name

[deleted]
(comment deleted)
mibbitier
How does it know the progress / ETA?

I was under the assumption that pipes do not know how much total data is on its way?

aidenn0
If you give it a file, or specify the amount on the commandline it knows, otherwise it doesn't
mbreese
Right. I usually use pv in place of 'cat' at the start of a long set of piped commands. Anywhere you could use 'cat' you can use pv. If I'm not mistaken, I think you can also use multiple 'pv' commands at a time too.
pronoiac
> If I'm not mistaken, I think you can also use multiple 'pv' commands at a time too.

I use multiple pv instances to track compression/decompression rates, and overall progress, for tarballs. Using the "-c" code for console codes keeps the instances from stomping over each other's output.

lloeki
This is indeed visible in the example, only source has progress while bzcat and gzip only have flow.
dbuthay
It's amazingly useful to have realtime throughput rates

tail -f query.log | pv -tri 1 > /dev/null

pramodbiligiri
Nifty command. Been using this for a month or so. One use case is measuring if a downstream process can consume data as fast the previous process is producing it (by alternating second process with /dev/null or some such).
spudlyo
It's also great that pv supports rate limiting, so you can choose if you want to saturate your network when you use it with netcat.
gouranga
Nice - that'll spice up my rather boring svnadmin dump sessions - thanks for posting!
ralph
Note, a plain dd will report progress on stderr if sent a SIGUSR1. That can sometimes be enough to avoid the overhead of pv(1) and sending huge amounts of data through another pipe.
protopete
About the pv overhead, I sent a note to the pv author saying that on Linux at least the transfer function could take advantage of the splice system call. Splice "copies" data from one file descriptor to another file descriptor, avoiding having to copy the data from the kernel to userspace and back.
[deleted]
(comment deleted)
Erwin
On FreeBSD (and maybe Mac OS X) you can press Control-T to send a SIGINFO signal (non-standard extension. This prints out how far dd has gotten (and perhaps other FreeBSD utilities too).
sneak
N.B. also that dd on osx will EXIT IMMEDIATELY if sent a SIGUSR1. It double-sucks because one tends to do this on long-running dd processes that you're wondering are done yet...
kilovoltaire
yes | pv > /dev/null
sp332 

  pv /dev/zero > /dev/null
kilovoltaire
awesome, i was actually wondering if there was a "yes-esque" (yesque?) /dev/ file.

my throughput from yes is 49MB/s whereas from /dev/zero it's 12GB/s

potatohead00
Found out about pv ~year ago when working with large log files. Very useful. I also like it for testing how quickly a script can consume data.
b0b0b0b
If I'm dealing with [compressed] files on disk or across a network, what's the practical overhead of pv? A few extra percent CPU, or some material decrease in through put?
blibble
very very little, it's just doing read(), counting the bytes, then write(), with some screen output every few seconds.

(hell it could even use splice() and never even see the bytes)