"cat foo > foo" breaks, because the shell truncates the file "foo" while it's setting up the redirection, before it launches "cat foo", and so cat has nothing to read.
"cat foo | sponge foo" works because the shell is not responsible for writing to foo; it launches cat (which opens the file for reading) then launches sponge (which eventually opens the file for writing).
I would thus expect "cat foo | tac | tac > foo" to fail in the same way as "cat foo > foo" because the shell is still going to open the file foo and truncate it, but experimentation shows that it does actually work. Is it because the shell launches each pipeline one at a time, so cat has read the file before "tac > foo" truncates it? Is it a race-condition or a corner-case?
That's a good question. I had though it worked because the truncate would not actually happen until tac actually wrote something, but now that I look at it more I don't think that is actually the case. `cat > foo` immediately truncates foo before anything is written. Also `tac foo > foo` doesn't work, which discredits my hypothesis that tac's buffering was causing it.
Furthermore `cat foo | cat > foo` does not work, but `cat foo | cat | cat > foo` does work.
I suspect this behavior is actually a race condition of some sort.
Edit: definetly a race condition, shown by larger files:
Yes, either type G when presented with the top of the file or pass +G on the command line. It will then read all the intervening file to calculate the line numbers, and tells you it's doing this and to interrupt it to stop. To prevent it doing this in the first place use the -n option.
"less -n huge_file" and then G only reads the start and end of the file. Type G again to move to the latest end of the file if it's growing.
> less. Overkill, but SO MUCH BETTER THAN tail -f. Seriously.
If you do this on my servers, I am going to find you, and then I am going to kill... um... your process.
less with SHIFT-F takes a whole CPU core to run on a busy log file. tail -f takes almost nothing to run. Now combine that information with the reality of a whole bunch of devs who don't know it or don't care about it. :(
multitail[1] is also really nice for following multiple log files. It can do highlighting of lines based on regex's, and has built-in highlighting of a wide variety of log formats
Does anybody know if there is a good standalone regex highlighter? I usually use `ack --passthru`, but that has always seemed a tad.. overkillish to me.
The former will spot inode changes and reopen the filename if the file wraps (or has something else done to it to change its inode). The latter will just sit there seeing no new input in such cases.
When rewriting large amounts of ASCII text (or comments in Code), I really find "par" helpful.
: Just use it in :
: vi with your cursor in front of a badly wrapped block of text. :
: Press !}par<enter> and the text will be :
: piped to par :
: which fixes up :
: the :
: formatting and keeps aligned frame markers :
: intact magically. :
: Just use it in vi with your cursor in front of a badly wrapped :
: block of text. Press !}par<enter> and the text will be piped :
: to par which fixes up the formatting and keeps aligned frame :
: markers intact magically. :
Or you can just select the block of text and type gq. This will do what par does, and can be adjusted with vim's 'textwidth' and 'formatoptions' settings.
I forgot about the DNS querying of wikipedia. Probably could've used that to entertain myself on my flights over the holidays since I'm pretty sure gogo inflight passes DNS through.
Anyway, I'm a big fan of piping things into xargs. xxd is a terminal-based hex editor. It can convert both to and from hex to binary. I also use 'pgrep -lf 'partial_program_name' a lot in place of 'ps aux | grep partial_program_name'. 'pkill -9 partial_program_name' searches for and kills all processes matching the string.
Faucet is part of "netpipes", which is available for most Unixoid operating systems. I guess it was inspiration to build "netcat". But netcat was a absolute terrible hack of a program, so "socat" (socket-cat?) was born.
In case you need the functionality of bash's /dev/tcp [1] and you are pondering to install netcat or netpipes, do yourself a favor and just go for socat, it's insanely more powerful, read the manpage, read the examples!
function lc() { if [[ "$#" -gt 1 ]]; then for DIR in "$@"; do echo -n "$DIR - " ; ls -AU1 $DIR | wc -l ; done ; else ls -AU1 "$@" | wc -l ; fi; }
So, "lc /dir" will count the number of files in /dir and "lc /dir/*" will count the files in subdirectories of /dir. This is useful if you're working in an environment where you may have thousands of files in a directory, and a regular "ls -l" will lock your terminal while it eats your entire scrollback buffer.
Here's an idea: always pipe ls throug a pager! If it's less than a page, it would just print the results (you have to set some flags on less that I don't remember). If it's more, then press d or /term-to-search :)
I second this, I also find myself impulsively smashing enter multiple times sometimes between commands, if I am typing in a series of commands, its just so my brain can catch up with what I want to do.
That's gold. I've been using "clear" this whole time and hated it, particularly since I'm used to "cls" from MS-DOS. So, for me, the way to clear the screen was:
cls <enter>
clear<enter>
lol.
I see Cmd + K on the menu now, but I guess I wasn't curious enough to poke around. Thanks a bunch.
You can use this within Vim to pretty-format text tables, initialization of variables, etc. Just highlight the lines in question in visual line mode (V) and type
always worth looking at these types of lists imo, whilst some of the commands mightn't really fit into your workflow or seem useful immediately, they're often exactly the snippets of information that can save hours at a later stage..
Not really a unix command, but a vi command that has been really useful, specially if you do lots of editing protected files and want to keep your custom vim configuration:
have over doing it at the command line other than filling in the current filename for you? I suspect the original poster meant something like "gg!Gsudo tee %".
Along the lines of magic shell variables, there's also '!$', which selects the last argument of the previous command. Often I'm looking at different aspects of the same file:
I don't find this one useful. [up-arrow] automatically inserts the last command, which gives a generic mechanism to correct the command and it requires no more typing than !!: [up-arrow][control-a]sudo [enter]
grep -r . "some random debug message"
(searches for the passage recursively in all files from the directory it was executed, the main reason I like developing in linux more than windows)
If you are searching inside code specifically, you may like 'ack'. It does the same thing grep does, except it has preset 'excludes', is recursive by default, and the output is a bit prettier.
It took me a moment to figure out what was going on there (with the whole clippy thing). The bulk of this list is the top commands from commandlinefu which has been on HN before [0], [1].
You can even go meta and install a command to let you search commandlinefu [2].
I have a suspicion that it's actually a kind of targeted example-advertisement for the clippy thingo ;) and did somewhat work for me actually, I do now feel interested in learning more about the site
Very useful, if you want to make a checkpoint (possibly for /), break things and then get back to the original state.
If you'd do that for / , your own system would start behaving, like a system booted from LiveCD. And, after reboot, would return to the origanal state.
Overlayfs is pretty cool actually. Allows you to overlay a lower directory with some other upper directory and mount the result somewhere. All the changes that you are making in the mounted directory would be visible to you, and would be stored in the upper directory.
In the example above, after running this command, content of your $HOME would stop changing. But you still will be able to change it, with all the changed content going into tmpHome.
I disagree. I think typing one more command "sudo !!" is much easier and straight-forward than doing
1. Ctrl+p. Go to the previous line in bash history.
2. Ctrl+a. Go to the beginning of the new line.
3. Type "sudo ". While you're typing your whole commandline shifts around as well.
The latter mixes history commands, navigation commands and also modifies a previous command instead of just typing a new 7-letter command. It's also more in terms of keystrokes (though just barely).
You know writing out their function does not make them longer. ;) I agree with the parent because my left pinky is always on capsLock (my ctrl; I swapped caps with ctrl) so a ctrol ap is far easier than to go to shift + 1 twice(to type !). Again, this can be different in other keyboards/setups.
My favorite thing about `sudo !!` is that when you IM it to your friend who's learning Unix they'll take that literally rather than realize they should type it into their terminal, leading to a hilarious teachable moment months later when they actually watch you using the command line.
134 comments
[ 3.0 ms ] story [ 228 ms ] threadSpong 'soaks up' input an releases it all on one chunk so that you can do things like:
(`cat foo > foo` does not work)That's neat and all, but what if you don't have sponge? Well, just use tac twice!
(Obviously this is rather wasteful ;))"cat foo > foo" breaks, because the shell truncates the file "foo" while it's setting up the redirection, before it launches "cat foo", and so cat has nothing to read.
"cat foo | sponge foo" works because the shell is not responsible for writing to foo; it launches cat (which opens the file for reading) then launches sponge (which eventually opens the file for writing).
I would thus expect "cat foo | tac | tac > foo" to fail in the same way as "cat foo > foo" because the shell is still going to open the file foo and truncate it, but experimentation shows that it does actually work. Is it because the shell launches each pipeline one at a time, so cat has read the file before "tac > foo" truncates it? Is it a race-condition or a corner-case?
Furthermore `cat foo | cat > foo` does not work, but `cat foo | cat | cat > foo` does work.
I suspect this behavior is actually a race condition of some sort.
Edit: definetly a race condition, shown by larger files:
$ find . -type d -maxdepth 1
% ls -d */
find -maxdepth 1 -type d
I know, it's a bad habit
less. Overkill, but SO MUCH BETTER THAN tail -f. Seriously. You can toggle "follow mode" without losing your place in the incoming stream.
"less -n huge_file" and then G only reads the start and end of the file. Type G again to move to the latest end of the file if it's growing.
If you do this on my servers, I am going to find you, and then I am going to kill... um... your process.
less with SHIFT-F takes a whole CPU core to run on a busy log file. tail -f takes almost nothing to run. Now combine that information with the reality of a whole bunch of devs who don't know it or don't care about it. :(
[1] http://www.vanheusden.com/multitail/features.php
https://launchpad.net/byobu
https://www.google.de/search?q=byobu+tmux&tbm=isch
It's a fancy statusbar that shows CPU speed, IP-address, hostname, ...
[1] https://github.com/erikw/tmux-powerline
Screenshot: http://i.imgur.com/u6JY0.png
Config file: https://github.com/tylerkahn/dotfiles/blob/master/.tmux.conf...
Works with OSX and Linux.
Ensure you overwrite a file (or aren't):
Anyway, I'm a big fan of piping things into xargs. xxd is a terminal-based hex editor. It can convert both to and from hex to binary. I also use 'pgrep -lf 'partial_program_name' a lot in place of 'ps aux | grep partial_program_name'. 'pkill -9 partial_program_name' searches for and kills all processes matching the string.
I run an iodine server for myself, you can get free Internet at quite a few loginpage WiFi routers with it in airports, cafes etc.
Could it turn your old android device into an HTTP tunnel?
This is useful for creating a connection for receiving the input of what you get...
e.g. running
faucet 80 --in cat
and later
curl http://127.0.0.1:80
will deliver to you console :
GET / HTTP/1.1 User-Agent: curl/7.21.4 (i686-pc-linux-gnu) libcurl/7.21.4 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.19 Host: 127.0.0.1:900 Accept: /
Faucet is part of "netpipes", which is available for most Unixoid operating systems. I guess it was inspiration to build "netcat". But netcat was a absolute terrible hack of a program, so "socat" (socket-cat?) was born.
http://www.dest-unreach.org/socat/
In case you need the functionality of bash's /dev/tcp [1] and you are pondering to install netcat or netpipes, do yourself a favor and just go for socat, it's insanely more powerful, read the manpage, read the examples!
http://www.dest-unreach.org/socat/doc/socat.html#EXAMPLES
--
[1] but debian and ubuntu have decided to compile without it... https://bugs.launchpad.net/ubuntu/+source/bash/+bug/215034
function lc() { if [[ "$#" -gt 1 ]]; then for DIR in "$@"; do echo -n "$DIR - " ; ls -AU1 $DIR | wc -l ; done ; else ls -AU1 "$@" | wc -l ; fi; }
So, "lc /dir" will count the number of files in /dir and "lc /dir/*" will count the files in subdirectories of /dir. This is useful if you're working in an environment where you may have thousands of files in a directory, and a regular "ls -l" will lock your terminal while it eats your entire scrollback buffer.
> lc /dir" will count the number of files in /dir
> and "lc /dir/*" will count the files in subdirectories of /dir.Does anyone know if there's a key command for clearing the scroll buffer?
Ctrl+D for quick logout too.
cls <enter> clear<enter>
lol.
I see Cmd + K on the menu now, but I guess I wasn't curious enough to poke around. Thanks a bunch.
You can use this within Vim to pretty-format text tables, initialization of variables, etc. Just highlight the lines in question in visual line mode (V) and type
to pipe the lines through the column command.always worth looking at these types of lists imo, whilst some of the commands mightn't really fit into your workflow or seem useful immediately, they're often exactly the snippets of information that can save hours at a later stage..
!sudo tee %
:%!sudo tee %
Instant favorite
It is not a drop-in replacement for ack. e.g. there are some basic usages it does not support -- iirc counts (ag -c pattern).
findstr /SPINC:"some random debug message" *
Will do the same. FINDSTR is no grep, but for very simple cases like this is just fine
SPINC and SPIMC are my favorite options...
You can even go meta and install a command to let you search commandlinefu [2].
[0] http://news.ycombinator.com/item?id=527486
[1] http://news.ycombinator.com/item?id=3843373
[2] http://samirahmed.github.com/fu/
Great list of commands though.
(http://www.unixhub.com/docs/hpux/hpux_boot.html)
I wish I still had that machine, but I do not.
This lets me cd five directories at a time, and if I want to go back, I do a "popd"
If you'd do that for / , your own system would start behaving, like a system booted from LiveCD. And, after reboot, would return to the origanal state.
Overlayfs is pretty cool actually. Allows you to overlay a lower directory with some other upper directory and mount the result somewhere. All the changes that you are making in the mounted directory would be visible to you, and would be stored in the upper directory.
In the example above, after running this command, content of your $HOME would stop changing. But you still will be able to change it, with all the changed content going into tmpHome.
1. Ctrl+p. Go to the previous line in bash history.
2. Ctrl+a. Go to the beginning of the new line.
3. Type "sudo ". While you're typing your whole commandline shifts around as well.
The latter mixes history commands, navigation commands and also modifies a previous command instead of just typing a new 7-letter command. It's also more in terms of keystrokes (though just barely).
http://en.wikipedia.org/wiki/Magic_SysRq_key
zsh's file globbing is awesome. Some simple ones: