Ask HN: Best Unix Tricks?

83 points by scottjackson ↗ HN
HN, what are your killer Unix tips? I've been using and learning about Unix both at university and at home for almost a year, and I was wondering -- what are the things that, when some wizard showed them to you, you wrote down as soon as you could so you wouldn't forget them? Is there some trick that saves you a buttload of time? Some script that makes your job way easier? Some command that made you smack your forehead in frustration when you found out it could be that simple?

Note: I'm personally looking for tips for working in bash (specifically OS X stuff if that makes any difference), but if there's some amazing thing that only works in zsh under Debian or something, feel free to suggest those too -- it's sure to be useful for someone.

I had a quick search for previous topics on this, but all I found was http://news.ycombinator.com/item?id=103725 which had a couple of badass things in the discussion.

106 comments

[ 2.9 ms ] story [ 193 ms ] thread
It's a great kill-ur unix tip but you should never fall for it:

    :(){ :|:& };
Oh, the temptation... if you're thinking of cutting & pasting that into the shell of your machine google for 'bash fork bomb' first, I've left off the final ':' to set it off.

Anyway, on a more serious note, after an installation procedure that was a lot of work, if you want to document the whole thing use:

    history | cut -d ' ' -f 5-
To give you all the commands you executed in the last little while, cut & paste that and you have a pretty good start for a how-to.

It's not a 'great killer unix tip', but it works for me.

Edit: Another one just came to mind, the ps command has a very useful option that is quite undervalued imo:

    ps ax --forest
Further to filtering the history: I use "script" to record terminal sessions. See "man 1 script".
Up-modded. But also wanted to add a spot of thanks for the bash fork bomb tidbit. That was beautiful :)
I usually use ps -auxf for the best output on linux. unfortunately the OSX version of ps doesn't support foresting, so sad.

  !!:s/sl/ls
executes last command but replaces first occurence of sl with ls. (can also be written as ^sl^ls)

  !!:gs/sl/ls
same as above, but now global replace
I have that one aliased as sl2ls ...
I alias maek to make
Why wouldn't you just alias sl to ls? It'd be much less typing.
Similarly, !! refers to the last command, so you can write "sudo !!" to execute the last command as root. Helpful for apt-get, among other things.

!$ refers to the arg to the last command, so you can write:

  $ ls -l /foo/bar
    # .oO(aha, it's a stale link i don't want)
  $ rm !$
  rm /foo/bar
  $
ctrl + r

Starts a search through your history file.

And then ^o to execute that command, and put the next command from your history onto the command line ready for another ^o...
Thank you so much! No more `history | grep cmd` %)
Map <up> and <down> to do the same in .zsh:

  # search history on <up> key instead of paging previous commands:
  bindkey "\e[A" history-search-backward
  bindkey "\e[B" history-search-forward
Or in bash, using .inputrc (readline):

  "\e[A": history-search-backward
  "\e[B": history-search-forward
and crl+r again for the next match. The ctrl+o tip didn't work for me though (OS X).
(comment deleted)

  set -o vi # vi like behaviour of the command line.
Use "/" to search through the history, "y" to copy, "p" to paste etc..

  cat > somefile.txt <<EOF # quickly creates a text file
Type EOF in a new line to finish.
Or just omit the <<EOF and press Control-D to finish (assuming you have not done anything strange with stty).
touch somefile.txt
His "quickly create a file" also adds text to it.
This is more a general hint than a single tip. Years ago I tried to improve my Unix skills by doing everything at home from the command line -- burning CDs, playing music, copying trees etc. I learnt a lot but this exercise was much less convenient and more mentally demanding than using a gui. Recently I have become much more lazy.
Funny, we're pretty much opposite.

I see my 'window manager' as a neccesary evil to run browser and email clients, everything else runs in terminal windows.

Of course you could use 'pine' and 'lynx' but that gets old pretty quickly.

s/pine/mutt/ && s/lynx/w3m/ : these get old much less quickly, though YMMV.
My tip: pushd without an arg pushes the current directory onto the stack, and then switches to what popd would have. Hence, repeated invocations of pushd let you cycle between two directories without thinking.

$CDPATH is also amusing:

  $ mkdir /foo/bar/baz
  $ cd baz
  bash: cd: baz: no such file or directory
  $ export CDPATH="/foo/bar"
  $ cd baz
  $ pwd
  /foo/bar/baz
could you give an example of the pushd thing you mentioned?

    $ pwd
    /home/jrockway
    $ pushd foo
    $ pwd
    /home/jrockway/foo
    $ popd
    $ pwd
    /home/jrockway
    $ popd
    error: directory stack empty
    $ pushd
    error: directory stack empty
    $ pushd foo
    $ pwd
    /home/jrockway/foo
    $ pushd
    /home/jrockway
    $ pushd 
    /home/jrockway/foo
    $ pushd
    /home/jrockway
    ...
cd - (Go back to previous working directory)

sudo !! (Run the last command as root)

:w !sudo tee % (Save a readonly file in VIM)

> tmp.file (Empty a file)

cmd !$ (Run cmd with previous commands arguments)

They are just some of my favorites.

That should be cmd !*. !$ would only be the final argument to the previous command, which you can also insert with alt-. (and pressing alt-. again will cycle through final arguments to earlier commands).

Adding :p to the end of a history expansion (like !$:p) will cause it not to be executed, but the expanded command line will be echoed and put in your history so you can check edit it.

M-. also takes a numeric argument (M-number or ESC number) to yank the nth word instead of the last.
zsh tab-expands history expansions, so you can start editing right there
I think :w !sudo dd of=% is better. Can't remember why, though.
What is the difference with :w! ?
mkdir -p (create a chain of directories in one call)
Excellent list. I use all of those except the last, which is new to me and extremely useful--so, thanks! It's now aliased.

I find myself using the first two on an almost hourly basis.

I love the '-exec' flag for find
Me too. Find also has -delete, so you don't have to type -exec rm -rf '{}' ';'
It's worth noting that -delete is more efficient than the above, and indeed safer.

Efficient because additional exec's are removed, for the most part find can perform the unlink itself. Safer because a replacement version for rm cannot be inserted into the user's path.

Before -delete was added to find I always found perl faster:

  find2perl scratch -type f -size 0 -eval 'unlink' | perl
or

  find2perl scratch -type f -size 0 -exec rm {} \; | perl
The last case does not actually call rm but is hard-wired to use unlink().

It may still be faster.

...except, that you should rarely use it. :)

It is much, much faster to pipe commands to "xargs". For instance:

    find . -name core -print0 | xargs -0 rm -f
Since "find" doesn't know when your command is multi-file, it has to run the command once per file; whereas, "xargs" can run it as few times as necessary. It works well for any command that accepts "a list of files".

In the example above, the -print0/-0 are a safety net to null-delimit filenames so that it doesn't matter if any of the paths contains a space.

Also the -n and -P options to xargs. So you would do something like

    $ find . -name 'blah*' |wc -l
    2000
    $ find . -name 'blah*' |xargs -n 250 -P 8 ./mangle
That'll spawn 8 copies of mangle, each with 250 blah files on its command line.
...except, when I want to execute a command once per file. :-)

If I want this I have the choice between: '-exec sed -i s/a/b/g {} \;' in find or '| xargs -IX sed -i s/a/b/g X'

    > make me a sandwich
    what? make it yourself
    > sudo !!
    okay
screen is infinitely useful.
Yes, it is. I recommend remapping the standard C-a to C-z, as C-a acts as "jump to the beginning of line" in couple of shells, Emacs and Emacs clones.
I use ctrl-_ (aka ctrl-7).
I'm not sure that C-z is a great choice as it's often used for job control. I use C-o myself. I suppose you could always use C-z C-z if you wanted to suspend your current foreground process.
I settled with C-z as it is easy and fast to type with one hand. However I am occasionally too fast and hit C-z z instead of C-z C-z. 'fg' solves this.
as an emacs junkie I use backtick for my screen control. Works fine till you try to paste blocks of SQL containing loads of said backticks - easy way to fubar a screen session.

Perhaps its emacs' fault but finding an easy to use key or key combo that isn't already in use is far too hard!

tmux is good as well. Slightly different feature set, and its cleaner codebase means that it's more likely to get features that have been on screen's todo list for ages.

http://tmux.sourceforge.net/

On OS X pbcopy and pbpaste are useful for working with clipboard data. Copy data from a web-page or whatever, manipulate it on the command-line to get what you want. Putting reasonably large files in the clipboard can be useful too.

For example:

  curl http://news.bbc.co.uk/ | pbcopy # copies a web-page to clipboard
Depending on what you are doing it can be very useful:

  pbpaste | sort | pbcopy # sort the contents of the clipboard.
Another useful feature of OS X is open, which uses the finder to open a file:

  open file.pdf

  open music.mp3
The equivalent Linux commands are:

    # xclip:
    curl http://example.org | xclip
And instead of "open", you can use:

    xdg-open file.pdf
    xdg-open music.mpe
(or even gnome-open if you're on gnome) will open up the file with their default handler
And "open" works on bundles, allowing you to avoid descending into the /Contents/... part.

    open /Applications/FooBar.app
open also takes the -a argument to specify which application to use. It's very handy for things like this:

    alias shop="open -a Adobe\ Photoshop\ CS3"
You can also specify the app to open using the -a flag and specifying the full name of the app (with correct spacing and capitalization), e.g.:

    open -a 'TextEdit' foobar.txt
    open -a 'TextMate' foobar.txt
I also find it useful to have multiple instances of certain applications sometimes:

  open -na safari
open -a finder <dir> is a pretty handy way to open a finder window to current directory.
You can just pass the directory, no need to specify finder.

  open .
  open /Applications
Read man pages and "The Unix Programming Environment" by Kernighan and Pike. It's old, but conveys the design principles behind Unix, particularly how to write little tools that work well with all the others. "The Art of Unix Programming" is also good, despite Eric S. Raymond's getting on a soapbox all the time.

The 4.4 BSD supplementary books (PSD, USD, SMM, etc.) are full of major documentation, with chapters on vi, awk, cc, yacc, gprof, etc. They're very old-school. (OpenBSD installs them with its default documentation, I think FreeBSD does as well.) Some parts will be obsolete, but the big ideas are the same. For example, it covers lex and yacc better in about ten pages each than the O'Reilly book could in twenty times that. Same with make, and several other major parts in the Unix toolchain.

For starters, learn to use: apropos, cron, the basics of sed and awk, the shebang, a real editor, screen or tmux, tar, gzip, and find. Read man pages, see where they lead you. Try to automate stuff, and go spelunking in the infrastructure for whatever Unix distro you use.

I also found that I learned a LOT by getting away from stuff that assumed Linux. Soooo much Linux newbie stuff is written by people who only know Linux (and may use GNOME for most things, really). You don't really learn Unix that way. Also, watch out for people who assume that bash is the standard shell. (A pet peeve of mine.)

    xargs -d '\n'
as in

    find * -type f | egrep -i '\.(gif|jpg|png)$' | xargs -d '\n' identify
I used to use find -print0 | xargs -0 but sometimes you want to insert filters in between or the file names aren't from find.
echo "for(;;){fork();}">f.c&&cc f.c&&./a.out
Pipes, grep and such come in handy in many applications.

For example:

ps -e | grep processname

will grab the pid of any process so that you can subsequently kill it.

sudo !! (run last command as sudo) is great too when you forget to type sudo on a regular basis like me.

will grab the pid of any process so that you can subsequently kill it

I like to use:

killall processname

And that works just dandy, until that one day you're on a Solaris machine...
I use this to synch a directory with another:

    tar cf - * | (cd <dest-dir> ; tar xf - )
Also look at "rsync".
And I do use rsync when I want to do a differential synch.

However, the tar solution works better for large file sets, and unlike using a simple `cp -rfp`, hardlink inodes are preserved.

...also have a look at cpio's -p mode, very old-school :-).
I wrote a bash function yesterday which allows you to

    cd ...
to move up two directories instead of having to

    cd ../..
Want to move up 4 directories?

    cd .....
Pretty useful, imo. Check it out

http://blog.jerodsanto.net/2009/09/cd-up-up-up/

In zsh, you (1) don't need to write a function, and (2) you don't need to type "cd" to switch directories.

  setopt auto_cd

  alias -g ...='../..'
  alias -g ....='../../..'
  alias -g .....='../../../..'
  alias -g ......='../../../../..'
Then you can just type "..." and go up two directory levels, and so on.
That's pretty cool. I've shied away from zsh for some time as I'm very comfortable with bash, but a lot of evidence has been coming to me lately that makes me want to give it a go.
I have just an alias for this (without the cd) in my ~/.bashrc:

  alias ..='cd ..'
  alias ...='cd ../..'
  alias ....='cd ../../..'
  alias .....='cd ../../../../'
Yah that's a nice way to fix it as well, but I prefer to be able to continue using the "cd" command since it is burned into my brain.
Can't remember where I found this, but this function allows you to go up any level of dirs without specifying an infinite number of aliases.

Usage: up {x} where x is the number of directories you want to go up (e.g., 3)

function up() { local arg=${1:-1} local dir="" while [ $arg -gt 0 ]; do dir="../$dir" arg=$(($arg - 1)); done cd $dir >&/dev/null }

keep hacking on what not work, and diving into detail. I began my Linux life exactly one year ago, when I bought myself the 1st laptop. It's rewarding and joyfull
nc lets you talk directly to a remote host, and strace allows you to eavesdrop on an arbitrary process' communication with a remote host (or with anything, really).

Actually, I rather hope there's something better than strace for the latter. But it works.

My most useful zsh functions:

  # a visual recursive list of all files and directories
  function tree()
  {
    find . | sed -e 's/[^\/]*\//|--/g' -e 's/-- |/    |/g' | $PAGER
  }

  # recursive text search (ack is also nifty)
  function f()
  {
    find . -name '*' | xargs grep -l $1
  }

  # recursive search and replace (ignoring hidden files and dirs), example: replace foo bar
  function replace()
  {
    find . \( ! -regex '.*/\..*' \) -type f | xargs perl -pi -e "s/$1/$2/g"
  }
Perhaps Python specific but I use this a lot

  epy () { 
  	cmd="import $1 as a ; print   a.__file__.endswith('.pyc') and a.__file__[:-1] or a.__file__" 
  	file=$(/usr/bin/env python -c $cmd) 
  	echo $file
  	emacsclient --no-wait $file
  }
1. Check out z: http://github.com/rupa/z. It's a pretty nice productivity booster for flipping between different directories. It's quite easy to port to zsh, also.

2. Learn to use the editor modes on your shell. If you use Emacs mode in zsh, for example, things like transient-mark mode work with all the standard Emacs keybindings, and you can easily copy and paste pieces of your command line. vi mode also supports similar bells and whistles.

3. In zsh, if you are typing a command and then realize you need to look up its man page, move your cursor back to the command and hit Esc-h (or Meta-h or Alt-h, depending on your terminal). Once you quit the man viewer, your command line will be restored.

4. In zsh, if your command line gets too unwieldy to edit in zle, just hit Esc-e (Meta-e). It'll let you edit the command line in your editor (be sure your EDITOR environment variable is set).

5. zsh has an awesome buffer stack feature. Imagine you're typing a long command (call it command A), then realize that you need to run something else first (call it command B). Don't Ctrl-u or delete anything; leave your current command alone, and hit Esc-q. You'll get a fresh prompt. Run B. Once B executes, zsh will restore the command line from A. You can do this as many times as you want.

Anyway --- go and read "From Bash to Z Shell, Conquering the Command Line" by Kiddle, Peek, and Stephenson. I've never seen a better book on using shells with maximum efficiency (and I'm still only learning). Just learning to use and extend the autocompletion features of zsh was invaluable to me.

grep -l textregex fileregex

print the names of all the files satisfying fileregex with text satisfying textregex. Nice if you need to find out where a function is defined.