Never knew about the "cal" command. I especially like that you can quickly look up a previous or future month/year.
>> cal 3 1973
quickly shows a calendar of March 1973
The "+n" syntax is not standards compliant, from the tail info page (on fedora 19):
On older systems, the leading '-' can be replaced by '+' in the obsolete option syntax with the same meaning as in counts, and obsolete usage overrides normal usage when the two conflict. This obsolete behavior can be enabled or disabled with the '_POSIX2_VERSION' environment variable (*note Standards conformance::).
I recommend Quicksilver on mac, free and powerful. It lets you to find a lot of stuff (application, contact, email, etc.) and manipulate them (send, forward, print, etc.)
Alread rocks. You can develop workflows for common groups of tasks and it integrates well with iTunes, 1Password, Terminal and other system actions/apps.
I second the Quicksilver recommendation. When you combine Quicksilver triggers (keyboard shortcut available regardless of what application you're in) with applescript, and fabric it's very powerful. I have a trigger that that restarts all of my local services using fabric and opens up new terminal windows running tails of the relevant logs.
The Applescript is this if anyone wants to do something similar:
tell application "Terminal"
do script "tail -F /somepath/some.log"
do script "tail -F /anotherpath/another.log"
do script "sudo fab -f '/pathto/fabfile.py' local deploy"
activate
end tell
Here's a detailed explanation of setting up Triggers:
It can be really slow, even for a few tens of thousands of tiles. I remember watching Nautilus's "properties" panel slowly count up the number of files and directories.
I'm not sure what this has to do with piping (why would nautilus pipe the output of find when doing a directory listing?), but I doubt it's the pipe that's causing performance issues. Uncached file system access has a tendency to be slow[0].
> Can't they just directly access inodes for high speed counting?
Don't know what you mean by that exactly. But, I'd be stunned if find is actually reading the actual files rather than just reading the directory entries in directory "files". Which is what I'd call accessing inodes "directly"
There really isn't any way around this. The first pass over your data is going to be slow while the inode data is examined, but it will be cached for the next pass. If you do this before that data gets removed from the cache it will be much faster.
imac:~
$ time find . | wc -l
419191
real 0m37.865s
user 0m0.385s
sys 0m2.611s
imac:~
$ time find . | wc -l
419195
real 0m10.503s
user 0m0.330s
sys 0m1.430s
In this invocation find and wc both have two fixed arguments, you're never going to hit MAX_ARG_PAGES. The output from find is passed to wc over a pipe, and to my knowledge there's no upper limit on how much data you can transfer over a pipe. I've piped huge disk images.
I think you are confused — at no time are any files passed as arguments in the command "find . | wc -l".
Also, to count the number of files in a directory, you need to do readdir(), which will get you the names. Then, to see if any of them are directories to be recursed into you need to stat() the names individually. This is all "find" does. How, I'd very much like to know, is this "a nasty hack"?
I am not saying find is outputting files, I am saying it is decoding the full table entry for each file and formatting it, then passing that listing as an entry to wc. Which has to be a bunch of overhead that adds up when you have hundreds of thousands of files to count.
All we want from find is "count the number of inode entries in this tree branch that are type f". No formatting, no output, just count. That has to be much more efficient.
You are going to need to rely on the filesystem authors to write a tool to traverse their inode trees directly, or probably write something like that yourself.
There is a more important benefit to using the -prinf '1' approach - which is that it won't miscount the result if any of the filenames contain newlines.
The output from find is a list of filenames separated by newlines. The wc command is reading from stdin and counting those newlines[1] -- wc itself has just two arguments in its argv array here ("wc", "-l"). You will not exceed its argument list. Further, it is not at all a hack to count the number of files this way; rather using pipes to compose functionality is the Unix way[2].
But fwiw, if you want a count of the inodes in use in the entire filesystem, you can get that directly from "df -i".
In any case, as an exercise to the reader, I encourage you to grab the source to find and add the -count output option you desire.
[1] Technically a filename can contain newlines, so this would throw off the count. The fix for that would be to use find's -print0 output option and then pipe to something which can count on the nulls. In practice, you're unlikely to have such filenames.
If you're dealing with a truly large number of files you could use "find . -type f -printf 1 | wc -c" which would just print a single character to the pipe and count the characters. I really doubt this is significantly slower than one program that does both would be. It'll do roughly the same amount of work.
It's not the pipe through wc, but statting the directory entries which slows you down.
If your locatedb is current, you can use locate for this:
$ locate $PWD | wc -l
That runs in just a hair under a second (0.908s real) for the first instance. 'find' takes a few seconds on first pass (it's got to read from disk), but is surprising fast (0.227s real) on subsequent runs.
So I guess it depends on whether you're doing it just once or multiple times.
function lc() { if [[ "$#" -gt 1 ]]; then for DIR in "$@"; do echo -n "$DIR - " ; ls -AU1 $DIR | wc -l ; done ; else ls -AU1 "$@" | wc -l ; fi; }
Then "lc dir/" will print the number of files in the directory, and "lc dir/*" will print the number of files in each of the subdirectories. This exactly what you're looking for because it doesn't descend into the tree. If you're working with directories that may have tens or hundreds of thousands of files in them, it's useful to check first.
du just uses the fts API, so if you want something directly equivalent (omitting options and error handling) it was only barely too long to post: http://pastie.org/8315296
Compared to the equivalent find, code #1 is much faster in this completely scientific best of several runs:
$ time (find OSS/llvm -type f -or -type l | wc -l)
real 0m4.681s
$ time du -d0 OSS/llvm
real 0m4.863s
$ time ./fts OSS/llvm
real 0m0.559s
Ostensibly because of the lack of stat calls. On the other hand the second paste and find with no arguments are equivalent.
Since I didn't install vim on that box (just use basic vi), xxd wouldn't have installed as part of it. Where as in a jail on the same box, xxd does exist:
root@primus:/usr/ports # which vim xxd
vim: Command not found.
xxd: Command not found.
root@primus:/usr/ports # jexec 1 which vim xxd
/usr/local/bin/vim
/usr/local/bin/xxd
I've known about 'od', but I've always used 'hexdump' instead. Amusingly, I just noticed that they are literally the same (hard-linked) binary on OS X 10.6, even though the arguments differ somewhat. The man pages states that 'od' is part of POSIX, which aligns with your comment about portability.
This is also a common UNIXism: programs that behave differently depending on what they're called as. Another example is gzip (gunzip = gzip -d, gzcat = gzip -dc).
`man 7 charsets` is a great read which recommends checking out the pages for console(4), console_codes(4), console_ioctl(4), ascii(7), iso_8859-1(7), unicode(7) and utf-8(7).
`mdfind' in non-Mac UNIXs is spelled `locate' (and the corresponding `updatedb' updates the indexing db, obviously (although you don't have to do this often, as it runs as a cron job)).
I believe xxd is actually part of vim and not a Unix command itself. It's used to support hex-editing binary files, but you can see why it's useful in other roles as well.
The origins of xxd seem to be a bit murky, as I've seen others say the same thing about xxd being available only if vim is installed, but I've never seen a Unix system that didn't have vi(m) included.
$ man xxd | grep vi
Use xxd as a filter within an editor such as vim(1) to hexdump a region
Use xxd as a filter within an editor such as vim(1) to recover a binary
Use xxd as a filter within an editor such as vim(1) to recover one line
From what I've been able to find, vim was released in 1991, and xxd has been around since at least 1990, whereas vi was written by Bill Joy in 1976. (according to Wikipedia)
Interesting bit of trivia, perhaps someone with a good memory can clear it up for us?
There are no hits for xxd on the Unix Tree (http://minnie.tuhs.org/cgi-bin/utree.pl) so it's definitely not a traditional Unix tool. I'm sitting in front of a BSD system which includes nvi. I don't have xxd.
There was a time when I, annoyed at being thrown into vi(m) whenever something spawned an editor, rather than setting the EDITOR environment variable on login, removed vi and replaced it with symlinks to emacs on the servers I managed.
I'm a bit more tolerant of vi now, but just because I "never" see it courtesy of suitable EDITOR entries.
3. There is always nice command that does what you want but you somehow forget it. I can't believe that Common Lisp has just 900+ symbols but I routinely forget some of them when programming.
My favorite part is when you implement something you could've had running in a minute or so, but instead developed it for hours or days and it's still not as good as the existing solution. I often get that with C and libraries. There has to be a better way to search through existing stuff which is good and proven, not just for C of course.
Haskell has hoogle [0], that lets you search by type signature. If you already know roughly what kind of function you want, it's a lot easier to sift through a tiny handful of (or even a single) functions that match a given type signature.
ISTR Forth had something similar, or maybe I'm just thinking that in Forth, words are almost uniquely defined by their stack annotations.
it's quite a common pattern in commandline apps to check if the stdout is a tty or not, and do things differently if so. So when you pipe it through `cat', it does the no-fancy-included output, much like the colouring that `ls' will apply interactively, but not when piped/redirected.
It's important for grepping and sedding - the colors are escape codes and would mess with your regular expressions otherwise (you want to find foobar , but bar was blue so you got foo\e[1;34mbar\e[0m instead, and it doesn't match with foobar).
On the other hand, if you want colors displayed you can force grep/ack to display the color 'always' and then pipe the output into 'less -R'. The -R (or --RAW-CONTROL-CHARS) flag has less display output such that color escape sequences work as you might expect, coloring the output displayed by less.
Because, in many - if not most - situations, spawning another process is insanely lightweight compared to the cognitive/physical load of your alternative (which includes remembering the switch and its value ("never" is hardly intuitive), remembering whether or not you're using a system that supports it (surprise, surprise, OSX doesn't), and typing all those extra characters).
Admittedly, I never actually do this because I don't ever care to not see colour (what would be the motive?), but I can understand why the OP does what they do.
"typing a password on your local machine once, then using a secure identity to login to several remote machines without having to retype your password by using an ssh key agent. This is awesome. "
Sounds awesome, but I really didn't understand what he means. Is he talking about using a certificate (-i option) to log in ?
Yeah, you have a cert set up and put the public cert in the identities file for each server you want to connect to. Then just SSH into each box with agent forwarding ('-a' I think, but you're best to check that).
So you'll need to create an SSH cert first:
ssh-keygen -b 4096 -t rsa -C "$USER created on $(hostname)" -f $USER.key
Keep the private key, that's essential you keep this safe as it's literally your key to access any servers you do the next step for.
On each server SSH, append your public key to the of /home/$USER/.ssh/authorized_keys (creating the file if it doesn't exist). It must be the public key!
There'll be better guides online for this stuff so I suggest you have a read through them before blindly pasting the stuff I've posted into your terminal. But it's all quite simple stuff once you've grasped the difference between a public and private key (if you weren't already aware - you may well be)
The single most useful thing about ssh to me is that it connects stdout/in across the channel (this is behavior inherited from rsh). This allows for e.g.:
% (cd /foo; tar cpf - .) | ssh bar \(cd /baz\; tar xpf -\)
Or:
% cat ~/.ssh/id_rsa.pub | ssh bar tee -a .ssh/authorized_keys
Forgive my ignorance, but I don't know what what it means to connect "across the channel", but it sounds important. Do you know a link that explains the concept? I've failed to Google it.
He's talking about forwarding stdin and stdout across the SSH connection to the program that SSH is running on the other end; such that the stdin of ssh is passed unmodified to the program and vice-versa.
It means that whatever you put into the ssh process's STDIN gets sent to the STDIN of the remote process. For example you can do this:
tar cj $bigdir | ssh host 'tar xj'
That creates a tar.bz2 archive on your machine, sends it over ssh to the remote host, where it's unpacked. Ad hoc compressed and encrypted file transfers made easy.
Sure, of course. But I've used that tar|ssh combination so much that it was the first example of using the rsh-like stdout/in pairing that came to mind. As much as I bitch about Unix and POSIX and horrific shell brain damage, I still reach for crazy shell pipelines when I have data manipulation tasks up to some pretty large scale n.
Are you familiar with stdin/stdout/stderr on Unix and how they interact? What ssh does that I find so powerful is ensure that what you put into stdout on one side of a pipe flows out of stdin on the process that ssh starts on the remote host.
Of course, ssh also gives you the output of the remote process, so you can put an ssh command in the middle of a pipeline. My day to day life no longer involves interacting with lots of remote Unix systems, but when it did, it was really, really useful.
This is a bit of a pet peeve of mine, but cd/tar should likely be
joined with && - in other words, in what directory do you want tar
to (not) run if cd fails? So consider:
% (cd /foo && tar cpf - .) | ssh bar '(cd /baz && tar xpf -)'
It would be an interesting project to look at people's shell usage and try to determine when they started using Unix. I have loads of Irixisms that have only recently been crowded out of my head by Darwin BSDism.
Yeah, before I was heavily into automation (i.e. puppet) this was my go to command. Use a for loop and ssh to run things on a group of systems.
for host in h-abc h-def h-ghi h-jkl h-mno h-pqr h-stu h-vwx h-yza;
do
ssh -l root $host "hostname; echo "1.1.1.1 server" >> /etc/hosts";
done
This would iterate over hosts (h-abc .. h-yza) and run the commands "hostname; echo "1.1.1.1 server" >> /etc/hosts". Or maybe use a for loop and scp to push the files out and then run md5 to verify they are correct. It was a hack but useful for a small set of machines.
for host in ... ; do ssh -l root $host "..." & done; wait
and it's very parallel. All that key exchange takes a while, so it's not ridiculously fast, but it's pretty good. If you've got ControlMaster set up already to be per-host, this might very well be very fast the second time.
Also check out xargs with the -P 10 -n 1 flags (for example) -- to limit at most 10 parallel tasks running at the same time. Useful if you don't have GNU parallel installed where you happen to be working (everyone has xargs installed).
Scp doesn't get all file types correct (fifos, etc), while tar does.
Piped tar commands do, but are generally inferior to rsync (at least if you ever need to copy more than once). I believe I've heard (here) that piped tar commands can be useful for the first sync (possibly better compression, no overhead of file checksums), then rsync thereafter.
because so many programs behave differently when attached to a tty (like prompting for input) and I just want a plain text log that says when it succeeded or where it blew up.
Agreed, and my problem with even the usage above is that I often realize too late that "whatever" is not going to finish before I need to unplug. Then I have to kill "whatever" and restart with nohup and logging. tmux/screen always has me covered.
If anyone is interested there were several great posts on Hacker News a while about about useful UNIX commands [1, 2, 3]. I have also created several screencasts about command commands like the following [4], and one about the Filesystem Hierarchy Standard [5].
ls man pwd cd top ps df du cp mv rm mkdir rmdir less cat vi
Thanks for the suggestion! Actually, I initially wanted to do one on more advanced commands, but I thought I'd better lay a foundation first. I also want to cover using pipe (|) to chain these commands together. I'll make sure to include these.
Say you have a set of tasks you need to do, and some of those depend on others. Now, you want to perform those tasks in order. The question then becomes: in which order? That's the problem that tsort (1) solves.
It's a way to do some of what make does without having to write a Makefile. 99.9% of the time, it never comes up, but when it does, it is very, very handy to be able to this in a shell pipeline. HTH.
Great links, though I admit when I saw the OP I was expecting content like that and was pleasantly surprised that they really were "Unix commands I wish I'd discovered years earlier", and not the standard array of unix tips+tricks that you can get by without for awhile but as you become more advanced become second-nature.
"Man ascii." The number of times that I wound up generating my own ascii table from whatever language I was working with...
"xxd" I always either used search in emacs hex-mode (which I find cumbersome and annoying but haven't taken the time to customize), or rolled my own search using a programming language. This would have come in handy SO many times back when I used to muck around with videogame save files and the like. I never even knew it existed nor how to discover that it did.
'cal' I learned about early on so never missed it and the 'ssh' stuff kind of fits into the regular "unix tips and tricks" category.
You reminded me of something. There is actually a really cool and simple site called asciiflow [1], which I use all the time to draw diagrams for explaining things in email, etc. It's pretty cool, and was even submitted several times to HN [2, 3].
Wow. This is really cool!
Not only can you make the ascii drawings, but there is an integration with a service called ditaa (unaffiliated) that then turns your ascii drawings into actual images.
I am keeping these.
http://ditaa.sourceforge.net/
I can't browse www.asciiflow.com, the name does not resolve:
$ host www.asciiflow.com
Host www.asciiflow.com not found: 3(NXDOMAIN)
Caused by both authoritative nameservers for asciiflow.com which return the correct answer (a CNAME to ghs.google.com), but with status=NXDOMAIN (wtf?):
$ dig +short asciiflow.com ns
ns2.123-reg.co.uk.
ns.123-reg.co.uk.
$ ; <<>> DiG 9.8.1-P1 <<>> @ns2.123-reg.co.uk www.asciiflow.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 65092
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;www.asciiflow.com. IN A
;; ANSWER SECTION:
www.asciiflow.com. 86400 IN CNAME ghs.google.com.
;; AUTHORITY SECTION:
google.com. 14400 IN SOA ns.123-reg.co.uk. hostmaster.google.com. 2013052701 86400 0 604800 86400
;; Query time: 207 msec
;; SERVER: 92.51.159.40#53(92.51.159.40)
;; WHEN: Wed Sep 11 00:27:52 2013
;; MSG SIZE rcvd: 123
i usually click those links, see the stuff and roll up my eyes. nothing learned, but upvote so more people learn about them anyway.
this time, the very first one `man ascii` got me. Genius. Never though about looking for that in man before. But in my defense, this was added in 1999 :)
One of my favorite things about unix is that you can chain commands together with pipe (|). Say for example that you have a example data file that looks like this:
You can chain commands together to quickly get ball park figures. I do this all the time when reviewing log data to get simple counts and look for anomalies.
Let me break this down a little. I cat data.txt, which just prints the contents to the console, I use grep to remove the # header, then use awk to print the second data column, the sort the values for my next procedure, and lastly I use uniq -c, which takes the input and counts all the unique values. So you can see that "4 104" means that 4 instances of 104 were found. I use this often to look at httpd logs for IP addresses or to review sudo commands, etc. Very handy!
ps. there are optimizations that could be done here, but that what is great about unix and "one function" utilities. There are many ways to the same destination!
Yes, there are some ways to simplify. You can get rid of the "UUOC". Instead of
cat data.txt | grep -v '#'
you can do:
grep -v '#' data.txt
or you can simplify further by moving the grepping into awk, so that the full command becomes
awk '!/#/{print $2}' data.txt | sort -n | uniq -c
sort -n is good to use for sorting numbers. You could probably pull the sort | uniq -c into awk somehow, but I have already demonstrated the full extent of my awk knowledge. One of these days I will get around to actually learning it.
I find it fascinating that someone could be using something as complex and slow as Hadoop and not know of sort(1). Not faulting any lack of awareness, but it is really interesting that Google map/reduce and Hadoop marketing (word of mouth?) are so effective while the contents of /usr/bin are like buried treasure.
The funny part is that one of the early uses of Google's map/reduce approach was probably to distribute the job of sorting search results among many servers. And here you are sorting Hadoop results with something as small and simple as sort(1).
Wow, now I feel old. I thought everyone got started with SunOS on a Sparcstation 2? Seriously, don't most people encounter it either through installing a Linux distribution or firing up a terminal on OSX? Or is even my-first-Unix a virtual experience?
There are free unix shells out there, google will find them. Don't expect anything fancy, but you will be able to try all the commands talked about here.
To connect to the shell from Windows, you can use the Putty program (google for it), although the shell documentation will almost certainly tell you how to do it.
Better yet use 'less +F'. It's as easy to remember as tail -f.
The advantage of using less here is that you can easily interrupt the tailing (ctrl-c). For example, to search. Then just hit 'F' again to restart tailing. But now your search pattern will be highlighted as the data streams by.
I have been using both "tail -f" and less for years, and never knew about this. Thank you. In three lines of text, you have made my logfile-centric life oh-so-much easier to deal with.
For others who don't know it: http://en.wikipedia.org/wiki/Pushd_and_popd. It essentially is a stack for the working directory, so you can get back to where you were previously quickly and easily.
alias d='[ ${#DIRSTACK[@]} -gt 0 ] && for i in `jot ${#DIRSTACK[@]} 1 ${#DIRSTACK[@]} 1`; do echo ${DIRSTACK[$i-1]}; done; echo $PWD'
alias popd='[ ${#DIRSTACK[@]} -gt 0 ] && { cd "${DIRSTACK[${#DIRSTACK[@]}-1]}"; unset DIRSTACK[${#DIRSTACK[@]}-1]; }; d'
alias pushd='DIRSTACK[${#DIRSTACK[@]}]="$PWD"; cd'
I first ran into clipboard copy/paste on Mac. Discovered "xclip" on Linux. Aliased copy to 'xc' and paste to 'xp', later added 'xpc' to paste from the secondary (application) clipboard.
In 2005 I wrote similar utilities for windows called ccopy, ccut & ppaste. I recently dug them up and put them on GitHub. https://github.com/FigBug/ccopyppaste
258 comments
[ 3.2 ms ] story [ 261 ms ] threadNetBSD has had it for 11 years. FreeBSD has had it for three years and six months. DragonFly doesn't have it, OpenBSD doesn't have it...
http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/cal/cal.c?re... http://svnweb.freebsd.org/base?view=revision&revision=204697
What am I doing wrong?
This worked like a charm though:
cal dec 2002 | tail -6 | wc -w
(since it always takes up 6 lines, even sep 1752)
http://qsapp.com/
The Applescript is this if anyone wants to do something similar:
tell application "Terminal" do script "tail -F /somepath/some.log" do script "tail -F /anotherpath/another.log" do script "sudo fab -f '/pathto/fabfile.py' local deploy" activate end tell
Here's a detailed explanation of setting up Triggers:
http://leafraker.com/2007/09/17/how-to-create-a-quicksilver-...
Can't they just directly access inodes for high speed counting?
0 - https://gist.github.com/nahurst/3711264
Don't know what you mean by that exactly. But, I'd be stunned if find is actually reading the actual files rather than just reading the directory entries in directory "files". Which is what I'd call accessing inodes "directly"
Available in most repositories, source code available at: http://mama.indstate.edu/users/ice/tree/
What do you do when find cannot handle the sheer number of files, argument too long from the linux hardcoded MAX_ARG_PAGES
We need a dedicated utility that counts inodes.
Also, to count the number of files in a directory, you need to do readdir(), which will get you the names. Then, to see if any of them are directories to be recursed into you need to stat() the names individually. This is all "find" does. How, I'd very much like to know, is this "a nasty hack"?
All we want from find is "count the number of inode entries in this tree branch that are type f". No formatting, no output, just count. That has to be much more efficient.
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....
That's "time find . | wc -l"
It's going to be disk limited unless you're in cache.
Switching to "time find . -printf '1' | wc -c" lowers overhead to .7 seconds, so roughly 1us/file.
The output from find is a list of filenames separated by newlines. The wc command is reading from stdin and counting those newlines[1] -- wc itself has just two arguments in its argv array here ("wc", "-l"). You will not exceed its argument list. Further, it is not at all a hack to count the number of files this way; rather using pipes to compose functionality is the Unix way[2].
But fwiw, if you want a count of the inodes in use in the entire filesystem, you can get that directly from "df -i".
In any case, as an exercise to the reader, I encourage you to grab the source to find and add the -count output option you desire.
[1] Technically a filename can contain newlines, so this would throw off the count. The fix for that would be to use find's -print0 output option and then pipe to something which can count on the nulls. In practice, you're unlikely to have such filenames.
[2] cf. http://harmful.cat-v.org/cat-v/unix_prog_design.pdf
find . | sed -e 's=/[^/]*$==' | uniq -c
If your locatedb is current, you can use locate for this:
That runs in just a hair under a second (0.908s real) for the first instance. 'find' takes a few seconds on first pass (it's got to read from disk), but is surprising fast (0.227s real) on subsequent runs.So I guess it depends on whether you're doing it just once or multiple times.
function lc() { if [[ "$#" -gt 1 ]]; then for DIR in "$@"; do echo -n "$DIR - " ; ls -AU1 $DIR | wc -l ; done ; else ls -AU1 "$@" | wc -l ; fi; }
Then "lc dir/" will print the number of files in the directory, and "lc dir/*" will print the number of files in each of the subdirectories. This exactly what you're looking for because it doesn't descend into the tree. If you're working with directories that may have tens or hundreds of thousands of files in them, it's useful to check first.
Compared to the equivalent find, code #1 is much faster in this completely scientific best of several runs:
Ostensibly because of the lack of stat calls. On the other hand the second paste and find with no arguments are equivalent.edit: screwed up benchmarks
It doesn't show up in my ports either:
Obviously xxdiff is a different (if you pardon the pun) utility, but xxd should show up in that list also.edit 2:
Another comment on here might explain why you're seeing xxd and I'm not: https://news.ycombinator.com/item?id=6360814 (it's not a separate utility, it's part of vim)
Since I didn't install vim on that box (just use basic vi), xxd wouldn't have installed as part of it. Where as in a jail on the same box, xxd does exist:
Haha... What a trip. This is the sort of thing that I'll end up wasting half a day trying to get an answer to.
You're right, xxd is included with vim, but as an external program.
Vim provides this capability through the external program xxd, which is included by default in standard distributions of Vim.
My pedantic nature was hung up on the release dates of the respective programs, as xxd predates vim on Unix.
It's even listed as a 'Convert to HEX' item in the Vim GUI menu.
I can't remember how many times I implemented hexadecimal dumps and undumps because I didn't want to coerce od into working the way I wanted.
http://www.manpages.info/sunos/intro.1.html
Note the list of commands.
I guess you could ls /usr/bin and just start reading their man pages.
Summary: man ascii, cal, xxd, ssh, mdfind
:-)
From what I've been able to find, vim was released in 1991, and xxd has been around since at least 1990, whereas vi was written by Bill Joy in 1976. (according to Wikipedia)
Interesting bit of trivia, perhaps someone with a good memory can clear it up for us?
I'm a bit more tolerant of vi now, but just because I "never" see it courtesy of suitable EDITOR entries.
Want to find all the tools on your system that use the curses library (for user-friendly interfaces)? Try this:
Don't know how your current path finds a program? Use which: Annoyed that /sbin and /usr/sbin aren't in your path? Use whereis:1. Both are operating systems
2. Both are conceptually simple.
3. There is always nice command that does what you want but you somehow forget it. I can't believe that Common Lisp has just 900+ symbols but I routinely forget some of them when programming.
... I think it works when apropos isn't installed. Possibly it just calls out to it.
ISTR Forth had something similar, or maybe I'm just thinking that in Forth, words are almost uniquely defined by their stack annotations.
[0] http://www.haskell.org/hoogle/
http://www.gnu.org/software/gcal/
Admittedly, I never actually do this because I don't ever care to not see colour (what would be the motive?), but I can understand why the OP does what they do.
[http://partmaps.org/era/unix/award.html]
Your 'ls' must be an alias for the real ls with some options. For example, mine is:
A quick way to run the original 'ls' (not the aliased version), just prefix it with backslash: Or:Example:
ssh username@host "echo $HOSTNAME && sudo somecommand && cat somecommand.log"
There's probably a better way to do this, but in a pinch I can fix a problem on dozens of machines just by altering the host string.
"typing a password on your local machine once, then using a secure identity to login to several remote machines without having to retype your password by using an ssh key agent. This is awesome. "
Sounds awesome, but I really didn't understand what he means. Is he talking about using a certificate (-i option) to log in ?
So you'll need to create an SSH cert first:
Keep the private key, that's essential you keep this safe as it's literally your key to access any servers you do the next step for.On each server SSH, append your public key to the of /home/$USER/.ssh/authorized_keys (creating the file if it doesn't exist). It must be the public key!
There'll be better guides online for this stuff so I suggest you have a read through them before blindly pasting the stuff I've posted into your terminal. But it's all quite simple stuff once you've grasped the difference between a public and private key (if you weren't already aware - you may well be)
"ssh-ebT23030"
Meaning that anyone with root access who know the other machines that someone is accessing as well as their username can access those machines.
I've seen cases where in abnormally terminated sessions that file stays around instead of getting deleted when the session ends.
So basically, you get :
Hope that helps.Of course, ssh also gives you the output of the remote process, so you can put an ssh command in the middle of a pipeline. My day to day life no longer involves interacting with lots of remote Unix systems, but when it did, it was really, really useful.
ps. if you are interested in learning about puppet, I have put together a screencast about it @ http://sysadmincasts.com/episodes/8-learning-puppet-with-vag...
Piped tar commands do, but are generally inferior to rsync (at least if you ever need to copy more than once). I believe I've heard (here) that piped tar commands can be useful for the first sync (possibly better compression, no overhead of file checksums), then rsync thereafter.
[1]: http://emacsredux.com/blog/2013/07/05/locate/ (setq locate-command "mdfind")
[2] https://news.ycombinator.com/item?id=5022457
[3] https://news.ycombinator.com/item?id=4985393
[4] http://sysadmincasts.com/episodes/13-crash-course-on-common-...
[5] http://sysadmincasts.com/episodes/12-crash-course-on-the-fil...
(Actually, I have found use for those first two on occasion. ;)
It's a way to do some of what make does without having to write a Makefile. 99.9% of the time, it never comes up, but when it does, it is very, very handy to be able to this in a shell pipeline. HTH.
"Man ascii." The number of times that I wound up generating my own ascii table from whatever language I was working with...
"xxd" I always either used search in emacs hex-mode (which I find cumbersome and annoying but haven't taken the time to customize), or rolled my own search using a programming language. This would have come in handy SO many times back when I used to muck around with videogame save files and the like. I never even knew it existed nor how to discover that it did.
'cal' I learned about early on so never missed it and the 'ssh' stuff kind of fits into the regular "unix tips and tricks" category.
You reminded me of something. There is actually a really cool and simple site called asciiflow [1], which I use all the time to draw diagrams for explaining things in email, etc. It's pretty cool, and was even submitted several times to HN [2, 3].
[1] http://www.asciiflow.com/#Draw
[2] https://news.ycombinator.com/item?id=2847177
[3] https://news.ycombinator.com/item?id=3598177
this time, the very first one `man ascii` got me. Genius. Never though about looking for that in man before. But in my defense, this was added in 1999 :)
I'm sure that was a bit earlier than 1999.
http://man.cat-v.org/unix_8th/7/
It's freaking fast and convenient, sorts hadoop reduce results like a champ.
ps. there are optimizations that could be done here, but that what is great about unix and "one function" utilities. There are many ways to the same destination!
The funny part is that one of the early uses of Google's map/reduce approach was probably to distribute the job of sorting search results among many servers. And here you are sorting Hadoop results with something as small and simple as sort(1).
column - columnify the incoming stream into columns
pr - set up incoming stream for pretty printing including columnification along or across the screen
tr - sanitize input, collapse several delimiters into one
echo $PATH | tr : '\n'
Examples include: $ df -h | column -t $ column -t -s: /etc/passwd
To connect to the shell from Windows, you can use the Putty program (google for it), although the shell documentation will almost certainly tell you how to do it.
The advantage of using less here is that you can easily interrupt the tailing (ctrl-c). For example, to search. Then just hit 'F' again to restart tailing. But now your search pattern will be highlighted as the data streams by.
For others who don't know it: http://en.wikipedia.org/wiki/Pushd_and_popd. It essentially is a stack for the working directory, so you can get back to where you were previously quickly and easily.
http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-f...
For instance:
pbpaste | fgrep -i "`pbpaste -pboard find`"
To search the copy clipboard with the find clipboard.
Makes all sorts of hackery possible.