Ask HN: What Linux diagnostic tools/commands do you use?

23 points by adampie ↗ HN
I've been writing a basic script (https://github.com/adampie/granscan) and was looking for feedback, what diagnostic tools/commands do you guys use? How should I change the script?

11 comments

[ 3.7 ms ] story [ 32.0 ms ] thread
I guess it depends what I'm trying to debug, but one thing that seems to be missing from your wrapper is quick logfile inspection. Tailing the last n lines of syslog (or something more specific) is often a good starting point when diagnosing problems.

Perhaps something that performs basic connectivity tests might be useful, too -- pinging the router, an external IP (like 8.8.8.8), and a FQDN (like google.com) would be a good quick test of both internal connectivity, external connectivity, and DNS resolver.

Good idea, I will add that in once I figure out a better way to deal with the output; I feel like it needs to be cleaner. Might make it menu driven.
strace - to see what a program is doing / where exactly something is failing if the error message is bad

ltrace - same as above, if strace is not enough

perf trace - if you don't know the exact process to strace upfront

https://github.com/brendangregg/perf-tools - to track disk accesses or exec calls

env LD_DEBUG=all - debugging dynamic linker issues

Per-process I/O and network metrics are super important. I usually use pidstat (-d flag gives you I/O) for this, or read data from /proc directly.
(comment deleted)
top, ps aux, which, find, ls, rm, cd, cat, echo, touch, pbcopy, ssh

That's pretty much it, otherwise I have a whole lot of cli tools too. Things like git, tmux, mosh, rails.. but I guess they don't count.

nc, tcpdump, ethtool, htop
htop, iftop, ngrep, tcpdump, lsof, strace, wireshark, nc, telnet, ping, netstat, ps, grep, awk, pv, middleman, curl, jq, mitmproxy, tail, tr....

Okay, ok pretty much all of them. :)

mostly htop, iotop, pv -d...

EDIT: ... curl, jq, cat, less, grep, awk, lynx, watch, parallel, xargs, head, tail, sqlite3, rsync, scp...