htop is one of the most missed tools when I am on a non-Linux OS.
On servers I find atop to actually be a better tool for finding issues that happen incrementally and not an obvious issue. I think that atop and htop both are very complimentary.
"Atop is an ASCII full-screen performance monitor for Linux that is capable of reporting the activity of all processes (even if processes have finished during the interval), daily logging of system and process activity for long-term analysis, highlighting overloaded system resources by using colors, etc. At regular intervals, it shows system-level activity related to the CPU, memory, swap, disks (including LVM) and network layers, and for every process (and thread) it shows e.g. the CPU utilization, memory growth, disk utilization, priority, username, state, and exit code.
In combination with the optional kernel module netatop, it even shows network activity per process/thread."
I really didn't know about this command ... is it that bad to give me -1? ... I will never ask again here because of this and in the same time thank you you to the guys that actually lost some time to explain.
I'd like to strongly encourage you in the opposite direction, to always ask questions. Always do this, regardless of how others react.
Many younger people have security issues explaining their knowledge with others, and sometimes exhibit this as criticism and acidity. It's possible that's what happened here.
It's also possible that someone disliked your use of the phrase "two words", maybe because they interpreted it literally. Some things resist explanations that are both short and accurate.
Regarding od, it's a flexible format conversion tool that lets you convert arbitrary data to hexadecimal, octal and C escaped printable text. I use it a lot!
I switched to macOS from Linux two weeks ago and I am starting to regret that decision. Most — if not all — the commands that I thought I had mastered after years working on Linux environments stopped working because their BSD counterparts have slightly differences that completely break everything I try to do. It is even more frustrating when I realize that I have to deactivate security features built-in the new Apple computers in order to make things like "dtrace" (the strace alternative) work.
For general dev use in macOS I find that installing the GNU userland (through MacPorts) keeps me sane, "sudo port install coreutils moreutils htop gnutar" and then put /opt/local/libexec/gnubin/:/opt/local/bin at the start of your PATH.
to be fair, you should probably call the GNU behaviour "GNUisms". The BSD behaviour is the historical POSIX behaviour. The GNU behaviour is, in a twisted sense, the result of embrace, extend and extinguish by the GNU people :-)
With a couple of small exceptions[1], the GNU behavior is also the historical POSIX behavior; the differences that will be jarring are where both BSD and GNU implement things that aren't in POSIX. Yes, the GNU behavior is GNUisms, but the BSD behavior is also BSDisms.
[1]: The most notable thing is that GNU generally allows flags to come before or after non-flag arguments, where POSIX requires them to come before.
You should give Macports a try if you haven't already! I run it on my macbook, and it's made a world of difference. 99% of everything I use in Linux can be found in Macports, and it "just works" for the most part.
You can't get kernel-specific things like strace, but they do have just about everything else. The big sell for me was being able to install the GNU versions of everything.
Macports keeps everything in /opt, so you also never have to worry about it clobbering anything that MacOS expects to work in a certain way.
edit: some background on Macports vs Homebrew - it's basically a vi/emacs kind of a thing. Homebrew utilizes the MacOS system stuff a fair amount, and installs everything to /usr/local. Macports tries to keep everything fully separate, and installs into /opt.
My personal experiences with Homebrew weren't great, but others will probably have the opposite opinion.
It's also very difficult to roll back to a previous version of a package unless the maintainers specifically opt in to tagging them. Homebrew also updates itself when you install something now, so say if you install python, it might update readline, so your postgres client will break.
It's not the worst approximation of ports, but it doesn't share the quality and care that goes into package maintenance and dependency management.
Really? If you're going to do that, it seems like you might as well just merge all those users into a single account anyway, considering how trivial it would be for any one of them to execute arbitrary code as any other.
I moved away from homebrew, to compiling my own software on macOS which I just put in ~/build. I don't use sudo make install since I own the ~/build folder.
There's no requirement that Homebrew be in /usr/local. Some individual (poorly-written) packages may assume that, but I've been running Homebrew in $HOME/.brew for multiple years on multiple machines with no issues at all.
That's actually fucking brilliant, and aligns much better with what homebrew is. I'm setting up my new macbook from work tomorrow and this is absolutely the way I'm going to do it, and how I'll suggest any of my colleagues do as well.
This is only about /usr/local itself, not the directories under it, the one with executables - that's still owned by you, not root. Some difference in ease of initial installation, zero difference in terms of (in)security.
"You should give Macports a try if you haven't already! I run it on my macbook, and it's made a world of difference. 99% of everything I use in Linux can be found in Macports, and it "just works" for the most part."
First thing I installed in the mac store. Just like on linux, it just works and I get all my favourite tools.
Until I can, as an unprivileged user, watch my own process's syscalls with dtruss, it's not an alternative to strace. Requiring root is a major hindrance.
Not the OP, but the reason I switched to the Mac 12 years ago (and never looked back) was to bring together my nerdy, creative, and "end user" sides under a single OS.
I'd been using mostly Linux on the desktop since 1996, I manage servers for a living and couldn't live without a terminal. That's the nerdy side.
On the other hand, I was serious enough with photography and electronic music production to need best of breed tools, which are still largely Windows and Mac-only to this day.
There's also a time for everything, and there are days when regardless of your computing skills, you need your creative person/end user hat on and apps that "just work" to keep your right brain fully focused on the end result (e.g. "No thanks, I'm not going to get distracted compiling some Gimp plugin to get red-eye removal", which may or may not have changed since I last tried)
MacOS, in spite of its recent slight drop in quality, is still second to none in satisfying all these requirements at once.
You normally do not have to disable security features to run programming/debugging tools, contrary to most of the advice you'll see in forums (and sometimes even Stack Exchange). I haven't used dtrace but non-bundled debuggers were a bit of a pain. You can self-sign these using the procedure described by a quick Google.
> Most — if not all — the commands that I thought I had mastered after years working on Linux environments stopped working because their BSD counterparts have slightly differences that completely break everything I try to do.
Very much this. Sometimes the differences are very subtle which makes them more annoying IMO (can you omit the target directory when using `find` if you want to use `pwd`?). For extra points, since you probably aren't deploying on a non-Linux *nix, the PITA will continue in perpetuity.
"You'll probably want to keep cron. But if you don't, then you should stop and disable the service. Because otherwise when trying to remove it with apt remove cron it will try to install postfix!"
Dependencies in debian systems which rely on "virtual" functionality can often have surprising results. At a guess removing cron removes things which depend on it - perhaps one of those is Exim. Something else "requires" an MTA, so it installs postfix instead to fill the gap.
"[explanation of acpid] But I'm on a virtual server that I don't intend to suspend/resume. I am going to remove it for fun and see what happens. ... I was able to successfully reboot the droplet but after halt Digital Ocean thought it was still on so I had to Power Off using the web interface. So I should probably keep this."
Science! :D
I love the approach of systematically going through and prodding everything to see what things do and whether they're actually required. I remember doing this on my Windows 95 machine, stripping down running tasks using Task Manager. Of course, in Windows 95, if you killed the wrong task the system would just reboot... :P
I used to do a lot of that when I was first learning Linux (back in the day when each Linux distro came with a full "HOWTO" collection.) I have become much lazier since, usually going for defaults and not touching anything unless I need the behavior to change.
> If I had two cores, my CPU usage would be 50% since my computer can run two processes at the same time. The load average of a computer with 2 cores that has a 100% CPU usage would be 2.00.
If I recall correctly the CPU usage would still be 100% but could go up to 200% if both cores are fully used.
My understanding from my Solaris days was that CPU load is the number of processes queued for CPU time over the given period. It doesn't related to usage directly at all. (Other than higher cpu speed and io through out leads to a reduction in load because tasks finish quicker)
kill as a shell built-in also allows you to pass it job numbers rather than PIDs. For example, kill -9 %1 would kill the first process you sent into the background, either with & or ^Z
Well, his 'Extreme edition' is actually not so extreme. This is how Debian base installation looks like.
A server can perfectly run without dbus, accountsservice, logind, policykit and acpid. And, unlike he wrote, timesyncd NTP synchronization works well without dbus.
(Debian base has also cron and rsyslog running by default, for legacy reasons.)
I tend to use "atop" because it gives me a better overall picture of my system. But, there is something really cool about "htoo" that many users don't realise and that I hadn't realised until somebody pointed it to me recently:
You can use your mouse/TrackPoint in htop to click on menu options, processes etc...
atop is way more superior than htop, specially with the log history. I just use htop when I need to check cpu usage or a particular process. But atop gives a better system status overview.
egrep filter for files at least 10m ends up being a nice speed up, otherwise you end up spending most of the time sorting all the tiny files to the top while up writing hundreds of megabytes of /tmp/sortXXXX files in the process.
ncdu has an option to store the scanned fs and display it again without walking the FS; useful if you're dealing with deep mess and slow, remote or else data.
To be a bit snarky, polkit is freedesktop dbus centric (and thus over-engineered) reimplementation of sudo for the GUI.
It uses a session tracker (used to be Consolekit, these days it is systemd-logind) to tell if something is running under an active user session and, and checks that against a bunch of rules (stored in XML, natch) to see if what is being attempted is allows (and if not, spawn a password prompt so the user can elevate their privileges).
If you want to do anything "complicated" like powering down your PC by clicking a button in your DE of choice you need polkit (and entourage) to be present these days.
Its one of those CADT things that seems to lead to "year of the (linux) desktop" never happening.
Is there a resource out there on how to write these beautiful terminal HUDs?
I'd really like to step up my game for GUI monitoring apps without having to choose between either terminal log spam, PyQT, or a web server. Ideally I'd have something like htop/wavemon/power top that I could stream compactly via ssh.
Love this post, first function explanation for 'uptime' module gives a concise 'strace' demo showing how to determine the files the standard 'uptime' binary opens.
For another fun ncurses tool (on a similar wavelength) I have become fond of glances[1] - it breaks out docker containers nicely, along with the good top-like overview of all procs.
For general use I'd like to add multitail (logging done wonderfully right), canto-rss (RSS reader/server), ncTelegram (Telegram client), turses (Twitter client), centerim (multi protocol IM client), weechat (IRC) and iftop (traffic monitoring for me), alpine (mail client) and mc (file manager). At least they're the tools termux starts with on my VPS and I pretty much live on there.
103 comments
[ 2.8 ms ] story [ 169 ms ] threadOn servers I find atop to actually be a better tool for finding issues that happen incrementally and not an obvious issue. I think that atop and htop both are very complimentary.
http://www.atoptool.nl/
"Atop is an ASCII full-screen performance monitor for Linux that is capable of reporting the activity of all processes (even if processes have finished during the interval), daily logging of system and process activity for long-term analysis, highlighting overloaded system resources by using colors, etc. At regular intervals, it shows system-level activity related to the CPU, memory, swap, disks (including LVM) and network layers, and for every process (and thread) it shows e.g. the CPU utilization, memory growth, disk utilization, priority, username, state, and exit code. In combination with the optional kernel module netatop, it even shows network activity per process/thread."
Found a presentation from Hisham about the effort.
I applaud the way you approached it - figuring out every single thing about a tool.
Also, I didn't know about 'od' command. Thanks again!
(can also dump characters and hexadecimal)
echo hello | od
echo hello | od -c
Useful when you want to figure out the real contents of a file.
man od
You asked for two words ;).
Many younger people have security issues explaining their knowledge with others, and sometimes exhibit this as criticism and acidity. It's possible that's what happened here.
I'm reminded of http://www.huffingtonpost.com/catherine-clennan/being-a-dumb... - it highlights the importance of always asking questions regardless of the context.
It's also possible that someone disliked your use of the phrase "two words", maybe because they interpreted it literally. Some things resist explanations that are both short and accurate.
Regarding od, it's a flexible format conversion tool that lets you convert arbitrary data to hexadecimal, octal and C escaped printable text. I use it a lot!
(PS. gia-web looks interesting)
Good job.
Actually, the second number is the sum of the number of seconds that all cores on the machine have spent idle, so that number may be higher than the first one. Source: https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...
My source (first result in Google) was this link https://www.centos.org/docs/5/html/5.1/Deployment_Guide/s2-p... which looks like an older version of the documentation that does not have that last sentence.
Doesn't help much with dtrace though!
Example: `gsed` actually parses whitespace the way I think it will; the builtin `sed` does not.
to be fair, you should probably call the GNU behaviour "GNUisms". The BSD behaviour is the historical POSIX behaviour. The GNU behaviour is, in a twisted sense, the result of embrace, extend and extinguish by the GNU people :-)
[1]: The most notable thing is that GNU generally allows flags to come before or after non-flag arguments, where POSIX requires them to come before.
You can't get kernel-specific things like strace, but they do have just about everything else. The big sell for me was being able to install the GNU versions of everything.
Macports keeps everything in /opt, so you also never have to worry about it clobbering anything that MacOS expects to work in a certain way.
edit: some background on Macports vs Homebrew - it's basically a vi/emacs kind of a thing. Homebrew utilizes the MacOS system stuff a fair amount, and installs everything to /usr/local. Macports tries to keep everything fully separate, and installs into /opt.
My personal experiences with Homebrew weren't great, but others will probably have the opposite opinion.
eg for htop: `sudo port install htop`
This means:
1) It is a security hole; any application can now modify system-wide applications at any time.
2) You can't have multiple users able to use homebrew.
I was expecting homebrew to be something similar to apt/yum/etc, but this makes it pretty fundamentally different.
How does Macports handle this? Does $USER still need write access to /opt?
It's not the worst approximation of ports, but it doesn't share the quality and care that goes into package maintenance and dependency management.
``` sudo chmod -R 770 /usr/local/ sudo chmod -R +a "group:Local allow delete,readattr,writeattr,readextattr,writeextattr,list,search,add_file,add_subdirectory,delete_child,file_inherit,directory_inherit" /usr/local/ ```
I remember sometimes running into permissions issues but nothing re-running that command didn't fix.
I moved away from homebrew, to compiling my own software on macOS which I just put in ~/build. I don't use sudo make install since I own the ~/build folder.
What am I doing wrong in this case?
no, this is not required anymore... https://github.com/Homebrew/brew/commit/a2f19f0a675f7cb384b4...
> You can't have multiple users able to use homebrew
what i did is i run Homebrew as a user i called Admin and when logged in as another user i simply do a 'su - admin' to run the brew commands..
See this change: https://github.com/Homebrew/brew/pull/896
First thing I installed in the mac store. Just like on linux, it just works and I get all my favourite tools.
Alternative to strace would be dtruss, which is implemented through dtrace.
I'd been using mostly Linux on the desktop since 1996, I manage servers for a living and couldn't live without a terminal. That's the nerdy side.
On the other hand, I was serious enough with photography and electronic music production to need best of breed tools, which are still largely Windows and Mac-only to this day.
There's also a time for everything, and there are days when regardless of your computing skills, you need your creative person/end user hat on and apps that "just work" to keep your right brain fully focused on the end result (e.g. "No thanks, I'm not going to get distracted compiling some Gimp plugin to get red-eye removal", which may or may not have changed since I last tried)
MacOS, in spite of its recent slight drop in quality, is still second to none in satisfying all these requirements at once.
This is the true 'unix philosophy.'
...wat?
Is that actually a thing?
You can see this via:
Note that the `apt` package itself does not require a MTA.Science! :D
I love the approach of systematically going through and prodding everything to see what things do and whether they're actually required. I remember doing this on my Windows 95 machine, stripping down running tasks using Task Manager. Of course, in Windows 95, if you killed the wrong task the system would just reboot... :P
If I recall correctly the CPU usage would still be 100% but could go up to 200% if both cores are fully used.
If only the UI had something like hover-tips to show what each field actually means...
A server can perfectly run without dbus, accountsservice, logind, policykit and acpid. And, unlike he wrote, timesyncd NTP synchronization works well without dbus.
(Debian base has also cron and rsyslog running by default, for legacy reasons.)
You can use your mouse/TrackPoint in htop to click on menu options, processes etc...
Overall, I really enjoyed the article.
https://dev.yorhel.nl/ncdu
Goodbye polkit!
Pretty OT, but does anyone know if that's true (or worthwhile?)?
It uses a session tracker (used to be Consolekit, these days it is systemd-logind) to tell if something is running under an active user session and, and checks that against a bunch of rules (stored in XML, natch) to see if what is being attempted is allows (and if not, spawn a password prompt so the user can elevate their privileges).
If you want to do anything "complicated" like powering down your PC by clicking a button in your DE of choice you need polkit (and entourage) to be present these days.
Its one of those CADT things that seems to lead to "year of the (linux) desktop" never happening.
I'd really like to step up my game for GUI monitoring apps without having to choose between either terminal log spam, PyQT, or a web server. Ideally I'd have something like htop/wavemon/power top that I could stream compactly via ssh.
Really excellent example of showing your work!
1: https://github.com/nicolargo/glances