Init should be just this: http://git.suckless.org/sinit/tree/sinit.c , systemd incorparates too much funcitonality IMO, I cant get used to it. Well, but its a free world, I have even written a Go init, some guy did Rust based, and upstart is still being used by Ubuntu 14.04, I like choices, and happy to see this alternative as well.
1. If the only thing you needed from init was to reap reparented zombie processes and manage to not die, why have init at all? Why not tell the kernel to skip reparenting processes (and just reap them internally if their parent is dead) and not treat pid 1 as special, allowing it to exit? That's less userspace code and less kernel code.
2. Speaking of kernel code, have you looked at how much stuff is going on in the kernel? Why does pid 1 get this much scrutiny, but the kernel, which definitely can't die, gets a free pass? On my system there are 77 kernel threads. Seventy-seven! What are they all doing??
And there are plenty of alternative kernels you can use. Multiple ones even have Linux binary compatibility.
3. Is the argument for suckless init the same as that for EWONTFIX init (on which it is based)? If so, would a fork of systemd that split it into two processes, PID 1 that reaps reparented zombies and tells PID 2 about it (over a one-way pipe, so PID 1 has no increased attack surface), be an acceptable option?
Incidentally, why would a zombie process need to be reparented? Surely it has just not been removed from the process table and all that needs to occur is to remove it fully... I can understand reparenting an orphan process, but a zombie process?
A ps roughly like that is how I got the number. That doesn't explain what they do. I'm sure I could dig into kernel sources, but... for instance, why am I running "xfsalloc", "xfs_mru_cache", and "xfslogd" when I have no XFS file systems?
The point I'm making is not so much that I'm curious (although I am), but that the kernel is exempt from the scrutiny that systemd gets, and the kernel has a ton of fragile crap running at full privilege. If systemd even shipped three XFS-specific binaries, people would be upset, but the kernel can run three persistent kernel threads and nobody cares.
I mean, I know exactly why it's there. :-) GRUB's os-prober is trying to mount all filesystems as all possible types, which autoloads xfs.ko.
My point isn't so much what my three kernel threads are doing as why nobody asks these questions, in general, about the kernel. Just about every criticism of systemd applies double to the kernel, but nobody is forking Debian to replace the kernel. (And very few people are contributing to the existing efforts already in Debian to officially support alternative kernels.)
Ah, I'm following :-) I think perhaps the kernel is more esoteric than systemd, and as a team they have proven themselves over time. I think they get a fair amount of scrutiny from competitors (the BSDs, and Microsoft) and they have a very robust development process so that might explain why it's not as controversial.
The kernel also lives in kernel space, so unless one go micro-kernel, and dumps everything but the bare minimum into user space, a kernel will always be a big blob of code (most of it drivers, btw).
Well, it makes sense that whatever spawns the daemons also keeps track of them, restarts them on failure, controls containers, disposes of zombie processes, and helps debugging init scripts.
It makes no sense for such tool to control network interfaces, logging, device sync, power mode changes, and well... I'm certainly forgetting one or two dozen systemd modules.
Sure it does. Letting systemd handle startup dependencies is a very good thing, IMO. You configure some unit files to allow for one unit to start after another, otherwise they run in parallel. Not sure what the issue is with that?
As for those other tools... well, the systemd init process doesn't manage those either. The anciliary modules do, and quite well too really. I'm not entirely sure why people think that this is much different to running a SysV script to execute start a daemon!
You are quite right, as the overall Debian's stance on systemd was far from praiseworthy. They allowed themselves to be pushed around too far by the "end-user" (desktop-users) mentality of the gnome and few such other upstreams.
This is a very good news. Indeed it would have better if the Debian had been a little bit more flexible and sensible on this issue.
Congrats Devuan folks! My heartfelt thanks too.
Edit: added my thanks and clarification for desktop-user.
I am not against them considering the end users. But getting pushed around by "end-user" (please read desktop-user) focus is way too unacceptable behavior, especially for a community like Debian.
Perhaps because end users only focus on one thing, be it faster boot, or better service management, but are rarely well versed enough to understand the long term implications of using a monolith like systemd.
It seems that GNOME had developed a strong dependency on systemd, which drove the whole process. I'm guessing "end-user" here means desktop user. I've used debian as a desktop and a server, and in the past it's been pretty easy to have it fill both roles; at this point though, systemd is basically forced onto a server system which involves a lot of changes and very little improvements for the server environment: the previous init scripts worked, my servers don't need to respond to hardware changes, or sleep, i didn't need a new ntpd, dhcpcd, syslogd, named (well ok, maybe, but unbound is what i want), anything else d.
Thanks for clarifying "end-user" here means desktop user. I did mean desktop user. It's so sad that the Debian community chose to ignore server users to achieve the goal of pleasing desktop users. Servers and many other users do not need the gnome at all.
And I am a server user of Debian and happy with systemd. Finally an init system that can reliably detect failing daemons and that allows me to set restarting policies.
This is fairly untrue. If you had an opinion that you don't like systemd, that's perfectly OK...but to cast doubt on the fairly open (and very brutal) CTTE process to select systemd is very unfair.
IMHO it is always good to doubt the status quo..but to discredit the winning fight put up by systemd against Shuttleworth and Upstart is very biased. The technical tradeoffs in systemd may not be worth it to you... But it won on technical merit.
FWIW I can't wait to see the next Holy War around snap packages.
Do users care though? I don't. I select a Linux distro based on how popular it is because frankly they are all pretty excellent these days.
I go for popular distros because it's more likely that bugs are found and community help is available.
I also do very little customization. If the default window system is Gnome, that's what I use. If it's KDE or something else, that's fine too. Again, they are all good enough.
I use Linux, I'm a huge fan of it, but I'm not dogmatic. Using Linux is a way for me to get work done, but it isn't the goal itself.
Some care. I don't think it'll ever be "mainstream". But I certainly care. I switched from Arch to Gentoo and systemd was one of the reasons.
And no, I don't really expect Gentoo to be something most people use. But to me that's the beauty of open source. If someone, or a group of people care enough, they can make it happen, if someone else finds it useful too doesn't necessarily matter.
Exactly, I use Debian because it has a large user base and community. But that's precisely why it's important than the mainstream distributions continue to support the traditional Unix init. People don't want to have to switch to some obscure distribution simply to avoid systemd.
There isn't much reason for a mainstream user to care what the initialization sequence looks like. Can they run the applications that they need to? That's about all a mainstream user cares about. Linux is just boring infrastructure that needs to be there. All the interesting stuff happens in user land.
It's a reasonable counterpoint, but limited to the part of systemd that manages starting and stopping services.
It also takes over virtual terminals, syslog services, login management, network interface configuration, dhcp, system time, udev device management, containers (machined, dbus, cgroups).
Also, the point of switching and studying is not clear. If all you need is service startup and keeping up, a few more polished tools exist with a dramatically ssmaller footprint.
I like systemd from the perspective of a service-writer/maintainer, the service files are nice and simple and don't have the repeated boilerplate of old init scripts.
However I still dislike its monolithic nature, its folding in of journaling and udev, and all the rest of the creep.
Ditto the first part. I have a .service file for haproxy, one for my node app, one for rethinkdb, they generally just work.
I understand people that dislike monoliths, but I also appreciate the DRY nature of systemd - the System V, bash, cron, at, various process monitors, syslog combo was a lot of code with a lot of overlap. I'd expect systemd is actually smaller in LoC that what it replaced.
This is the real point. I have my issues with systemd.
For example, I just had an interesting discussion on GH with Varnish packers about changing a daemon arg (in that case, the bind address/port).
The way systemd "recommends" to change the arguments passed to a daemon is to write a drop-in file (i.e. /etc/systemd/system/service-name.service.d/something.conf) that overwrites the ExecStart line. To do so, you have to do:
[Service]
ExecStart=
ExecStart=<original ExecStart line with wanted changes>
For Varnish, this isn't the end of the world - it only needs a few arguments - but it's still a lot less simple than the pre-systemd "standard" for Debian, which is to use a /etc/defaults/<service> file to allow changing daemon args via variables. This method can be still used (via either EnvironmentFile, or via built-in systemd "Environment=" statements) but the former is specifically mentioned as "not recommended" by Lennart, and the use of Environment= in service files seems very much like a feature that no one is keen to acknowledge or make use of, and it has it's own issues, as the way variables, white space, escaping etc are handled is specific to systemd and not immediately intuitive for someone used to a shell environment.
But as you said - those issues, are not enough to give up on the benefits of Debian stable. In theory, all packages in Debian stable that need an init script are supposed to still supply a sysvinit script (I was actually quite surprised by how many packages rely on systemd's ability to load sysvinit scripts and don't ship their own systemd service file) so you could choose to use sysvinit instead of systemd, for now at least.
I do appreciate the theoretical goal - you describe how your daemon should be started, and let pid1 handle the specifics of actually starting it - to me that makes it much easier for a lot more people to create reliable daemon packages.
I just wish the project wasn't taking such a "everything plus the kitchen sink" approach.
The Environment= thing I haven't seen any official comment about its recommended-ness but it has some quirks in syntax so it needs pretty rigorous testing before being used for anything that might have values that need to be quoted/escaped.
One major problem with systemd is that beta software is released as stable and incorporated in distros. I've just realized that systemd-coredumpd is used to save crashes in the journal and write the files somewhere to /var.
This has the problem that I have to vacuum or nuke the journal if I want to remove knowledge of past crashes because coredumpctl tool has no delete command. And coredumpctl hard-codes GDB as the debugger.
To fix all of this I've overwritten the systctl.d file and made it write somewhere I know and can find without the use of an unfinished tool.
I have also regularly had it happen that bootup would intermittently hang up or shutdown as well. There are also unfixed warnings and errors during shutdown on multiple machines.
My conclusion is that systemd uses the broader population for beta testing, even if you don't follow Fedora Rawhide or Arch rolling.
Nope. I've scoured previous bug report and discussions and the only sure way was to prevent systemd from swallowing core dumps. It had the nice side effect that I learned the format string and configured it to my personal needs. But having to reverse engineer in order to remove crashes is not stable software and just a tech preview.
If I wanted to be cynical, I'd believe anything coming out of systemd, regardless of quality, is accepted as the new way while discarding existing ways as obsolete.
I should add to my startup hangup remark that I never before had a startup problem that was intermittent with plain old stupid init from 1994 up until systemd became the distro default.
No, I didn't because I didn't want systemd-coredump to handle (and grab) core dumps in the first place and existing discussions and bug reports didn't appear to miss public awareness of the issue. Also previous experience with the maintainers left a bad taste in my mouth. I should be more professional, but the mental barrier to engage with the systemd community is rather high. I cannot say the same about other freedesktop teams with which I regularly communicate in bugzilla.
You don't have to be sorry, it's just that systemd as integrated in distros asks for regular QA work by the users, which I don't want to spend my time on. I'm happy to QA certain applications I follow closely, but in the past a Linux system once configured just worked even across major release upgrades. The patch OpenSUSE integrated to work around coredumpd was funny and quite all encompassing, all the while looking like they're trying very hard to close all possible avenues for it to creep back in. It's somewhere linked in an ArchLinux forum thread as a link to an opensuse github commit. Considering OpenSUSE distro devs as informed and capable that commit convinced me that at some point the workaround of overriding kernel.core_pattern in /etc/sysctl.d/50-coredumpd.conf might stop working. I don't look forward to that day and might switch to another distro before that.
Actually, that's not true. I've had situations where my configurations broke between major releases of Ubuntu. I reported the issue and eventually it got resolved (no idea how I'm afraid), but whatever the issue for me it must have been an edge case as it wasn't widely reported.
That seems to be the case for you as well. You have my sympathies, it's most frustrating when it occurs.
> I reported the issue and eventually it got resolved (no idea how I'm afraid), but whatever the issue for me it must have been an edge case as it wasn't widely reported.
Recently there have been major regressions in the kernel due to DRM atomic modesetting with bug reports with many confirmations and zero action from the maintainers while not being able to go past kernel 4.1 which is fortunately one of the long-term branches. 4.2 is also okay but there's no 4.2 lts branch.
I never used Ubuntu as my workstation OS, so I cannot talk about their QA standards or how well it would have worked for me, but I've observed sloppiness in their packaging and integration work on others' systems. I mean, it's also important to remember that the advertised number of Ubuntu packages is not truthful if you're honest, because it's a snapshot of Debian's repositories and thus anything in Universe is probably unmaintained when compared to the repository in Debian. That means, the curated set of packages is quite small in comparison. I hope the situation improved in the last years.
In that case I misunderstood you and owe you an apology. Sorry about that. I appreciate your viewpoint, even if I disagree with you. I think systemd is a bit of a sore spot for me so I'm probably a bit too insistent and forceful on this issue so if I overstepped the mark I apologise for that also.
Same thing with Lennart's previous project PulseAudio which also has no real stable releases and basically relies on distros to try and take the beta-quality upstream releases and turn them into something that actually works reliably. (My favorite was the PulseAudio release that shipped with a change that was literally incomplete, changing the interal resampling API without changing all the resamplers affected. Crashing ensued. Was rolled back in the next release, but that came with a whole new set of feature patches with the same level of QA applied.)
It probably doesn't help that they're both heavily developed by Red Hat, who see taking open source software and turning it into something stable and end-user viable as their secret sauce that justifies paying them lots of money.
I've been able to avoid Pulseaudio and therefore didn't experience the issues it had in the past but I've heard that the new maintainers fixed stuff and are doing a better job, though I still don't use it and hence don't have first hand experience.
I wouldn't blame Red Hat per se but the GNOME fall back they ship with RHEL is a joke and in no way resembles the old GNOME desktop Pixar was used to.
It's 2016 and I still have issues with pulseaudio, frankly I was somewhat surprised they let him replace init with systemd - given the terrible mess that pulseaudio has been historically.
And audio professionals working on JACK don't seem to have any interest in pushing it while Lennart has been selling PulseAudio as the low-power alternative to JACK. Maybe he would have been a better fit within Android.
I agree with the rest of your post but I can assure you it's not the policy of Red Hat to release unstable upstream software. I am a Red Hat employee and I'm as annoyed as anyone that systemd doesn't have separate upstream development and stable releases.
Also what's with the stupid versioning of systemd/udev? Having a single version number is simply a dumb idea.
Now that is a real problem! I checked the git tags, and the major versions are indeed added over different time intervals... I had no idea there wasn't a time based release schedule. Do you know what the reason for that is?
I can't believe anyone would like to keep using sysvinit-scripts or RC-scripts.
I love bash, I love nitpicking about how to use bash along with the other bash nazis in #bash@freenode. But I would never put myself through sysvinit again. I thoroughly enjoy using distros like CentOS 7, Fedora 23, RHEL7. They're a joy to manage and work with, in part thanks to Systemd.
I never want to debug, troubleshoot, or even look at another sysvinit-script again. :)
Until you install something like Gerrit, or GoCD, which have absolutely appalling init scripts. And then you find out that e.g. Gerrit sets some ulimits in the init script. Yuck!
If all systemd did was sysvinit-script replacement, I'd be 100% behind it. I agree with you, sysvinit-scripts are terrible. It's all the other stuff, and the lack of modularity, that makes me dislike it.
BSD handles all of this better than Linux. Text file. Easy. Linux has grown far too "complex". I like how OpenBSD adds modern touches yet keeps the OS free from binary blobs. systemd addresses some issues, but at the cost of complexity. I dislike binary blobs in my systems, hence, my servers run OpenBSD.
What binary blobs are you referring to? Aside from the, uh, binaries that run systemd, systemd services, and systemd utilities, the only other thing that is binary that I'm aware of is the binary log file, which can be reconfigured to revert to text if you so desire.
Aside from the binary logs, which are really not a great idea, systemd adds orders of magnitude more complexity to a system that really didn't need it. The init system could have been addressed without something like systemd. I'm old enough to remember and use Unix. Like a lot of IT guys my age, we all tend to think that adding complexity is wrong. This notion of one ring to rule them all mentality is really starting to take effect in IT. Unix-like operating systems are supposed to be non-complex. My example above, of OpenBSD, is true. Maintaining OpenBSD systems is trivial compared to similar Linux systems, and I maintain both. Sadly, I admin a series of CentOS PBX servers that are an absolute pain. A friend runs his on FreeBSD and has less than half the pain points I do. And we're running the same PBX software.
I believe in lowest common denominator everytime. Binary anything is bad when alternatives exist. Linux has given itself over to binary blobs in the name of being more attractive to corporate users. Device drivers under Linux are now almost all binary blobs. You will not see this in OpenBSD, for example. The BSD folk reverse engineer the drivers and write their own free drivers -- and they are usually far and away better afterwards.
Take the wireless chipset drivers for Ralink. The BSD guys took a 30,000 line driver and rewrote the entire thing in less than 1/3 of the code. That's just the whole attitude between Linux and BSD. Linux is a series of cobbled-together hacks on top of an ever-growing, ever-complex kernel (that really needs legacy crap removed). BSD, on the other hand, is engineered as a single OS with a small collection of pruned base programs. Big difference. If you've ever used BSD, you will know the joy of /etc/rc.conf vs the nightmare of Linux config files that live in myriad locations depending on the distro.
I can't see how that is the case. Why is this any less clear? I don't think following the Fedora debug guide is any less difficult than troubleshooting a SysV-init script.
It's curious that he says "Binary anything is bad when alternatives exist. Linux has given itself over to binary blobs in the name of being more attractive to corporate users. Device drivers under Linux are now almost all binary blobs."
That's not exactly true now, is it? He seems to be implying that there are closed source binaries in the Linux kernel...
You already answered the question and acted like it wasn't an answer: "Aside from the, uh, binaries that run systemd, systemd services, and systemd utilities," - yeah, those binary blobs (inherently unstable blobs of logic).
There isn't anything "inherently unstable" about them!
The only true competitor to systemd is s6, and you should see the amount of "binary" he has had to develop to get his service manager to run outside of PID 1!
However... it's all a moot point, because that isn't the definition of a "binary blob". In the context of Linux, or "free and open-source software, a binary blob is a closed-source binary-only piece of software without publicly available source code." [1]
You seem to forget that the bash scripts are the entire rc system. In systemd, the init binary and utilities are the entire rc system.
Those commands will not help you if you have to find a bug into systemd code to find something like a constant on the maximum size of a core dump and work around it. Then open a bug and hope it does not get ignored for ~3 years. But who needs core dumps anyway, right?
What happens when there's a bug in bash? Same thing. init is not the only component or source of complexity in the old way. Sometimes the problem will be in the script, sometimes it'll be in the thing that runs the script. Initscripts are vastly more complex than systemd service definition files (and the latter is declarative rather than procedural). Systemd pushes some of the complexity down the stack, and that is a reasonable choice. There will be more bugs in systemd than init and bash (or ash or dash), simply based on age and maturity of the respective projects.
There's complaints to be made about systemd, but I don't find this complaint compelling, at all. I think moving from procedural programs that are sometimes hundreds of lines long to declarative config files that are never more than a couple dozen is an overwhelmingly positive thing.
On Debian they're specifically not bash scripts, they're posix shell scripts and /bin/sh is provided by dash which is much lighter than bash.
I like the declarative nature of service files I'm just not a fan of the hoops it makes us jump through to achieve the same results (see my other comment about changing daemon args)
The bash scripts invoke programs and wrapper programs. It's a bit ridiculous to say that she'll scripting is the only thing that operates startup of services in a system.
It's not like new services and startup programs aren't constantly being introduced - so whilst the old scripts were pretty steady, new ones weren't always foolproof.
I'm not adverse to SysV and rc scripts, but they were never the panacea that some grey-beards make out. Dependency orders and parallelisation are particularly difficult, but it could be argued the scripts give control during the boot process.
I think in practice, it's fairly rate that the scripts were really needed - certainly systemd's more declarative nature for a problem that naturally lends itself to a dependency structure that has a directed acyclic graph seems like it's quite an effective an elegant approach!
Aside from the reasonable daemon-arguments issue you raise in the other comment (I just read this, interesting observation!) I really feel that systemd largely eliminates huge quantities of shell scripting in the startup process!
I've been wondering what scenarios there are that would require scripting. I thought that perhaps a more complex setup might be that a service needs to wait on two or three processes before starting, in which case a child can have multiple parents? Can systemd handle that?
Does it really work for others? Because I've yet to see any init system that provides even a small proportion of the functionality of systemd that I actively rely on.
I'd argue that one of the biggest things that made sysvinit so ugly was that distro maintainers didn't enforce any type of startup script hygiene. Take a look at BSD-style init scripts some time. They're beautiful in their simplicity and readability. They source a common set of functions, have well-defined variables to control execution, and end up with a length about that of systemd units, without sacrificing the power of shell scripting. Systemd threw the baby out with the bathwater, and just makes diagnosing boot problems all that more difficult when you have to start tracing through magic symlinks.
I actually came from FreeBSD to Debian 8 years ago and the funny thing is that I don't remember editing rc scripts anywhere near as often as I edited sysvinit scripts on Debian.
But that could easily be explained by the average complexity of services increasing along with my career path into Linux and Devops.
I don't remember any clear differences between sysvinit and rc, the scripting is still left up to the user which means code style can diverge from whatever the maintainers had in mind.
Systemd, albeit a binary blob, at least eliminates that from the equation and breaks down services into a limited number of types that it handles equally every time.
I actually liked upstart too. :) I might sound mad for saying that but as a user of Linux upstart gave me just what I wanted.
Bottom line is that I think discussions about scripting style have no place in the context of service management. Services should be dealt with in a simple way like upstart or systemd. If you want to add more complexity you can write a start script for the service but that should not be the norm.
This is commonly provided by the port maintainer - I haven't conducted a survey but I'm pretty confident that service scripts in FreeBSD ports rarely make it into rc.d without a deliberate freebsdization.
I have read all of the rc scripts that come with base FreeBSD and PC-BSD, and some of the NetBSD ones too. Coding style with Mewburn rc is nowhere near as divergent as it is with van Smoorenburg rc.
Yes I think Ports played a big role in keeping me out of rc scripts on FreeBSD. Ports just had everything you needed. But again, packages could play the same role on Linux but the reason they don't always do that is because of custom services, in-house developed, or proprietary purchased.
The biggest problem with java is knowing when the service is started. And that's actually a problem with other clustered services like wsrep in mariadb for example. You need to get an OK from the service, by checking logs or APIs, before you know that the restart worked.
So services like that will always require an added layer of abstraction. But that's not the norm, far from it.
On the contrary, and as I already said, in some fields it appears that it is. Moreover, as demonstrated, those services do not require all of that extra layering. Indeed, it actively breaks functionality.
Ironically, MariaDB is not one of those fields, and your chosen example is actually a counterexample. The push in MariaDB is to use systemd's "notify" readiness notification protocol and actually get rid of the wrapper shell scripts completely.
Why in the world would you need to "start tracing through magic symlinks" with Systemd?
If you mean to figure out what was actually executed for a unit "systemctl cat [service name]" will tell you the full path of each file systemd processed for that service and what content was pulled from it.
Because I had a situation where systemd broke a symlink causing a unit to be permanently disabled. Thus, I needed to look through the filesystem to find the offending mislinked magic and deleted unit files and repair them.
OpenRC fixes a lot of that, without adding the giant pile of dependencies that systemd imposes. As someone who is already comfortable with bash, OpenRC scripts should be easy.
> I can't believe anyone would like to keep using sysvinit-scripts or RC-scripts.
There are more options than sticking with Debian's ancient scripts.
The larger problem is that init is only a tiny part of systemd. Systemd's mandatory link-time dependencies and tightly-coupled (monolithic) features are trying to be an entire wrapper around important parts of the kernel. Even if you like the init features of systemd, you should be looking at all the changes; do you really want to depend on all of systemd's features in the future? Do you enjoy depending on a moving target that changes at the whim of the systemd cabal and their reputation for WONTFIXing anything that they don't understand or haven't considered as a use case?
"Systemd's mandatory link-time dependencies and tightly-coupled (monolithic) features are trying to be an entire wrapper around important parts of the kernel."
What parts of the kernel are you referring to? Genuinely interested...
> The larger problem is that init is only a tiny part of systemd.
That's because you can't reasonably fix the many problems with sysvinit (or OpenRC) without also providing a proper process manager, and dependency system that does far more than just trigger scripts in the right order.
The biggest problem with init is that it's just init.
> Do you enjoy depending on a moving target that changes at the whim of the systemd cabal and their reputation for WONTFIXing anything that they don't understand or haven't considered as a use case?
I don't enjoy that, but the alternative of relying on sysvinit or OpenRC is so many times worse.
Give me a better alternative, and I'll throw Systemd out of the window immediately. But to be better it will need to at a minimum:
- Be able to sanely handle service restarts, including configurable behaviour on failure.
- Be able to ensure service output is sanely logged.
- Be able to reliably keep track of processes belonging to a service even if they double fork or the prent process dies.
- Have a dependency system that allows starting and stopping services automatically depending on the status of other services, the availability of a directory or file.
- Be able to schedule execution (like cron) but still subject them to the process management and dependency rules.
> The biggest problem with init is that it's just init.
Why does init have to service supervision? What's wrong with having init just launch the service supervision process and afterwards just reap dead processes (which allows init to be a very simple program indeed [1]).
As for the requirements to the init+rc system.. mine look a bit different: it should be reliable and predictable first.. performance not so much.
OpenRC works pretty well for me, the only thing I would change right now is to replace that start-stop-daemon rubbish with runit (and there is work under way to allow that if I am not mistaken).
Now you will probably say that anything still using start-stop-daemon is strictly worse than whatever systemd does, because it won't restart the daemon at all if it crashes (and is ugly in some other ways too).
I would agree with that in theory, however in my case this happens rarely enough that it is the smaller evil versus the systemd system sometimes not booting properly at all.
And to them systemd is mana from heaven, as it allows them to bring up and shut down any number of nodes in their cattle farm swiftly.
Where before admins fretted about making sure a server stayed up when booted (and if it went down, make damn sure to figure out why so it didn't happen again), devops seems to adopt what i consider uptime by machinegun.
That server won't stay up for some reason? No problem. Lets just throw thousands more just like it at the task, like a machinegun throws bullets at a target.
Here are how services are monitored on a few different projects:
daemontools
A series of scripts that run the supervise program. It initially runs svscanboot [1], which enables logging and uses svscan [2] to look through a service directory in order to execute the supervise executable for each service. [3] Supervise loads the run command, and unlike runit (which was inspired by daemontools) when run ends, it waits a second and executes run again. As with runit, if it can read a down file, then it does not start the service. [3]
To control the service, you run svc [command] services, which writes to a the named pipe control. This pipe is being managed by supervise, which uses it to send a signal to the process depending on the control value it reads. [4]
When this uses System V init, you must add svscanboot to the inittab as a respawn process.
---
runit
Executes /etc/runit/2 and if this fails, restarts it. /etc/runit/2 generally runs runsvdir on a particular directory (normally /etc/service) [5] and once every 5 seconds checks if the time of last modification, the inode, or the device has changed, in which case it starts a new runsv process on the new directory.
runsv switches to the directory of the service and if the file down doesn't exist it executes the run script. Once the script exits, if there is a finish script it executes this. If there is no finish script, then runsv executes the run script again. [6]
To handle abnormal failure, you must write a script that handles it. When the run command exits, runsv passes the exit code (-1 if abnormal exit) and least significant byte of the exit status as determined by waitpid - useful if a signal is sent. [6]
You start and stop a service by using the sv program. The mechanism used in to create a pipe named control in the service directory, which runsv reads and when a control character is written to the pipe it first checks to see if there is a custom command it needs to run, if so then it runs it. runsv will send a signal to the process it is monitoring depending on what control character is written to the pipe, or unless the custom command returns 0 in which case no signal is sent. [6]
runit is meant to replace init, and run under PID 1. [7] However, it can still use it with the existing init on your system. [8] You can configure it to replace the logging facility via the <service>/log/run script - runit just redirects the services standard output to it as standard input and you just send that input to the logger.
---
OpenRC
Not a replacement for init, but instead is invoked by init. /sbin/openrc reads /etc/runlevels, builds a dependency graph and then starts the run scripts for each of the services in the correct order. Then executable ends. [9]
---
s6
Basically inspired by runit, except the guy who makes it feels that dependency management is important and should be run by PID1. [14]
To do this, he has created a minimal shell language called execline [10] which he has run as the first process by his program execlineb [11], creates a tmpfs filesystem and copies a base image prepared by s6-linux-init into it [12], has the process reload its environment variables then forks and blocks a child process. Next, it executes as PID 1 into the service dependency tool s6-svscan, which scans a service directory and loads some initial minimal services (the logger and an optional getty, etc).
Once this is done the child process unblocks and executes s6-rc-init, which requires a compiled service database and a services directory for it to scan. After this is done, the system will be in a fully operational state.
To shutdown, PID 1 is sent a s6-svscanctl command and a shutdown script is issued.
---
In Summary
Here's the thing I see. With the exception of s6, which makes sense but seems overly complicated to me, all of these init systems have issues. daemontools polls to see if services need to be loaded, and has no dependency order. Runit...
> systemd uses socket activation for it's dependency tracking.
Not just that though. Otherwise it would have no dependency tracking at all for all those daemons that don't support socket activation (let alone stuff that has to be started in a certain order and doesn't communicate via sockets).
In regards to runit: it does have a mechanism for handling dependencies - starting the other service in the start script (not great imho, but works).
Also I don't quite see/understand your argument (edit: say why I should switch away from it) against OpenRC (or OpenRC+runit) - since it does do dependency tracking and has sane service scripts.
And regards to systemd making everyones life easier: not mine sadly - had plenty of problems I never had with other init+rc systems (OpenRC, the OpenBSD one, the old Arch one, etc.), while gaining nothing I didn't already have previously (hell.. doesn't even boot faster - on average it boots & shuts down slower than before - median about the same).
> And as with other systems, you use a utility to start and stop services - which some object to, but when you objectively look at it I really think you'd be hard pressed to be terribly worried about it.
You mean that you start/stop services with something like systemctl start foo?
Yeah, I don't see why one would have a problem with that.
> The ancilliary utilities are not running in PID 1 though.
I am not a fan of integrating everything under the kitchen sink (network management, dhcp, NTP, udev, cron, logging, etc.) into one big project though.
Sysvinit scripts were shit. But as the AVGN might say, systemd took that shit sandwich and served it up with a side of buffalo vomit. It doesn't solve the "rat's nest of dependencies" problem, it makes it worse. Because now you have more moving parts which hard-depend on each other in various intricate ways and introduce unreliability into a formerly reliable startup process, combining the worst aspects of monolithic software with the worst aspects of modular software. Because development of components of systemd takes place in a bubble, there's not much room for feedback -- like is this a sane way to do system logging, network discovery, process group management, etc. That's easy to integrate with other pieces of software from the community? The systemd way to solve the integration issues is "pressure the major distros to adopt systemd. Then they'll HAVE to integrate with us!"
It's possible to run sysvinit sanely, by having a few startup scripts that source in the appropriate boilerplate. Slackware's startup for instance has always been manageable.
You must have a machine built for it, since I've had tons and tons of niggling little issues with systemd. Random, unnamed services sometimes fail to start or shutdown. One of systemd's big claim to awesomeness is that the start scripts run simultaneously, yet that's all thrown out the window when "Attempting to start, attempt 55 1m30s remaining" bullshit happens constantly and randomly, without enough information to debug properly.
The troubleshooting is ridiculous, too, because of the ridiculousness that comes with journald. Why does it feel the need to make me run an awfully formatted, and potentially complex set of parameters to print that? Especially considering that 9/10 times, the error message from journald is vague, ungreppable without -A or -B, and is often significantly shorter than the "use journald to print the error" output. So frustrating. I don't know how people deal with it.
That said, I'm not a fan of sysvinit, either, but I don't have to deal with as much smoke and mirror behavior with it.
Try installing syslogd then in journald.conf you just change storage to 1 and ForwardToSyslog to 1. That should hopefully get your old text based logs back.
One giant program to rule them all was never the Unix way. System's monolithic architecture will be its downfall. Rather than stabilize I predict that systemd will get more and more complicated and eventualy fall due to unwieldy maintenance problems at which point someone will come up with a better more modular system.
Wasn't it? Multics was the modular system, Unix was the one where they said "That's silly, let's stick everything in a single kernel and not bother with all the flexibility because we don't need it."
People seem to ignore the kernel for some reason, but it is literally one giant program to rule them all, and it works very well. There have been many efforts to modularize it and move most of the complexity into independent units, and none have particularly worked out.
The thing is that the kernel, while large, has one defined purpose -- to provide abstractions of hardware resources and arbitrate those resources between programs -- and thus conceptual simplicity, even if the source tree is huge. The POSIX syscall API is simple and the rest is just implementation.
In contrast, systemd is conceptually complex -- its scope continues to expand to manage every little nook and cranny and configurable tidbit in the system. (I'm not arguing one way or another, just pointing out that the type of complexity is different.)
Really? How is it complex conceptually? The main concepts you really need to understand is that of units, and for that I found a very well written piece by Digital Ocean on the topic:
Aside from that, there are a number of services and utilities, but I'm not sure how they are conceptually harder to understand that any other similar program or utility on Linux...
The core of systemd revolves around units, but the "systemd ecosystem" has expanded to logging (journalctl), bus management, a new way to set the hostname (hostnamectl), a login manager, fstab parsing and disk management, a new session/login management framework, networking (systemd-netword, or is it netctl now, or do people use NetworkManager? I'm on Arch and ended up choosing netctl but I still don't understand all the tradeoffs), and I'm probably missing something.
Don't get me wrong -- I actually like some of the new thinking, appreciate parallelized bootups, and think service units are cleaner than the old /etc/rc.d mudball. I'd rather have things Just Work(tm) than play with wpa_supplicant and hibernate configurations on my laptop. But it's been a lot of new stuff to learn for someone used to the Old Ways, and it's kind of astonishing how fast systemd pieces have taken over different domains.
Yeah, it can make my head spin too sometimes. But I just don't think that rapid change makes things more inherently complex for a system per se, it just means folks like us have to do more reading (and more grousing, I quite enjoy doing that on occasion, I must admit).
Oh boy, I've just realised - I'm growing older! Eep.
In kernel space, the global lock is gone, there is a central API that everybody else uses with code that mostly don't touch each other.
Linux is very modular. It is just not a microkernel, and that is probably more due to historical performance bottlenecks than to architectural concerns.
It occurs to me that there is quite a bit of confusion around the terms "monolithic", "microkernel" and "modularity". The misconception is that a monolithic kernel isn't modular, which is very far from the truth.
Modules are so closely tied to the base kernel that they require recompilation for about every stable release and code changes for about every actual release. Almost nobody uses FUSE in production. DRI sorta works, but it's so closely tied to the kernel that people have loud objections to out-of-tree non-GPL modules using even bits of it.
Linux is pretty modular as an engineering project, in that there are ways for developers of one part of the code not to need to care about all the rest of it. But in the sense that systemd gets criticized for not being modular, Linux is also not modular. It is technically possible to implement either systemd's or Linux's APIs faithfully enough for things written to those APIs to work with your alternative implementation. systemd gets criticized because doing this is difficult in practice; it's far more difficult with Linux.
The Linux kernel itself is a a monolithic kernel and it beat out Hurd which was a micro kernel. I would say Linux has a history of certain programs being monolithic being a good thing.
That's because there are only a couple of guys working on it. Linux has had hundreds of contributors over the years, many of which work for large tech companies like Intel and get paid by their company for their contributions.
I would be fascinated if Hurd could get the same sort of developer momentum to be comparable to Linux. A completely new (as in 1.0 released) and usable FOSS Unix-alike would be quite welcome amongst Linux and the BSD's I think.
GNU's kernel efforts had an entire team debating, discussing, prototyping, and eventually working on Hurd before Linux even existed.
Linux was initially written and released by one guy.
Linux obviously gained a wider team while Hurd languished, but it's still interesting that GNU's kernel efforts started with far more initial human resources and were still quickly overshadowed by Linux.
Linux is a kernel. Monolithic and micro kernel architectures are a different problem to the Unix philosophy which specifically concerns design of userspace programs. I don't see how having pipes between kernel components would be a useful feature for a user.
Also, the userspace of GNU/Linux isn't called Linux. Linux isn't an operating system.
I think you're post-rationalizing. The kernel is an exception today, true, but that doesn't mean it needs to be special.
The established Unix philosophy of everything being "one program that does one thing" would be hugely beneficial if applied to an OS kernel. Wouldn't it be great you could just run different drivers and file system in userspace rather than having to compile a big kernel?
Not to mention that it could be much easier to develop them if they could just run as processes and die if they failed, as opposed to inducing a kernel panic.
> Wouldn't it be great you could just run different drivers and file system in userspace rather than having to compile a big kernel?
Except that it makes debugging almost impossible and the overhead of IPC is very large. I'd argue that most popular kernels today (k{Free,Open,Net}BSD, illumos, Linux) are monolithic because that model makes development much simpler than a microkernel (not for historical reasons). Hybrid kernels like NT are a compromise for efficiency against the benefits of having a microkernel.
The panic argument is a good point, until you realise that kernel drivers don't panic --they oops. And a kernel oops just kills the thread it was running in. Linux has kernel threads that can be killed, but they're all playing in the same address space. So you can think of that as having the advantages of a microkernel without the debugging issues.
Debugging would be easier in user mode, surely. You could attach with gdb or whatever.
Last I checked, IPC overhead was not a barrier to performance in modern microkernels. QNX and XNU (used by OS X) both use microkernel-type IPC and are fast. XNU is even able to survive with an old Mach kernel.
If a kernel driver ends up in a faulty state (accesses invalid memory, does the wrong interrupt thing) then surely the whole kernel will panic?
NT is frequently referred to as "hybrid", but it's a frequently misunderstood design. First, while it uses IPC internally, it's not really comparable to microkernels since there's no context switch; every participant in the IPC communication is in the kernel. This is done purely as a way to isolate internals behind a rigorous, message-oriented API. Secondly, almost all of NT's services run in the kernel. A fairly small number of services run in userland: There's the user-mode driver system (UMDF, available since Vista), and graphics drivers also have the option of running partly in usermode.
systemd has neither a monolithic architecture, nor does the systemd process itself do anything other than the following:
"[systemd] provides a system and service manager that runs as PID 1 and starts the rest of the system. systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, supports snapshotting and restoring of the system state, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic." [1]
It comes with a set of utilities that do a lot of things. But that doesn't make it monolithic. In fact, systemd is actually extremely modular.
Ironically, systemd has not gotten any harder to use since it was introduced, at least IMHO, but I guess as with anything YMMV.
But this myth about systemd being monolithic, it really needs to stop because it's not correct.
> It comes with a set of utilities that do a lot of things
"monolithic" vs "modular" has isn't about the number of binaries you build. This is about coupling. Replacing systemd itself is the modularity we're talking about, which becomes harder to replace (less modular) every time systemd annexes another set of features.
Do those utilities use only well-defined, simple interfaces both to the system and between utilities? Can I replace one or more of those utilities with something else that implements those simple, stable, and well-defined interfaces? This what we mean by modularity; each piece of the complex system can be considered in isolation, with only minimal dependencies.
When I can remove systemd-PID-1, OR keep using systemd-PID-1 but replace those utilities with other tools, only then will systemd be a "modular" set of tools. If I have to recompile daemons to remove systemd dependencies (libsystemd-journal0, libsystemd-login0), those daemons are now part of the systemd monolith.
> needs to stop
The only things that should stop are these attempts at changing word definitions.
Really? How do you figure that? It shouldn't be that hard to replace most of the components that come with systemd with replacements.
Do those utilities use only well-defined, simple interfaces both to the system and between utilities? Can I replace one or more of those utilities with something else that implements those simple, stable, and well-defined interfaces? This what we mean by modularity; each piece of the complex system can be considered in isolation, with only minimal dependencies.
Um, yes - you can.
When I can remove systemd-PID-1, OR keep using systemd-PID-1 but replace those utilities with other tools, only then will systemd be a "modular" set of tools. If I have to recompile daemons to remove systemd dependencies (libsystemd-journal0, libsystemd-login0), those daemons are now part of the systemd monolith.
There is no factual basis to this claim. There is only really one component that can't be easily replaced, and that's journald.
I challenge you to tell me which component of systemd other than journald can't be swapped out if you so desired.
Systemd's own documentation[1] disagrees. Several important interfaces - which are used by other parts of systemd- are marked as not being independently reimplementable. Another way to say that is that systemd considers those to be tightly coupled.
> There is only really one component that can't be easily replaced, and that's journald.
You're contradicting yourself. If I remove all of systemd, existing daemons will need to be recompiled because libsystemd-journald.so is missing. You don't just to pretend that isn't a dependency.
The common response that we should just keep the library file is an attempt to pretend this problem does not exist. Klaus Knopper even gave a talk recently[1] about this very problem, which is causing problems (some of which have workarounds) for the next version of Knoppix.
Interestingly, most of it is reimplementable independently elsewhere. But I think it's instructive to quote why they say what they say:
"A number of systemd's APIs expose Linux or systemd-specific features that cannot sensibly be implemented elsewhere. Please consult the table below for information about which ones these are.
Note that not all of these interfaces are our invention (but most), we just adopted them in systemd to make them more prominently implemented. For example, we adopted many Debian facilities in systemd to push it into the other distributions as well."
So yes, there are some interfaces that are tightly coupled to Linux interfaces, and there are a few things like unit configurations, generators, system updates and presets that are very specific to systemd and, like they say, don't make any sense being implemented outside of systemd!
> You're contradicting yourself. If I remove all of systemd, existing daemons will need to be recompiled because libsystemd-journald.so is missing. You don't just to pretend that isn't a dependency.
I don't believe I have contradicted myself. That component is a sticking point, and is a problematic dependency. I'm curious what the devuan guys did to sort this out. I assume they recompiled everything.
Edit: I'm watching the talk by Klaus, thanks for the link - very interesting! I think I'm a bit confused about his point about the kernel taking an extra 15 seconds to load... does that have anything to do with systemd? I would have thought that's something else.
I'm also not following his point about systemctl being a "GUI"... that's a command line utility, not a GUI based tool?
OK, he's not happy about journald's binary format. I get it, I'm not keen on it either. But I guess he didn't realise that in journald.conf you just change storage to 1 and ForwardToSyslog to 1 also...
He's concerned about parallelization on slower hardware. Fair enough... in that case, setup the unit files to set the order?
OK, and now he's concerned about Ubuntu's packaging... ?? I'm not entirely sure how that is a problem for systemd... and seems to be concerned about a sysvinit package that's not a standard part of Ubuntu any more? That seems a bit of a strawman...
Yup, that's what I thought! It takes 15 seconds when he replaces systemd - so not an issue with systemd in terms of bootup speed.
And now he's saying he doesn't like the fact that his system now requires multi-user session management. What? Oh come on, that's silly. He also doesn't know why the error is occuring...
Right, so the issues that he seems to have - I see the main issue is that some software links directly against systemd libraries. That is indeed not very good.
However, his issues about the difficulties of swapping over packages - well, he has his own distro and he's actually worked out after a bit how to replace it with System V. Linking libraries is not very good, but that's surely more of an issue with developers making bad dependency choices? or perhaps they find it convenient. And he worked out how to get around it with a shim anyway, which obviously increases his workload, but then again he's the one who wants to diverge from what all the other distros are doing!
Of course, there is the other point in that it could be argued that the SystemV system doesn't have features that are provided by systemd... which would tend to indicate that perhaps it's lacking in a few areas.
He also belabours the point of parallelisation. I cannot understand why he just doesn't change the before or after directives in the unit config files. That's really not much different to the pain of reordering services in the existing System V-based systems!
So I'm really not convinced :-) Anyway, that was rather a fun talk, I liked how he was troubleshooting the issue, more power to him that he got his system in the way he likes it.
No point in discussing systemd anymore quite frankly, it won, doubt that it's good it did, doubt the methods by which it won - I do both - but it won. Many of the major distributions are not really following the Unix philosophy anymore, heck, some stopped quite a long time ago to do so.
I run sever operating systems because I like to have the control, the minimalism, the elegance, the security. By trying to be more of a desktop orientated system, Linux actually looses what attracts people like myself to using it. I doubt they care, I don't think they have to, just as I don't have to use it.
"This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."[1]
"Much of the power of the UNIX operating system comes from a style of program design that makes programs easy to use and, more important, easy to combine with other programs. This style has been called the use of software tools , and depends more on how the programs fit into the programming environment - how they can be used with other programs - than on how they are designed internally. But as the system has become commercially successful and has spread widely, this style has often been compromised, to the detriment of all users. Old programs have become encrusted with dubious features. Newer programs are not always written with attention to proper separation of function and design for interconnection. [...]
This style was based on the use of tools : using programs separately or in combination to get a job done, rather than doing it by hand, by monolithic self-sufficient subsystems, or by special-purpose, one-time programs. [...]
One thing that UNIX does not need is more features. It is successful in part because it has a small number of good ideas that work well together. Merely adding features does not make it easier for users to do things - it just makes the manual thicker. The right solution in the right place is always more effective than haphazard hacking."[2]
Still reading at this point? Great systemd free Linux distributions: Gentoo, Funtoo, Void Linux, among many others and there is also something called BSD (FreeBSD, OpenBSD, DragonflyBSD)... you should check it out. Also see www.suckless.org
The "Unix philosophy" of text streams hasn't worked, and arguably never worked. BSD sockets aren't quite text streams. Terminals aren't quite text streams. Shellshock existed because someone decided that serializing functions via text streams was a good idea.
And if you look at the things you care about when running a daemon process -- sessions, controlling terminals, reliable restarts (without killing the wrong process), logging, starting dependencies in the right order, dropping privileges reliably etc. -- none of those things are good ideas that work well together. You can clearly see that things like double-forking and detaching from controlling terminals and syslog were all bolted on later, because the original system didn't work well, and the hacks don't work well either.
I run servers, and I'm excited about systemd because it makes all these things actually work. The Unix philosophy gave me tools that do one thing and kinda sorta do them okay, if you're lucky. systemd does multiple things, and it does them all well. Back when we ran FreeBSD, we had hard-coded sleeps to wait for the network and NFS to come up before we continued with boot, and we had regular problems with being too aggressive with those sorts of hacks and having machines that would get stuck before launching sshd. That is straight-up not a problem with systemd.
mountcritremote appears to be an rc.d script? So it either mounts a filesystem during boot, or doesn't. That doesn't solve the problem that I want most things to wait on NFS, but SSH and enough to make SSH useful to start up without it, so that I can get in in case NFS doesn't successfully mount.
It's also documented far more poorly than just about anything in systemd.
I'm a tad confused... you cite that famous Unix philosophy quote "This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together"... but I don't see how that doesn't apply to systemd?
I would imagine people have a hard time lumping: service management, logging, login, network management, etc together as 'one thing'. As a CentOS 7 user, I certainly wouldn't describe it as 'doing it well'.
I think people initially were pretty stoked for the revamped init process, and then it started heaping on a lot more stuff. Poettering's communication style and other history didn't help either.
Really? System V init, which is what it replaced, did all of those things. It handles daemons and starts login, depending on the runlevel, and the startup scripts turn on logging, network services - and you use those same scripts to stop and start services.
So what in particular, other than Lennart's supposed abrasive nature, do you have an issue with? I'm genuinely not following some of this reasoning, at least from a purely technical level.
Incidentally, really need to speak to some of the BSD guys about what they think of SysV-style init. Now that could prove to be an interesting flame war :-)
Scripts are just a simple abstraction that allows to manage all kinds of daemons, which allows SysV to happily work with anything it finds in the init directories.
We give you this thing that replaces half the system with utilities (re)written by us, but hey don't complain you're free use something else!
"Systemd" isn't really a single program, it's a collection of 69 binaries, that together handle a number of functions that were traditionally handled by separately-developed pieces of software. It handles init, user logins, the console, system event logging, network interface configuration, time+date settings, and device management (udev was merged into Systemd in 2012, apparently).
A lot of those components have interdependencies (so they can't all be replaced piecemeal), and some userspace things like Gnome have come to have dependencies on things provided by parts of Systemd.
This has benefits: Things tied tighter together know more about the states of other parts, and that enables various optimizations. This has issues: Replacing individual components (if you don't like their behavior) becomes more difficult. Whether it's worth it or not seems like a philosophical debate.
Udev (AFAIK a program that recognises and mounts plug-in devices, but I'm not sure) is developped by systemd guys and they recently told that they won't care about the case where it is used w/o systemd. It is an important part of GNU/Linux distros. Tho I used Arch for about two-three years I guess (since just before it switched to systemd as default til two monts ago), and I never had to fiddle with it, so I don't know how it affects people. Gentoo forked it to maintain a version that did not depend on systemd.
Yeah, that's precisely what it does - it's a daemon that interacts with the kernel for hotplug devices, etc.
Is udevd used by anything other than the kernel though? If not, then I don't think it's a huge issue if those who want to use a different init system maintain their own version!
Why waste all that effort of maintaining forks when it can be avoided? Systemd folks are being hostile to the wider community of Linux users with that.
Can the person down voting please explain why? I'm going to assume you believe I've been factually inaccurate. If that's the case, I don't mind the downvotes as much as you not explaining where I'm wrong or making any attempt to engage in discourse.
If I'm not wrong, then perhaps you should reconsider your actions, because I'll still say what I believe even though you may not like it, karma be damned!
> Have a look at Devuan, I think you'll find that they use the same udev as Ubuntu and Debian.
Yes, in amongst all of this systemd discussion, please do look at the notional actual topic of this page. And discover vdev. It's explicitly mentioned in the release notes.
The people currently mantaining udev do not want them to.
> Unless the systemd-haters prepare another
kdbus userspace until then this will effectively also mean that we will
not support non-systemd systems with udev anymore starting at that
point. Gentoo folks, this is your wakeup call.
udevd is used by everything, sometimes directly, other times indirectly. It's needed to create device nodes for everything from audio devices to webcams to storage devices to stuff like modems and terminals, graphics cards, etc. Most desktop environments call it directly for stuff like USB drive automounting. Some non-desktop systems like Android and OpenWRT with very constrained hardware have their own replacements for it, but that's not officially supported by the kernel developers and the kernel APIs it relies on aren't documented or guaranteed to remain stable - they want you to use udev (and soon that'll mean systemd too).
Basically, the only people who've managed to get away with not using udev are those with their own kernel fork and a completely customised stack on top. I'm not sure if any of them even use glibc or the GNU userland either.
The basic problem is that the same people that maintain udev also maintain the kernel API it relies on. How Torvalds have allowed this to continue for so long i do not know.
Never mind that udev existed as a independent project for nearly a decade before being folded into systemd, thus leaving various distros high and dry. Either adopt systemd, or develop an alternative to udev (that still have to rely on the badly documented and changing kernel API).
The way systemd is structured it's next to impossible to use any parts of it outside of systemd, nomenclature is ridiculous (system daemon, what's that even supposed to mean?), it suffers greatly from feature creep, took over logging using a binary format, gained basic firewall controls , assimilated udev, incorporated a bootloader, ... and some additional things people took/take offence with: introduced to quickly, criticism was shut out instead of being debated, promises were broken, main developers behind it are quite arrogant and often ignorant when dealing with complaints/bugs... et cetera
there is a reason why Lennart is good in stirring up controversy, and it's not because everyone is just against him for no reason.
some further reading/links that might interest you:
edit: systemd might be perfect for you or other people, I don't doubt that, it might make things easier for some, in some respects in might do some things better, but if the debate (as it is now) is whether it follows the unix philosophy or not, I don't think one can reasonably make the case that it does
> edit: systemd might be perfect for you or other people, I don't doubt that, it might make things easier for some, in some respects in might do some things better, but if the debate (as it is now) is whether it follows the unix philosophy or not, I don't think one can reasonably make the case that it does
I can and have made that case :-) I guess we'll agree to disagree.
> The way systemd is structured it's next to impossible to use any parts of it outside of systemd
> nomenclature is ridiculous (system daemon, what's that even supposed to mean?)
It's the daemon that runs the system. init, you realise, is a daemon process? As in, it's a process that runs in the background that is not under direct user control?
> it suffers greatly from feature creep
A lot of folks quite like the features it has introduced, but that's subjective
> took over logging using a binary format
journald can be configured to do text logging...
> gained basic firewall controls
It's a component of the system, which doesn't have to be used.
> assimilated udev
I just cannot understand why this is such a big deal. udev is a userspace daemon that interacts with the kernel, it has had zero negative effect on anyone and it makes total sense for the ones maintaining the init system to integrate it into their tree and maintain it going forward!
> incorporated a bootloader
You can swap it out. Why is this a concern?
> ... and some additional things people took/take offence with: introduced to quickly, criticism was shut out instead of being debated, promises were broken, main developers behind it are quite arrogant and often ignorant when dealing with complaints/bugs... et cetera
there is a reason why Lennart is good in stirring up controversy, and it's not because everyone is just against him for no reason.
I'll stick with technical issues, not personalities.
No technical reasons given as to the concern over systemd, only some personality clashes ("Kick these fucking carpetbaggers to the curb!" - now there's a convincing argument!)
Same author, same arguments - but with a side-dish of "what would the grey-beards think?" and concocted outrage over an apparent schism within the Linux community.
I understand this because I can't get used to systemd. The learning curve should be better than rc.local for my taste.
Problem is systemd helps you booting fast and this is important both in the cloud (where every minute is a billed), and in telecom (where you have a stringent SLA to cover).
Why? When I used Arch Linux I did not ever turn off my laptop, but hibernate it. I only ever booted it when I stupidly forgot to plug it in before the battery runs out. And 10-20 seconds spend on a sequential boot every 10-20 days is just no time. We spend at least 100x that time in the toilet every day.
Meh. Systemd is pretty great. The *Nix purists hate it, but it solves a lot of problems and does it fairly consistently. I love using it in Fedora now where journald has basically replaced syslog completely.
Honestly journald is probably the only thing I don't really like about systemd and not for the reason you might think I'm about to say. I just rarely need to use it, because everything pretty much just works 99% of the time. So I end up forgetting how to use journalctl by the time I need it again and have to Google/man. Yeah I know, that's my own problem.
Everybody mentions the Unix philosophy and that is a very valid point, but I would like to mention the freedom philosophy that open-source/Linux gives (or used to).
These guys who wanted systemd muscled their way in to get this software into the Debian system. They also adopted an "all or nothing" approach, depriving the greater community of something bigger than "do 1 thing and do it well", which is "freedom to do as you wish and choose".
Debian wouldn't have split had they maintained they made systemd opt-in or Poettering and co. maintained their own Debian-systemd version. That didn't gel well with Redhats version of what Open Source is, and so many were deprived of their freedom to NOT CHOOSE systemd BUT still use Debian.
For all the talk of how good/bad systemd is, the proponents for systemd should remember how they deprived others with the freedom to choose, no matter how "ignorant" or "backwards" the detractors are (or how "shit" sysvinit is).
What you observe is emblematic of the problem with Devuan, and a few of the Devuan people themselves acknowledge it. So much of the discussion centres upon "how to remove evil libsystemd0" and suchlike that what is lost is the supposedly intentional focus of the Devuan project on init freedom. That surely means the freedom to run systemd as well as to not run it.
There's such a long history of projects starting out of a desire to not change, and so many of them (most, I would wager) fizzle out soon after. Being against some specific feature or architectural change is not the same as having a future vision that can drive a person to work on something, for no pay, for years or decades. Anger fades...the people you're angry at ignore the new project, making it unsatisfying in terms of vengeance. And, again, it just doesn't stick with you the way a desire to do something new and better does.
I'm not even saying systemd is great (it's good and bad in all sorts of novel ways), I'm just saying that the people who are just reacting against it, when the industry and community has been so clear about it being the future they are passionate about, are unlikely to do anything more than spin their wheels for a few months, and then fade away (as a project, I hope they all stick around as contributors to OSS things).
188 comments
[ 3.0 ms ] story [ 236 ms ] thread2. Speaking of kernel code, have you looked at how much stuff is going on in the kernel? Why does pid 1 get this much scrutiny, but the kernel, which definitely can't die, gets a free pass? On my system there are 77 kernel threads. Seventy-seven! What are they all doing??
And there are plenty of alternative kernels you can use. Multiple ones even have Linux binary compatibility.
3. Is the argument for suckless init the same as that for EWONTFIX init (on which it is based)? If so, would a fork of systemd that split it into two processes, PID 1 that reaps reparented zombies and tells PID 2 about it (over a one-way pipe, so PID 1 has no increased attack surface), be an acceptable option?
The point I'm making is not so much that I'm curious (although I am), but that the kernel is exempt from the scrutiny that systemd gets, and the kernel has a ton of fragile crap running at full privilege. If systemd even shipped three XFS-specific binaries, people would be upset, but the kernel can run three persistent kernel threads and nobody cares.
I just tested this out:
My point isn't so much what my three kernel threads are doing as why nobody asks these questions, in general, about the kernel. Just about every criticism of systemd applies double to the kernel, but nobody is forking Debian to replace the kernel. (And very few people are contributing to the existing efforts already in Debian to officially support alternative kernels.)
That is, actually doing the work of init i.e. spawning daemons and services.
It makes no sense for such tool to control network interfaces, logging, device sync, power mode changes, and well... I'm certainly forgetting one or two dozen systemd modules.
As for those other tools... well, the systemd init process doesn't manage those either. The anciliary modules do, and quite well too really. I'm not entirely sure why people think that this is much different to running a SysV script to execute start a daemon!
* https://news.ycombinator.com/item?id=8904429
* http://unix.stackexchange.com/a/197472/5132
This is a very good news. Indeed it would have better if the Debian had been a little bit more flexible and sensible on this issue.
Congrats Devuan folks! My heartfelt thanks too.
Edit: added my thanks and clarification for desktop-user.
edit: added clarification for end-user.
why?
* http://askubuntu.com/a/626858/43344
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy...
* http://askubuntu.com/a/613814/43344
And read the doco that comes with your operating system (presumed to be Ubuntu, given that you say that you use upstart).
* https://wiki.ubuntu.com/SystemdForUpstartUsers
IMHO it is always good to doubt the status quo..but to discredit the winning fight put up by systemd against Shuttleworth and Upstart is very biased. The technical tradeoffs in systemd may not be worth it to you... But it won on technical merit.
FWIW I can't wait to see the next Holy War around snap packages.
https://www.debian.org/vote/2014/vote_003
I go for popular distros because it's more likely that bugs are found and community help is available.
I also do very little customization. If the default window system is Gnome, that's what I use. If it's KDE or something else, that's fine too. Again, they are all good enough.
I use Linux, I'm a huge fan of it, but I'm not dogmatic. Using Linux is a way for me to get work done, but it isn't the goal itself.
And no, I don't really expect Gentoo to be something most people use. But to me that's the beauty of open source. If someone, or a group of people care enough, they can make it happen, if someone else finds it useful too doesn't necessarily matter.
It also takes over virtual terminals, syslog services, login management, network interface configuration, dhcp, system time, udev device management, containers (machined, dbus, cgroups).
That, to me, is where more of the concern lies.
Which tools did you have in mind?
* http://blog.darknedgy.net/technology/2015/09/05/0/
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/da...
However I still dislike its monolithic nature, its folding in of journaling and udev, and all the rest of the creep.
I understand people that dislike monoliths, but I also appreciate the DRY nature of systemd - the System V, bash, cron, at, various process monitors, syslog combo was a lot of code with a lot of overlap. I'd expect systemd is actually smaller in LoC that what it replaced.
https://www.devuan.org
Downloading via torrent now, it's 10G via torrent? Anyways it seems the file server there is under pressure now.
I for one don't want to deal with systemd from the start. Never liked its intrusive, all-in-one, hard to debug design approach.
For example, I just had an interesting discussion on GH with Varnish packers about changing a daemon arg (in that case, the bind address/port).
The way systemd "recommends" to change the arguments passed to a daemon is to write a drop-in file (i.e. /etc/systemd/system/service-name.service.d/something.conf) that overwrites the ExecStart line. To do so, you have to do:
For Varnish, this isn't the end of the world - it only needs a few arguments - but it's still a lot less simple than the pre-systemd "standard" for Debian, which is to use a /etc/defaults/<service> file to allow changing daemon args via variables. This method can be still used (via either EnvironmentFile, or via built-in systemd "Environment=" statements) but the former is specifically mentioned as "not recommended" by Lennart, and the use of Environment= in service files seems very much like a feature that no one is keen to acknowledge or make use of, and it has it's own issues, as the way variables, white space, escaping etc are handled is specific to systemd and not immediately intuitive for someone used to a shell environment.But as you said - those issues, are not enough to give up on the benefits of Debian stable. In theory, all packages in Debian stable that need an init script are supposed to still supply a sysvinit script (I was actually quite surprised by how many packages rely on systemd's ability to load sysvinit scripts and don't ship their own systemd service file) so you could choose to use sysvinit instead of systemd, for now at least.
I do appreciate the theoretical goal - you describe how your daemon should be started, and let pid1 handle the specifics of actually starting it - to me that makes it much easier for a lot more people to create reliable daemon packages.
I just wish the project wasn't taking such a "everything plus the kitchen sink" approach.
The Environment= thing I haven't seen any official comment about its recommended-ness but it has some quirks in syntax so it needs pretty rigorous testing before being used for anything that might have values that need to be quoted/escaped.
This has the problem that I have to vacuum or nuke the journal if I want to remove knowledge of past crashes because coredumpctl tool has no delete command. And coredumpctl hard-codes GDB as the debugger.
To fix all of this I've overwritten the systctl.d file and made it write somewhere I know and can find without the use of an unfinished tool.
I have also regularly had it happen that bootup would intermittently hang up or shutdown as well. There are also unfixed warnings and errors during shutdown on multiple machines.
My conclusion is that systemd uses the broader population for beta testing, even if you don't follow Fedora Rawhide or Arch rolling.
1. https://www.freedesktop.org/software/systemd/man/coredump.co...
If I wanted to be cynical, I'd believe anything coming out of systemd, regardless of quality, is accepted as the new way while discarding existing ways as obsolete.
I should add to my startup hangup remark that I never before had a startup problem that was intermittent with plain old stupid init from 1994 up until systemd became the distro default.
That seems to be the case for you as well. You have my sympathies, it's most frustrating when it occurs.
Recently there have been major regressions in the kernel due to DRM atomic modesetting with bug reports with many confirmations and zero action from the maintainers while not being able to go past kernel 4.1 which is fortunately one of the long-term branches. 4.2 is also okay but there's no 4.2 lts branch.
I never used Ubuntu as my workstation OS, so I cannot talk about their QA standards or how well it would have worked for me, but I've observed sloppiness in their packaging and integration work on others' systems. I mean, it's also important to remember that the advertised number of Ubuntu packages is not truthful if you're honest, because it's a snapshot of Debian's repositories and thus anything in Universe is probably unmaintained when compared to the repository in Debian. That means, the curated set of packages is quite small in comparison. I hope the situation improved in the last years.
Peace :-)
It probably doesn't help that they're both heavily developed by Red Hat, who see taking open source software and turning it into something stable and end-user viable as their secret sauce that justifies paying them lots of money.
I wouldn't blame Red Hat per se but the GNOME fall back they ship with RHEL is a joke and in no way resembles the old GNOME desktop Pixar was used to.
Also what's with the stupid versioning of systemd/udev? Having a single version number is simply a dumb idea.
I love bash, I love nitpicking about how to use bash along with the other bash nazis in #bash@freenode. But I would never put myself through sysvinit again. I thoroughly enjoy using distros like CentOS 7, Fedora 23, RHEL7. They're a joy to manage and work with, in part thanks to Systemd.
I never want to debug, troubleshoot, or even look at another sysvinit-script again. :)
I believe in lowest common denominator everytime. Binary anything is bad when alternatives exist. Linux has given itself over to binary blobs in the name of being more attractive to corporate users. Device drivers under Linux are now almost all binary blobs. You will not see this in OpenBSD, for example. The BSD folk reverse engineer the drivers and write their own free drivers -- and they are usually far and away better afterwards.
Take the wireless chipset drivers for Ralink. The BSD guys took a 30,000 line driver and rewrote the entire thing in less than 1/3 of the code. That's just the whole attitude between Linux and BSD. Linux is a series of cobbled-together hacks on top of an ever-growing, ever-complex kernel (that really needs legacy crap removed). BSD, on the other hand, is engineered as a single OS with a small collection of pruned base programs. Big difference. If you've ever used BSD, you will know the joy of /etc/rc.conf vs the nightmare of Linux config files that live in myriad locations depending on the distro.
https://fedoraproject.org/wiki/How_to_debug_Systemd_problems
It's curious that he says "Binary anything is bad when alternatives exist. Linux has given itself over to binary blobs in the name of being more attractive to corporate users. Device drivers under Linux are now almost all binary blobs."
That's not exactly true now, is it? He seems to be implying that there are closed source binaries in the Linux kernel...
http://linoxide.com/linux-how-to/linux-systemd/
The only true competitor to systemd is s6, and you should see the amount of "binary" he has had to develop to get his service manager to run outside of PID 1!
However... it's all a moot point, because that isn't the definition of a "binary blob". In the context of Linux, or "free and open-source software, a binary blob is a closed-source binary-only piece of software without publicly available source code." [1]
1. https://en.wikipedia.org/wiki/Binary_blob
Those commands will not help you if you have to find a bug into systemd code to find something like a constant on the maximum size of a core dump and work around it. Then open a bug and hope it does not get ignored for ~3 years. But who needs core dumps anyway, right?
There's complaints to be made about systemd, but I don't find this complaint compelling, at all. I think moving from procedural programs that are sometimes hundreds of lines long to declarative config files that are never more than a couple dozen is an overwhelmingly positive thing.
On Debian they're specifically not bash scripts, they're posix shell scripts and /bin/sh is provided by dash which is much lighter than bash.
I like the declarative nature of service files I'm just not a fan of the hoops it makes us jump through to achieve the same results (see my other comment about changing daemon args)
It's not like new services and startup programs aren't constantly being introduced - so whilst the old scripts were pretty steady, new ones weren't always foolproof.
I'm not adverse to SysV and rc scripts, but they were never the panacea that some grey-beards make out. Dependency orders and parallelisation are particularly difficult, but it could be argued the scripts give control during the boot process.
I think in practice, it's fairly rate that the scripts were really needed - certainly systemd's more declarative nature for a problem that naturally lends itself to a dependency structure that has a directed acyclic graph seems like it's quite an effective an elegant approach!
Aside from the reasonable daemon-arguments issue you raise in the other comment (I just read this, interesting observation!) I really feel that systemd largely eliminates huge quantities of shell scripting in the startup process!
I've been wondering what scenarios there are that would require scripting. I thought that perhaps a more complex setup might be that a service needs to wait on two or three processes before starting, in which case a child can have multiple parents? Can systemd handle that?
I'm still very curious as to what binary blobs you are referring to though.
You mean all those init systems you relied upon for the decades before systemd...or did you just start using linux recently?
I rarely make use of the BSD scripts provided but I like them as simple examples of how various services are intended to be run.
I find there is a certain amount of uniformity and consistency and arguably coherence to the rc.d system.
But that could easily be explained by the average complexity of services increasing along with my career path into Linux and Devops.
I don't remember any clear differences between sysvinit and rc, the scripting is still left up to the user which means code style can diverge from whatever the maintainers had in mind.
Systemd, albeit a binary blob, at least eliminates that from the equation and breaks down services into a limited number of types that it handles equally every time.
I actually liked upstart too. :) I might sound mad for saying that but as a user of Linux upstart gave me just what I wanted.
Bottom line is that I think discussions about scripting style have no place in the context of service management. Services should be dealt with in a simple way like upstart or systemd. If you want to add more complexity you can write a start script for the service but that should not be the norm.
This is commonly provided by the port maintainer - I haven't conducted a survey but I'm pretty confident that service scripts in FreeBSD ports rarely make it into rc.d without a deliberate freebsdization.
van Smoorenburg rc and Mewburn rc are noticably different.
* https://news.ycombinator.com/item?id=11550802
They are both rc, by the way. Don't conflate init and rc.
> If you want to add more complexity you can write a start script for the service but that should not be the norm.
In some fields, it appears that it is.
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/sy...
The biggest problem with java is knowing when the service is started. And that's actually a problem with other clustered services like wsrep in mariadb for example. You need to get an OK from the service, by checking logs or APIs, before you know that the restart worked.
So services like that will always require an added layer of abstraction. But that's not the norm, far from it.
Ironically, MariaDB is not one of those fields, and your chosen example is actually a counterexample. The push in MariaDB is to use systemd's "notify" readiness notification protocol and actually get rid of the wrapper shell scripts completely.
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/un...
* https://mariadb.atlassian.net/browse/MDEV-5713
* https://github.com/MariaDB/server/pull/26
* https://github.com/MariaDB/server/pull/83
* https://github.com/MariaDB/server/commit/20c2ae39db3dd0ec4c3...
As in MySQL and PostgreSQL.
* http://bugs.mysql.com/bug.php?id=65809
* http://www.postgresql.org/message-id/E1aQnE5-0003Cj-CN@gemul...
* http://postgresql.nabble.com/PATCH-better-systemd-integratio...
If you mean to figure out what was actually executed for a unit "systemctl cat [service name]" will tell you the full path of each file systemd processed for that service and what content was pulled from it.
OpenRC fixes a lot of that, without adding the giant pile of dependencies that systemd imposes. As someone who is already comfortable with bash, OpenRC scripts should be easy.
> I can't believe anyone would like to keep using sysvinit-scripts or RC-scripts.
There are more options than sticking with Debian's ancient scripts.
The larger problem is that init is only a tiny part of systemd. Systemd's mandatory link-time dependencies and tightly-coupled (monolithic) features are trying to be an entire wrapper around important parts of the kernel. Even if you like the init features of systemd, you should be looking at all the changes; do you really want to depend on all of systemd's features in the future? Do you enjoy depending on a moving target that changes at the whim of the systemd cabal and their reputation for WONTFIXing anything that they don't understand or haven't considered as a use case?
What parts of the kernel are you referring to? Genuinely interested...
That's because you can't reasonably fix the many problems with sysvinit (or OpenRC) without also providing a proper process manager, and dependency system that does far more than just trigger scripts in the right order.
The biggest problem with init is that it's just init.
> Do you enjoy depending on a moving target that changes at the whim of the systemd cabal and their reputation for WONTFIXing anything that they don't understand or haven't considered as a use case?
I don't enjoy that, but the alternative of relying on sysvinit or OpenRC is so many times worse.
Give me a better alternative, and I'll throw Systemd out of the window immediately. But to be better it will need to at a minimum:
- Be able to sanely handle service restarts, including configurable behaviour on failure.
- Be able to ensure service output is sanely logged.
- Be able to reliably keep track of processes belonging to a service even if they double fork or the prent process dies.
- Have a dependency system that allows starting and stopping services automatically depending on the status of other services, the availability of a directory or file.
- Be able to schedule execution (like cron) but still subject them to the process management and dependency rules.
.. to mention some.
Why does init have to service supervision? What's wrong with having init just launch the service supervision process and afterwards just reap dead processes (which allows init to be a very simple program indeed [1]).
As for the requirements to the init+rc system.. mine look a bit different: it should be reliable and predictable first.. performance not so much.
OpenRC works pretty well for me, the only thing I would change right now is to replace that start-stop-daemon rubbish with runit (and there is work under way to allow that if I am not mistaken).
Now you will probably say that anything still using start-stop-daemon is strictly worse than whatever systemd does, because it won't restart the daemon at all if it crashes (and is ugly in some other ways too). I would agree with that in theory, however in my case this happens rarely enough that it is the smaller evil versus the systemd system sometimes not booting properly at all.
[1] https://github.com/rxse/lyrica/blob/master/lyrica.ml
And to them systemd is mana from heaven, as it allows them to bring up and shut down any number of nodes in their cattle farm swiftly.
Where before admins fretted about making sure a server stayed up when booted (and if it went down, make damn sure to figure out why so it didn't happen again), devops seems to adopt what i consider uptime by machinegun.
That server won't stay up for some reason? No problem. Lets just throw thousands more just like it at the task, like a machinegun throws bullets at a target.
daemontools
A series of scripts that run the supervise program. It initially runs svscanboot [1], which enables logging and uses svscan [2] to look through a service directory in order to execute the supervise executable for each service. [3] Supervise loads the run command, and unlike runit (which was inspired by daemontools) when run ends, it waits a second and executes run again. As with runit, if it can read a down file, then it does not start the service. [3]
To control the service, you run svc [command] services, which writes to a the named pipe control. This pipe is being managed by supervise, which uses it to send a signal to the process depending on the control value it reads. [4]
When this uses System V init, you must add svscanboot to the inittab as a respawn process.
---
runit
Executes /etc/runit/2 and if this fails, restarts it. /etc/runit/2 generally runs runsvdir on a particular directory (normally /etc/service) [5] and once every 5 seconds checks if the time of last modification, the inode, or the device has changed, in which case it starts a new runsv process on the new directory.
runsv switches to the directory of the service and if the file down doesn't exist it executes the run script. Once the script exits, if there is a finish script it executes this. If there is no finish script, then runsv executes the run script again. [6]
To handle abnormal failure, you must write a script that handles it. When the run command exits, runsv passes the exit code (-1 if abnormal exit) and least significant byte of the exit status as determined by waitpid - useful if a signal is sent. [6]
You start and stop a service by using the sv program. The mechanism used in to create a pipe named control in the service directory, which runsv reads and when a control character is written to the pipe it first checks to see if there is a custom command it needs to run, if so then it runs it. runsv will send a signal to the process it is monitoring depending on what control character is written to the pipe, or unless the custom command returns 0 in which case no signal is sent. [6]
runit is meant to replace init, and run under PID 1. [7] However, it can still use it with the existing init on your system. [8] You can configure it to replace the logging facility via the <service>/log/run script - runit just redirects the services standard output to it as standard input and you just send that input to the logger.
---
OpenRC
Not a replacement for init, but instead is invoked by init. /sbin/openrc reads /etc/runlevels, builds a dependency graph and then starts the run scripts for each of the services in the correct order. Then executable ends. [9]
---
s6
Basically inspired by runit, except the guy who makes it feels that dependency management is important and should be run by PID1. [14]
To do this, he has created a minimal shell language called execline [10] which he has run as the first process by his program execlineb [11], creates a tmpfs filesystem and copies a base image prepared by s6-linux-init into it [12], has the process reload its environment variables then forks and blocks a child process. Next, it executes as PID 1 into the service dependency tool s6-svscan, which scans a service directory and loads some initial minimal services (the logger and an optional getty, etc).
Once this is done the child process unblocks and executes s6-rc-init, which requires a compiled service database and a services directory for it to scan. After this is done, the system will be in a fully operational state.
To shutdown, PID 1 is sent a s6-svscanctl command and a shutdown script is issued.
---
In Summary
Here's the thing I see. With the exception of s6, which makes sense but seems overly complicated to me, all of these init systems have issues. daemontools polls to see if services need to be loaded, and has no dependency order. Runit...
Not just that though. Otherwise it would have no dependency tracking at all for all those daemons that don't support socket activation (let alone stuff that has to be started in a certain order and doesn't communicate via sockets).
In regards to runit: it does have a mechanism for handling dependencies - starting the other service in the start script (not great imho, but works).
Also I don't quite see/understand your argument (edit: say why I should switch away from it) against OpenRC (or OpenRC+runit) - since it does do dependency tracking and has sane service scripts.
And regards to systemd making everyones life easier: not mine sadly - had plenty of problems I never had with other init+rc systems (OpenRC, the OpenBSD one, the old Arch one, etc.), while gaining nothing I didn't already have previously (hell.. doesn't even boot faster - on average it boots & shuts down slower than before - median about the same).
> And as with other systems, you use a utility to start and stop services - which some object to, but when you objectively look at it I really think you'd be hard pressed to be terribly worried about it.
You mean that you start/stop services with something like systemctl start foo? Yeah, I don't see why one would have a problem with that.
> The ancilliary utilities are not running in PID 1 though.
I am not a fan of integrating everything under the kitchen sink (network management, dhcp, NTP, udev, cron, logging, etc.) into one big project though.
It's possible to run sysvinit sanely, by having a few startup scripts that source in the appropriate boilerplate. Slackware's startup for instance has always been manageable.
Systemd, like a lot of things coming out of redhat these days (firewalld), feels like bloatware to me.
Systemd, Firewalld, D-Bus, etc.. have no place in a server IMHO.
The troubleshooting is ridiculous, too, because of the ridiculousness that comes with journald. Why does it feel the need to make me run an awfully formatted, and potentially complex set of parameters to print that? Especially considering that 9/10 times, the error message from journald is vague, ungreppable without -A or -B, and is often significantly shorter than the "use journald to print the error" output. So frustrating. I don't know how people deal with it.
That said, I'm not a fan of sysvinit, either, but I don't have to deal with as much smoke and mirror behavior with it.
People seem to ignore the kernel for some reason, but it is literally one giant program to rule them all, and it works very well. There have been many efforts to modularize it and move most of the complexity into independent units, and none have particularly worked out.
In contrast, systemd is conceptually complex -- its scope continues to expand to manage every little nook and cranny and configurable tidbit in the system. (I'm not arguing one way or another, just pointing out that the type of complexity is different.)
https://www.digitalocean.com/community/tutorials/understandi...
Aside from that, there are a number of services and utilities, but I'm not sure how they are conceptually harder to understand that any other similar program or utility on Linux...
Don't get me wrong -- I actually like some of the new thinking, appreciate parallelized bootups, and think service units are cleaner than the old /etc/rc.d mudball. I'd rather have things Just Work(tm) than play with wpa_supplicant and hibernate configurations on my laptop. But it's been a lot of new stuff to learn for someone used to the Old Ways, and it's kind of astonishing how fast systemd pieces have taken over different domains.
Oh boy, I've just realised - I'm growing older! Eep.
In kernel space, the global lock is gone, there is a central API that everybody else uses with code that mostly don't touch each other.
Linux is very modular. It is just not a microkernel, and that is probably more due to historical performance bottlenecks than to architectural concerns.
Linux is pretty modular as an engineering project, in that there are ways for developers of one part of the code not to need to care about all the rest of it. But in the sense that systemd gets criticized for not being modular, Linux is also not modular. It is technically possible to implement either systemd's or Linux's APIs faithfully enough for things written to those APIs to work with your alternative implementation. systemd gets criticized because doing this is difficult in practice; it's far more difficult with Linux.
I would be fascinated if Hurd could get the same sort of developer momentum to be comparable to Linux. A completely new (as in 1.0 released) and usable FOSS Unix-alike would be quite welcome amongst Linux and the BSD's I think.
Linux was initially written and released by one guy.
Linux obviously gained a wider team while Hurd languished, but it's still interesting that GNU's kernel efforts started with far more initial human resources and were still quickly overshadowed by Linux.
Also, the userspace of GNU/Linux isn't called Linux. Linux isn't an operating system.
The established Unix philosophy of everything being "one program that does one thing" would be hugely beneficial if applied to an OS kernel. Wouldn't it be great you could just run different drivers and file system in userspace rather than having to compile a big kernel?
Not to mention that it could be much easier to develop them if they could just run as processes and die if they failed, as opposed to inducing a kernel panic.
Except that it makes debugging almost impossible and the overhead of IPC is very large. I'd argue that most popular kernels today (k{Free,Open,Net}BSD, illumos, Linux) are monolithic because that model makes development much simpler than a microkernel (not for historical reasons). Hybrid kernels like NT are a compromise for efficiency against the benefits of having a microkernel.
The panic argument is a good point, until you realise that kernel drivers don't panic --they oops. And a kernel oops just kills the thread it was running in. Linux has kernel threads that can be killed, but they're all playing in the same address space. So you can think of that as having the advantages of a microkernel without the debugging issues.
* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/mi...
Last I checked, IPC overhead was not a barrier to performance in modern microkernels. QNX and XNU (used by OS X) both use microkernel-type IPC and are fast. XNU is even able to survive with an old Mach kernel.
If a kernel driver ends up in a faulty state (accesses invalid memory, does the wrong interrupt thing) then surely the whole kernel will panic?
NT is frequently referred to as "hybrid", but it's a frequently misunderstood design. First, while it uses IPC internally, it's not really comparable to microkernels since there's no context switch; every participant in the IPC communication is in the kernel. This is done purely as a way to isolate internals behind a rigorous, message-oriented API. Secondly, almost all of NT's services run in the kernel. A fairly small number of services run in userland: There's the user-mode driver system (UMDF, available since Vista), and graphics drivers also have the option of running partly in usermode.
"[systemd] provides a system and service manager that runs as PID 1 and starts the rest of the system. systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, supports snapshotting and restoring of the system state, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic." [1]
It comes with a set of utilities that do a lot of things. But that doesn't make it monolithic. In fact, systemd is actually extremely modular.
Ironically, systemd has not gotten any harder to use since it was introduced, at least IMHO, but I guess as with anything YMMV.
But this myth about systemd being monolithic, it really needs to stop because it's not correct.
1. https://freedesktop.org/wiki/Software/systemd/
> It comes with a set of utilities that do a lot of things
"monolithic" vs "modular" has isn't about the number of binaries you build. This is about coupling. Replacing systemd itself is the modularity we're talking about, which becomes harder to replace (less modular) every time systemd annexes another set of features.
Do those utilities use only well-defined, simple interfaces both to the system and between utilities? Can I replace one or more of those utilities with something else that implements those simple, stable, and well-defined interfaces? This what we mean by modularity; each piece of the complex system can be considered in isolation, with only minimal dependencies.
When I can remove systemd-PID-1, OR keep using systemd-PID-1 but replace those utilities with other tools, only then will systemd be a "modular" set of tools. If I have to recompile daemons to remove systemd dependencies (libsystemd-journal0, libsystemd-login0), those daemons are now part of the systemd monolith.
> needs to stop
The only things that should stop are these attempts at changing word definitions.
Do those utilities use only well-defined, simple interfaces both to the system and between utilities? Can I replace one or more of those utilities with something else that implements those simple, stable, and well-defined interfaces? This what we mean by modularity; each piece of the complex system can be considered in isolation, with only minimal dependencies.
Um, yes - you can.
When I can remove systemd-PID-1, OR keep using systemd-PID-1 but replace those utilities with other tools, only then will systemd be a "modular" set of tools. If I have to recompile daemons to remove systemd dependencies (libsystemd-journal0, libsystemd-login0), those daemons are now part of the systemd monolith.
There is no factual basis to this claim. There is only really one component that can't be easily replaced, and that's journald.
I challenge you to tell me which component of systemd other than journald can't be swapped out if you so desired.
Systemd's own documentation[1] disagrees. Several important interfaces - which are used by other parts of systemd- are marked as not being independently reimplementable. Another way to say that is that systemd considers those to be tightly coupled.
> There is only really one component that can't be easily replaced, and that's journald.
You're contradicting yourself. If I remove all of systemd, existing daemons will need to be recompiled because libsystemd-journald.so is missing. You don't just to pretend that isn't a dependency.
The common response that we should just keep the library file is an attempt to pretend this problem does not exist. Klaus Knopper even gave a talk recently[1] about this very problem, which is causing problems (some of which have workarounds) for the next version of Knoppix.
[1] https://www.freedesktop.org/wiki/Software/systemd/InterfaceP...
[2] https://www.youtube.com/watch?v=lDXsw2ijRkw
Interestingly, most of it is reimplementable independently elsewhere. But I think it's instructive to quote why they say what they say:
"A number of systemd's APIs expose Linux or systemd-specific features that cannot sensibly be implemented elsewhere. Please consult the table below for information about which ones these are.
Note that not all of these interfaces are our invention (but most), we just adopted them in systemd to make them more prominently implemented. For example, we adopted many Debian facilities in systemd to push it into the other distributions as well."
So yes, there are some interfaces that are tightly coupled to Linux interfaces, and there are a few things like unit configurations, generators, system updates and presets that are very specific to systemd and, like they say, don't make any sense being implemented outside of systemd!
> You're contradicting yourself. If I remove all of systemd, existing daemons will need to be recompiled because libsystemd-journald.so is missing. You don't just to pretend that isn't a dependency.
I don't believe I have contradicted myself. That component is a sticking point, and is a problematic dependency. I'm curious what the devuan guys did to sort this out. I assume they recompiled everything.
Edit: I'm watching the talk by Klaus, thanks for the link - very interesting! I think I'm a bit confused about his point about the kernel taking an extra 15 seconds to load... does that have anything to do with systemd? I would have thought that's something else.
I'm also not following his point about systemctl being a "GUI"... that's a command line utility, not a GUI based tool?
OK, he's not happy about journald's binary format. I get it, I'm not keen on it either. But I guess he didn't realise that in journald.conf you just change storage to 1 and ForwardToSyslog to 1 also...
He's concerned about parallelization on slower hardware. Fair enough... in that case, setup the unit files to set the order?
OK, and now he's concerned about Ubuntu's packaging... ?? I'm not entirely sure how that is a problem for systemd... and seems to be concerned about a sysvinit package that's not a standard part of Ubuntu any more? That seems a bit of a strawman...
Yup, that's what I thought! It takes 15 seconds when he replaces systemd - so not an issue with systemd in terms of bootup speed.
And now he's saying he doesn't like the fact that his system now requires multi-user session management. What? Oh come on, that's silly. He also doesn't know why the error is occuring...
However, his issues about the difficulties of swapping over packages - well, he has his own distro and he's actually worked out after a bit how to replace it with System V. Linking libraries is not very good, but that's surely more of an issue with developers making bad dependency choices? or perhaps they find it convenient. And he worked out how to get around it with a shim anyway, which obviously increases his workload, but then again he's the one who wants to diverge from what all the other distros are doing!
Of course, there is the other point in that it could be argued that the SystemV system doesn't have features that are provided by systemd... which would tend to indicate that perhaps it's lacking in a few areas.
He also belabours the point of parallelisation. I cannot understand why he just doesn't change the before or after directives in the unit config files. That's really not much different to the pain of reordering services in the existing System V-based systems!
So I'm really not convinced :-) Anyway, that was rather a fun talk, I liked how he was troubleshooting the issue, more power to him that he got his system in the way he likes it.
I run sever operating systems because I like to have the control, the minimalism, the elegance, the security. By trying to be more of a desktop orientated system, Linux actually looses what attracts people like myself to using it. I doubt they care, I don't think they have to, just as I don't have to use it.
"This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."[1]
"Much of the power of the UNIX operating system comes from a style of program design that makes programs easy to use and, more important, easy to combine with other programs. This style has been called the use of software tools , and depends more on how the programs fit into the programming environment - how they can be used with other programs - than on how they are designed internally. But as the system has become commercially successful and has spread widely, this style has often been compromised, to the detriment of all users. Old programs have become encrusted with dubious features. Newer programs are not always written with attention to proper separation of function and design for interconnection. [...]
This style was based on the use of tools : using programs separately or in combination to get a job done, rather than doing it by hand, by monolithic self-sufficient subsystems, or by special-purpose, one-time programs. [...]
One thing that UNIX does not need is more features. It is successful in part because it has a small number of good ideas that work well together. Merely adding features does not make it easier for users to do things - it just makes the manual thicker. The right solution in the right place is always more effective than haphazard hacking."[2]
Still reading at this point? Great systemd free Linux distributions: Gentoo, Funtoo, Void Linux, among many others and there is also something called BSD (FreeBSD, OpenBSD, DragonflyBSD)... you should check it out. Also see www.suckless.org
[1] http://www.amazon.com/Quarter-Century-UNIX-Peter-Salus/dp/02...
[2] http://harmful.cat-v.org/cat-v/unix_prog_design.pdf
And if you look at the things you care about when running a daemon process -- sessions, controlling terminals, reliable restarts (without killing the wrong process), logging, starting dependencies in the right order, dropping privileges reliably etc. -- none of those things are good ideas that work well together. You can clearly see that things like double-forking and detaching from controlling terminals and syslog were all bolted on later, because the original system didn't work well, and the hacks don't work well either.
I run servers, and I'm excited about systemd because it makes all these things actually work. The Unix philosophy gave me tools that do one thing and kinda sorta do them okay, if you're lucky. systemd does multiple things, and it does them all well. Back when we ran FreeBSD, we had hard-coded sleeps to wait for the network and NFS to come up before we continued with boot, and we had regular problems with being too aggressive with those sorts of hacks and having machines that would get stuck before launching sshd. That is straight-up not a problem with systemd.
It's also documented far more poorly than just about anything in systemd.
Could you clarify?
I think people initially were pretty stoked for the revamped init process, and then it started heaping on a lot more stuff. Poettering's communication style and other history didn't help either.
So what in particular, other than Lennart's supposed abrasive nature, do you have an issue with? I'm genuinely not following some of this reasoning, at least from a purely technical level.
Incidentally, really need to speak to some of the BSD guys about what they think of SysV-style init. Now that could prove to be an interesting flame war :-)
We give you this thing that replaces half the system with utilities (re)written by us, but hey don't complain you're free use something else!
A lot of those components have interdependencies (so they can't all be replaced piecemeal), and some userspace things like Gnome have come to have dependencies on things provided by parts of Systemd.
This has benefits: Things tied tighter together know more about the states of other parts, and that enables various optimizations. This has issues: Replacing individual components (if you don't like their behavior) becomes more difficult. Whether it's worth it or not seems like a philosophical debate.
In terms of Gnome - well, Gnome is Gnome. I'm not going to start defending their decisions, I disagree with quite a few of them myself.
Is udevd used by anything other than the kernel though? If not, then I don't think it's a huge issue if those who want to use a different init system maintain their own version!
https://wiki.gentoo.org/wiki/Project:Eudev
Have a look at Devuan, I think you'll find that they use the same udev as Ubuntu and Debian.
It's not hostile if someone forks a project to meet their requirements but the old software is used by everyone else!
If I'm not wrong, then perhaps you should reconsider your actions, because I'll still say what I believe even though you may not like it, karma be damned!
Yes, in amongst all of this systemd discussion, please do look at the notional actual topic of this page. And discover vdev. It's explicitly mentioned in the release notes.
* https://github.com/jcnelson/vdev
* http://judecnelson.blogspot.co.uk/2015/01/introducing-vdev.h...
* http://www.linuxquestions.org/questions/debian-26/devuan-417...
> Unless the systemd-haters prepare another kdbus userspace until then this will effectively also mean that we will not support non-systemd systems with udev anymore starting at that point. Gentoo folks, this is your wakeup call.
https://lists.freedesktop.org/archives/systemd-devel/2014-Ma...
Basically, the only people who've managed to get away with not using udev are those with their own kernel fork and a completely customised stack on top. I'm not sure if any of them even use glibc or the GNU userland either.
Never mind that udev existed as a independent project for nearly a decade before being folded into systemd, thus leaving various distros high and dry. Either adopt systemd, or develop an alternative to udev (that still have to rely on the badly documented and changing kernel API).
http://www.landley.net/notes-2015.html#05-07-2015
At this point in time it seems that GregKH has mastered the art of passive-aggressiveness...
there is a reason why Lennart is good in stirring up controversy, and it's not because everyone is just against him for no reason.
some further reading/links that might interest you:
https://lkml.org/lkml/2014/8/12/459
http://without-systemd.org/wiki/index.php/Arguments_against_...
http://www.infoworld.com/article/2608798/data-center/systemd...
http://www.infoworld.com/article/2608864/data-center/choose-...
http://www.infoworld.com/article/2608870/linux/you-have-your...
http://blog.darknedgy.net/technology/2015/10/11/0//
http://suckless.org/sucks/systemd
edit: systemd might be perfect for you or other people, I don't doubt that, it might make things easier for some, in some respects in might do some things better, but if the debate (as it is now) is whether it follows the unix philosophy or not, I don't think one can reasonably make the case that it does
> edit: systemd might be perfect for you or other people, I don't doubt that, it might make things easier for some, in some respects in might do some things better, but if the debate (as it is now) is whether it follows the unix philosophy or not, I don't think one can reasonably make the case that it does
I can and have made that case :-) I guess we'll agree to disagree.
> The way systemd is structured it's next to impossible to use any parts of it outside of systemd
Well that's not true. If you want to swap out a systemd daemon with another one, then you just have to write another service configuration file - see here: https://www.freedesktop.org/software/systemd/man/systemd.ser...
> nomenclature is ridiculous (system daemon, what's that even supposed to mean?)
It's the daemon that runs the system. init, you realise, is a daemon process? As in, it's a process that runs in the background that is not under direct user control?
> it suffers greatly from feature creep
A lot of folks quite like the features it has introduced, but that's subjective
> took over logging using a binary format
journald can be configured to do text logging...
> gained basic firewall controls
It's a component of the system, which doesn't have to be used.
> assimilated udev
I just cannot understand why this is such a big deal. udev is a userspace daemon that interacts with the kernel, it has had zero negative effect on anyone and it makes total sense for the ones maintaining the init system to integrate it into their tree and maintain it going forward!
> incorporated a bootloader
You can swap it out. Why is this a concern?
> ... and some additional things people took/take offence with: introduced to quickly, criticism was shut out instead of being debated, promises were broken, main developers behind it are quite arrogant and often ignorant when dealing with complaints/bugs... et cetera there is a reason why Lennart is good in stirring up controversy, and it's not because everyone is just against him for no reason.
I'll stick with technical issues, not personalities.
Just reviewing your links:
https://lkml.org/lkml/2014/8/12/459
No technical reasons given as to the concern over systemd, only some personality clashes ("Kick these fucking carpetbaggers to the curb!" - now there's a convincing argument!)
---
http://without-systemd.org/wiki/index.php/Arguments_against_...
Yeah, I've read most of these, I'm really not convinced by any of the arguments
---
http://www.infoworld.com/article/2608798/data-center/systemd...
The guy rehashes the same arguments you are making, and I've already addressed each of them - I disagree!
---
http://www.infoworld.com/article/2608864/data-center/choose-...
Same author, same arguments - but with a side-dish of "what would the grey-beards think?" and concocted outrage over an apparent schism within the Linux community.
---
Problem is systemd helps you booting fast and this is important both in the cloud (where every minute is a billed), and in telecom (where you have a stringent SLA to cover).
These guys who wanted systemd muscled their way in to get this software into the Debian system. They also adopted an "all or nothing" approach, depriving the greater community of something bigger than "do 1 thing and do it well", which is "freedom to do as you wish and choose".
Debian wouldn't have split had they maintained they made systemd opt-in or Poettering and co. maintained their own Debian-systemd version. That didn't gel well with Redhats version of what Open Source is, and so many were deprived of their freedom to NOT CHOOSE systemd BUT still use Debian.
For all the talk of how good/bad systemd is, the proponents for systemd should remember how they deprived others with the freedom to choose, no matter how "ignorant" or "backwards" the detractors are (or how "shit" sysvinit is).
What you observe is emblematic of the problem with Devuan, and a few of the Devuan people themselves acknowledge it. So much of the discussion centres upon "how to remove evil libsystemd0" and suchlike that what is lost is the supposedly intentional focus of the Devuan project on init freedom. That surely means the freedom to run systemd as well as to not run it.
* https://lists.dyne.org/lurker/message/20150208.110824.579d17...
And as you can see, that bleeds over into the press coverage seeking headline material.
Does anyone know if Devuan have given it any consideration?
[0] https://lwn.net/Articles/512719/
There's such a long history of projects starting out of a desire to not change, and so many of them (most, I would wager) fizzle out soon after. Being against some specific feature or architectural change is not the same as having a future vision that can drive a person to work on something, for no pay, for years or decades. Anger fades...the people you're angry at ignore the new project, making it unsatisfying in terms of vengeance. And, again, it just doesn't stick with you the way a desire to do something new and better does.
I'm not even saying systemd is great (it's good and bad in all sorts of novel ways), I'm just saying that the people who are just reacting against it, when the industry and community has been so clear about it being the future they are passionate about, are unlikely to do anything more than spin their wheels for a few months, and then fade away (as a project, I hope they all stick around as contributors to OSS things).