Another vector: some services may create users and services on request. You don't have to trick the administrator to setup the account if the main purpose of some SaaS/PaaS is to take user's code and run it in local environment - possibly as a service, with the username controlled by the user.
The attack vector this article doesn't mention is systems where systemd unit files are generated automatically.
For example, on some shared clusters I used there was a capability to request a service run as your user. This was before systemd at the time so it would end up generating an /etc/init.d entry, but I imagine a modern equivalent might generate a unit file.
So this article downplays the vulnerability a bit by claiming you have to trick a sysadmin when you could actually just use an automated service management system.
Setting aside issue with user names starting with a digit: Why on earth would anyone use "root" as default, fall back value for user name? In case of unparseable/unrecognized value the sane thing to do would be to raise an error.
The name is just something easy to remember. But "root" will always be user 0 (zero).
Gobolinux as a joke used to ship with user 0 named gobo rather than root, but i think they stopped doing so because it broke certain scripts etc that assumed root and 0 was equivalent.
My impression is that the script is being run as root. It's going along, running everything as root... It encounters an attempt to run a command as a different user, and being unable to find a user matching that, says okay, whatever, I'll just run that as root too.
It is kinda funny how systemd will bail to the emergency console if it can't figure out the dependencies chain. but it will happily run something as root if told to not do so, but can't find the user account it it told to use.
If you provide a valid username (by systemd definition) of a user who doesn't exist, systemd won't run the unit. The behavior is just consistent with unknown/invalid options (IMO, User= should be an exception to this).
It isn't even consistent with unknown options, because the option User is known. Once this is recognized, there should be no difference between nonexistent and invalid usernames.
I don't even want to know how their code is written so that it allows such error to happen and remain undetected.
If it can't find a matching user, it aborts as an error. When it doesn't think the value could possibly be a valid user, then it ignores the rule to run as a different user completely.
Err, there are such things as user level systemd unit files[0]. I have no idea if this bug gets triggered by those as well, but it's a pretty big deal if it is.
> There's a new systemd bug that gets the haters aroused!
Oiy... I feel like I am walking into the middle of a Vim/Emacs flame war. Very unfortunate.
[EDIT] As noted downstream, the parsing error causes the 'User=' directive to just be ignored, so a user unit file will be run by the user. Still too many vectors of attack, sadly.
Yeah, Mattias is super defensive in this. It doesn't help your cause to downplay serious vulnerabilities in the software you're championing. Anyone acting like this is a point for upstart/sysvinit/whatever is obviously misguided; there's no need to preempt that misguidance.
> I have no idea if this bug gets triggered by those as well
Why should it? The bug is not that it runs as root but that it encounters a parsing error and uses the default. The default for root is root and the default for user A is user A.
Thank you for the correct, if condescending, answer. It was unclear from the original bug, or this article, if it was falling back on a hardcoded 0 or the user used when the directive is not present.
Since systemd's own behavior is inconsistent, I can't depend on that behavior to derive the reasonable answer. That inconsistency stems from the fact that a "valid" user name that doesn't exist triggers different behavior than an "invalid" user name; and the definition of "valid" is different between systemd and the rest of Linux.
Another attack vector: An administrator deliberately creates a user starting with a 0, for example if they chose to name it after a system they are using (e.g. www.0xproject.com).
They assume that the program is running as a user with limited privileges. Then, an exploit gets found in that program, and now instead of an attacker gaining user-level access, they now have root-level access.
By far the most worrying part of the issue for me was that the developers said "oh it's not a bug" and just closed the issue, without properly thinking through the implications of the behaviour (such as the other attack vectors mentioned here). That's gross negligence.
I agree finding a bug is part of the development lifecycle. Having the developer just close the bug and claim its not an issue is the worrisome part. When the next bug is found there is less reason to report it.
No, i am saying that at the height of gamersgate there seemed to be a mob of people that went around harassing (trolling if you will) the people involved simply because they found doing so amusing.
And gamersgate lost media attention, my impression that a sizeable chunk of that mob has latched onto systemd as their target.
The problem is that the people being harassed can't separate the harassers from the people voicing actual complaints, lumping them all into the trolls/haters bin.
And frankly gamersgate was just the "peak" of a sorts, the root of it all lay further back. It is basically a mutated, nastier, form of the antics done under the "anonymous" moniker.
I see no reason not to hound out bad people from the developer community. It is a legitimate reaction to people whose offence goes far beyond writing bad software.
1) while being adult men in their prime, they spare no time running coordinated trolling campaigns not unlike people from places like 4chan. This is sick; 2) They attempt to outshout and bury valid criticism online and offline, responding with ad hominem attacks to very correct, purely technical articles; 3) Purposefully derailing conversations pointing to all aforementioned; 4) Aggressive advocacy - leveraging their their network of influence to sabotage 3rd party software by intentionally breaking compatibility.
I have no different words other than "waging war" for what SystemD and RedHat do to Linux development community
Are you talking about the /g/ "systemd is bloat"
meme? Because /g/ hates everything, and /g/ and Gamergate are hardly coterminous.
To be clear, I'm not objecting to potshots taken at either; /g/ is trash, and whining about vidya gaem journalism is childish and stupid - and lashing out viciously at people who work in the field is worse by far than that. I'm just saying it's worth checking your target first.
"So, yeah, I don't think there's anything to fix in systemd here. I understand this is annoying, but still: the username is clearly not valid."
And that's basically the problem. Not that the bug exists. But the systemd author doesn't recognize it as such, and refuses to fix it. This seems to be a recurring theme with systemd.
I understand the criticism towards systemd but let's not start making things up. From useradd (8) manual:
"Usernames must start with a lower case letter or an underscore, followed by lower case letters, digits, underscores, or dashes. They can end with a dollar sign. In regular expression terms: [a-z_][a-z0-9_-]*[$]?"
It is usually recommended to only use usernames that begin with a lower case letter or an underscore, followed by lower case letters, digits, underscores, or
dashes. They can end with a dollar sign. In regular expression terms: [a-z_][a-z0-9_-]*[$]?
On Debian, the only constraints are that usernames must neither start with a dash ('-') nor plus ('+') nor tilde ('~') nor contain a colon (':'), a comma (','),
or a whitespace (space: ' ', end of line: '\n', tabulation: '\t', etc.). Note that using a slash ('/') may break the default algorithm for the definition of the
user's home directory.
As dlgtho notes, your system's useradd(8) man page is not a description of all existing systems. Historically, the prohibitions were against ‘:’ (breaks passwd(5)), initial ‘-’ (mistaken for an option by login(1)), and upper case without lower case (makes getty(8) think you have an upper-case-only terminal).
(Not much point in continuing this, since the post has dropped from front page to 7th in half an hour, and will never be seen again.)
What the standards say supports that assertion even less than what the other manual pages say. There is a standard for this. It is IEEE 1003.1 a.k.a. The Single Unix Specification.
So in that case, when systemd encounters a username it is rejecting, it should throw an error instead of launching the process as root.
What happens when admin typos a username in this fashion? The system they think is conforming to least privilege suddenly is granting most privilege. Thanks systemd!
It's like the code is as self-righteous as its author.
This very article demonstrates that the largest enterprise Linux distros -- Red Hat and CentOS 7.3-- both recognize and support usernames beginning with a digit.
I see no reason not to hound out bad people from the developer community. It is a legitimate reaction to people whose offence of Linux development community goes far beyond writing bad software.
No, personal attacks are never acceptable. Doubly so if you're an outsider who has done nothing for the project except attack the main developer.
The legitimate way would be to take part in the development and convince based on technical reasons. If people disagree with you, consider that you might be wrong. If you think you're right anyway, fork it, and see if /other/ people agree with you. Chances are they won't.
The people actually maintaining all the distros didn't all switch to systemd within a few years for no reason.
>The people actually maintaining all the distros didn't all switch to systemd within a few years for no reason.
They switched out of prospect having unmaintainable initscripts and having DEs being turned useless. RH/SystemD liaisoning with distro maintainers at around 2013 can be said to be trollish blackmail.
Again, the point here is how apaling is the conduct of RH/SystemD core developers. SystemD could've been treated as a simple piece of bad software, if not for the group of people pushing it having such toxic personalities.
You do not confront trolling with bugreports and patches.
May I propose Moomin's Law: Any statement about how hard a security issue is exploit is unlikely to be true unless made by someone who has exploited it.
To me, this reflects as much on "many eyes make all bugs shallow" as on systemd. How many have used this feature without noticing that it has a serious bug? How many pairs of eyes have even passed over that particular piece of code without seeing what was wrong with it? More eyes only make bugs shallow when they're trained and diligent eyes.
Systemd is the disaster that happens when
we ignore the lessons of UNIX.
UNIX is all about small, composable tools.
Do one job and do it well.
One of these tools didn't stand up to the test
of time (SysV init) and needed to be replaced.
We could have chosen one of the mature candidates
(upstart, runit, minit) or even built a great new
one from scratch. Life would have been good.
But instead we ended up with this trainwreck
of a monolith that not only replaced init, but at the
same time tried to reinvent almost every critical
userspace daemon from Syslog to NTP.
It predictably failed on every metric.
Now it is time for the major Linux distributions to
conclude this experiment, roll back the Systemd mistake
and return to the modularity that made UNIX successful.
This attitude does not match history. If Unix really was about that philosophy, we would have never seen tools like emacs and perl gain any sort of popularity.
UNIX philosophy[1] is not about popularity but about choice.
We get to choose between Emacs/Vim, Perl/Python,
Syslog-ng/RSyslog, ntpd/openntpd and so on.
This has resulted in multiple mature implementations
for every core system task.
Yet instead of sticking with this code that has stood the
test of time over decades, we threw it all out for a poorly
written jack of all trades that gives root to usernames
starting with a digit...
Perhaps someone can explain why it was chosen as the replacement for SysV init, when all of the parent's comments/critiques of systemd were known in advance of it being pulled into major distros.
It is init, inetd, session manager (logind), udev, dns client, dhcp client, cron, su, and the list keeps growing.
Basic thing is that first of all Poettering to it a toehold at Fedora (he works at Red Hat, so surprise surprise). Then he offered Gnome the code for supporting logind on a silver platter.
End result is that to offer Gnome as the desktop you either have to patch it, shim it (ask Canonical how well that worked out) or adopt systemd.
Effectively what is happening is that Fedora is becoming the de-facto standard for Linux. In part because it is effectively the testing distro for Red Hat Enterprise Linux, and thus a large portion of userland devs (and a number of kernel devs) use it as their "dogfood".
I don't know the author and he does not provide context, so I don't really get what the fuss is about. Mentions haters, personal threats of violence but this looks like the usual: a bug in systemd due to an incorrect assumption by the developers, proposed solution: decide that there is no bug and systemd is correct, the use case is wrong.
I experienced one first hand, a definition in fstab for a removable drive suddenly (after a systemd update and no warning) caused the computer to fail at boot time because the removable drive was not present. The fun part is that the emergency shell had a bug too that prevented itself from starting and looped to infinity instead.
It was nice to have to drive 6h each way to deal with this systemd new "feature".
There are no words. How can someone so obviously incompetent be in charge of such an important open source project? Unless he's hired by some 3-letter agency to leave as many security holes in systemd as possible.
60 comments
[ 2.3 ms ] story [ 154 ms ] threadFor example, on some shared clusters I used there was a capability to request a service run as your user. This was before systemd at the time so it would end up generating an /etc/init.d entry, but I imagine a modern equivalent might generate a unit file.
So this article downplays the vulnerability a bit by claiming you have to trick a sysadmin when you could actually just use an automated service management system.
Cute...
That said, refusing to run the unit would be a better default behavior.
Gobolinux as a joke used to ship with user 0 named gobo rather than root, but i think they stopped doing so because it broke certain scripts etc that assumed root and 0 was equivalent.
I don't even want to know how their code is written so that it allows such error to happen and remain undetected.
> There's a new systemd bug that gets the haters aroused!
Oiy... I feel like I am walking into the middle of a Vim/Emacs flame war. Very unfortunate.
[0] https://www.freedesktop.org/software/systemd/man/systemd.uni...
[EDIT] As noted downstream, the parsing error causes the 'User=' directive to just be ignored, so a user unit file will be run by the user. Still too many vectors of attack, sadly.
Why should it? The bug is not that it runs as root but that it encounters a parsing error and uses the default. The default for root is root and the default for user A is user A.
Thank you for the correct, if condescending, answer. It was unclear from the original bug, or this article, if it was falling back on a hardcoded 0 or the user used when the directive is not present.
Since systemd's own behavior is inconsistent, I can't depend on that behavior to derive the reasonable answer. That inconsistency stems from the fact that a "valid" user name that doesn't exist triggers different behavior than an "invalid" user name; and the definition of "valid" is different between systemd and the rest of Linux.
They assume that the program is running as a user with limited privileges. Then, an exploit gets found in that program, and now instead of an attacker gaining user-level access, they now have root-level access.
By far the most worrying part of the issue for me was that the developers said "oh it's not a bug" and just closed the issue, without properly thinking through the implications of the behaviour (such as the other attack vectors mentioned here). That's gross negligence.
Are you saying everyone you disagree with is "gamer gate"?
And gamersgate lost media attention, my impression that a sizeable chunk of that mob has latched onto systemd as their target.
The problem is that the people being harassed can't separate the harassers from the people voicing actual complaints, lumping them all into the trolls/haters bin.
And frankly gamersgate was just the "peak" of a sorts, the root of it all lay further back. It is basically a mutated, nastier, form of the antics done under the "anonymous" moniker.
I am not a gamer, I have never trolled. I am a sysadmin in an 11,000+ person organisation and I have qualms with systemd.
And frankly speaking, proponents of systemd usually fobbed us off by saying that we're dinosaurs. Or intentionally misrepresenting our arguments.
Now we're being fobbed off as being some weird gaming industry crap? No.
I think there are legitimate complaints against systemd.
but at the same time i also think there is harassment going on.
From our side of the fence it is easy to tell the two apart.
But from the systemd developer side, and apparently also the systemd proponent side, the two intermix.
This akin to how hooligans join protest marches simply to start fights and riots, knowing it will be blamed on the legitimate protesters.
1) while being adult men in their prime, they spare no time running coordinated trolling campaigns not unlike people from places like 4chan. This is sick; 2) They attempt to outshout and bury valid criticism online and offline, responding with ad hominem attacks to very correct, purely technical articles; 3) Purposefully derailing conversations pointing to all aforementioned; 4) Aggressive advocacy - leveraging their their network of influence to sabotage 3rd party software by intentionally breaking compatibility.
I have no different words other than "waging war" for what SystemD and RedHat do to Linux development community
To be clear, I'm not objecting to potshots taken at either; /g/ is trash, and whining about vidya gaem journalism is childish and stupid - and lashing out viciously at people who work in the field is worse by far than that. I'm just saying it's worth checking your target first.
"Should this be fixed? Yes, it's an obvious bug"
The author of systemd says:
"So, yeah, I don't think there's anything to fix in systemd here. I understand this is annoying, but still: the username is clearly not valid."
And that's basically the problem. Not that the bug exists. But the systemd author doesn't recognize it as such, and refuses to fix it. This seems to be a recurring theme with systemd.
"Usernames must start with a lower case letter or an underscore, followed by lower case letters, digits, underscores, or dashes. They can end with a dollar sign. In regular expression terms: [a-z_][a-z0-9_-]*[$]?"
(Not much point in continuing this, since the post has dropped from front page to 7th in half an hour, and will never be seen again.)
* http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...
* http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...
Valid portable user names can have digits and letters intermixed. They are simply restricted to not starting with a minus.
* https://github.com/systemd/systemd/issues/6237 (https://news.ycombinator.com/item?id=14681377)
In the actual bug report, as opposed to Mattias Geniar's article headlined here, this is already mentioned.
What happens when admin typos a username in this fashion? The system they think is conforming to least privilege suddenly is granting most privilege. Thanks systemd!
It's like the code is as self-righteous as its author.
You're advocating for bullying people just because they were involved in writing a piece of software.
Seek help.
The legitimate way would be to take part in the development and convince based on technical reasons. If people disagree with you, consider that you might be wrong. If you think you're right anyway, fork it, and see if /other/ people agree with you. Chances are they won't.
The people actually maintaining all the distros didn't all switch to systemd within a few years for no reason.
They switched out of prospect having unmaintainable initscripts and having DEs being turned useless. RH/SystemD liaisoning with distro maintainers at around 2013 can be said to be trollish blackmail.
Again, the point here is how apaling is the conduct of RH/SystemD core developers. SystemD could've been treated as a simple piece of bad software, if not for the group of people pushing it having such toxic personalities.
You do not confront trolling with bugreports and patches.
I agree, but I do also wish Lennart understood that too.
If you mangle the dependency chain in any way, you are likely to find yourself looking at the emergency console.
UNIX is all about small, composable tools. Do one job and do it well.
One of these tools didn't stand up to the test of time (SysV init) and needed to be replaced.
We could have chosen one of the mature candidates (upstart, runit, minit) or even built a great new one from scratch. Life would have been good.
But instead we ended up with this trainwreck of a monolith that not only replaced init, but at the same time tried to reinvent almost every critical userspace daemon from Syslog to NTP.
It predictably failed on every metric.
Now it is time for the major Linux distributions to conclude this experiment, roll back the Systemd mistake and return to the modularity that made UNIX successful.
We get to choose between Emacs/Vim, Perl/Python, Syslog-ng/RSyslog, ntpd/openntpd and so on.
This has resulted in multiple mature implementations for every core system task.
Yet instead of sticking with this code that has stood the test of time over decades, we threw it all out for a poorly written jack of all trades that gives root to usernames starting with a digit...
[1] https://en.wikipedia.org/wiki/Unix_philosophy#Origin
systemd as a package is more than just init.
It is init, inetd, session manager (logind), udev, dns client, dhcp client, cron, su, and the list keeps growing.
Basic thing is that first of all Poettering to it a toehold at Fedora (he works at Red Hat, so surprise surprise). Then he offered Gnome the code for supporting logind on a silver platter.
End result is that to offer Gnome as the desktop you either have to patch it, shim it (ask Canonical how well that worked out) or adopt systemd.
Effectively what is happening is that Fedora is becoming the de-facto standard for Linux. In part because it is effectively the testing distro for Red Hat Enterprise Linux, and thus a large portion of userland devs (and a number of kernel devs) use it as their "dogfood".
I experienced one first hand, a definition in fstab for a removable drive suddenly (after a systemd update and no warning) caused the computer to fail at boot time because the removable drive was not present. The fun part is that the emergency shell had a bug too that prevented itself from starting and looped to infinity instead.
It was nice to have to drive 6h each way to deal with this systemd new "feature".
There are no words. How can someone so obviously incompetent be in charge of such an important open source project? Unless he's hired by some 3-letter agency to leave as many security holes in systemd as possible.