GNU is more than a shitty POSIX userland. Most of the interesting stuff is happening outside coreutils: Emacs, gcc, Guile, Guix, GIMP. The next phase of the project is to bootstrap a trustworthy free software OS from as small a binary kernel as possible, which is what GNU Mes is all about.
>small a binary kernel as possible, which is what GNU Mes is all about
Hurd or seL4...but obviously no one is interested in such an Kernel for servers, mobile or the desktop.
From MY perspective, the OSS OS community was in a much healthier state in the ~05's then it is now (Linux, BSD, Solaris, Minix etc) where all maintained and gave plenty of selection of ~OSS operating-systems, now it's just that Linux.
Not as a Server/Desktop OS. Both are not intended as a generic OS, but rather specialized solutions..just like for example QNX. Don't get me wrong, i would love being wrong about that.
I wonder if ell see a three layer run time going forward. Bottom layer is about hardware partitioning and Ressource sharing.
Middle layer is networking, file systems, ACL, monitoring, graphics and other acceleration in a rich os. Or monolithic deterministic real time blobs. Or small dedicated OS partitions for, security or similar hardware jailed applications.
Top layer is containers/jails inside the rich OSs.
Then fuchsia is a candidate for the bottom layer and for the security OSs and maybe for specialized OCI payloads.
Not even Hurd or seL4, but like... some binary runtime environment small enough to be audited at the machine code level, that could bootstrap something bigger, that could bootstrap a tiny simple C compiler, that could bootstrap GCC, that could then be used to bootstrap the most successful GNU kernel, Linux-libre.
While I generally agree with the author that the userland monoculture isn't helpful for portability, that argument belies the fact that most people I've worked with _want_ those extra features that GNU gives them. I started using Linux and IRIX around the same time - but I actually didn't really "understand" the difference, because someone had gone out of their way to install the GNU userland.
That's something the article leaves out: while GNU and Linux are many times seen as a unit, there's a rich history of installing the GNU tools on other Unixes, because of the extra functionality they offer.
There is, of course, the argument against portability. At my last job, as the only Linux user on my team, I more than once wrote shell scripts that only worked for me - as soon as someone on macOS ran them, they'd run into a missing feature on `sed`, or a different calling convention on `date`. Which is why, at my current job, I'm very begrudgingly a macOS user - it makes me a better coworker.
This has always been a problem for me as well, but we’ve pretty much solved the issue at my current job by using nix to manage the development environment. With nix-shell and direnv, we can know that anyone working on the project is using the same versions of coreutils, sed, grep, make, etc. The learning curve on nix is a little steep at first, but it’s been worth it: we’ve got devs running five distributions of Linux, plus a couple on MacOS.
One of the first things we did on Solaris installs was install the GNU userland (it's been so many decades, I forget the specifics - packaged somehow, not ./configure; make) so we could get GNU sed and awk (and bash and vim) amongst all the other things. My anecdotal experience matches yours, we wanted those GNU features on our Solaris experience, they were/are useful extensions.
In some ways FLOSS almost encourages a mentality akin to the old Microsoft "embrace, extend, extinguish" minus the actual extinguishment. The best way to get people to use a free Unix, besides making it free, is to make it /better/ than the competition while also compatible.
Besides the lack of portability to other Unix systems isn't a real problem. The operating system itself is portable! /s
The whole "GNU/Linux" story always struck me as incredibly arrogant and ridiculous. A typical modern Linux system has more lines of web browser code than lines of GNU code. Should we start calling them "Chromium/Linux"?
If you follow the FSF's actions closely, you will find they're incredibly insecure about people switching away from their software to alternatives. Typically the alternatives are not GPLed but rather under a permissive BSD-like license, and they latch onto this to deride any new competitors [1]. But those competitors don't typically exist because people hate the GPL; they exist because -surprise- GNU software isn't the be-all end-all, and it's a fairly common pattern for GNU maintainers to be reluctant to change at best, or actively hostile to third parties at worst (anyone remember glibc's Ulrich Drepper?).
Then there's how Stallman vetoed GCC having a useful AST output mode (a requirement to build smart IDEs and other development tools - yes, including such features in emacs) because he was scared of third party proprietary extensions. That's one reason why clang took off - its extensibility and flexibility, which the FSF was always against GCC having. The FSF (and particularly Stallman) hates clang, again reaching for licensing and moral arguments, because they just can't accept that some people may have written technically superior software to theirs, and done so with a more permissive license. [2]
In the end, it's hard to see the FSF's response to these things are anything but controlling and attempting to stifle competition. And we absolutely need competition for a healthy free software ecosystem. There is no value in trying to represent GNU as some kind of indispensable component of a Linux-based OS - they aren't, and pretending they are hurts us.
Well, if chrome is the primary set of userspace tools, "chromebook" might be a reasonable name.
I'm not sure how to differentiate android/linux (or even Chrome/linux) from UNIX-philosophy/linux, whether UNIX-philosophy ends up being busybox or GNU or something else.
Honestly, where do you even draw the line? Android has a shell, it has userland tools (toybox), it has a service manager (not unlike systemd), it has a compositor (not unlike Wayland) and an audio server (not unlike PulseAudio). Sure, we can look at it and say "this somehow doesn't look like a Linux desktop", but that's mostly based on lineage. It's the ship of Theseus story - if Linux desktops today are so different from those of old that we've basically replaced all the pieces, how are they different from an OS that made more of a clean break?
Your comment makes a very good point. Android really does seem like a Linux system when described in this way. I may need to re-evaluate my attitudes here.
Except most of that stuff is only visible via Java APIs, NDK userspace is hands-off to such tools
A fact that termux people refuse to accept, hence why it doesn't run on the latest versions, unless they now finally accepted that POSIX isn't part of the official NDK stable APIs and have started calling into JNI.
Plenty of Linux software won't be compilable just with these APIs, unless they are game based,
That's an upper level platform policy question, which is rather difficult to compare to because... Linux doesn't really have anything like that. Try running a dynamically linked executable with more than the basic dependencies on a Linux distro 2 years newer and see how many missing dependency libraries you run into due to ABI changes and soname bumps...
FWIW, Termux works perfectly fine on the latest version of Android - what they can't do is push updates through the Play Store, because the minimum required API level now imposes security restrictions (no running executables from your data directory) that break it. But that doesn't make Android not a Linux system; you can still adb shell into it and run commands and shell scripts and copy new binaries in and run them.
This is not unlike Linux container systems, which also break various apps. It comes with the territory of containerizing applications that you have to break insecure legacy APIs; this is not unique to Android.
Most of the people that dislike GNU base their dislike on political reasons (because they hate the GPL or Stallman or whatever) and then try to embarrassingly justify their hate by shoehorning technical jargon.
Most of the people that support the FSF base their support on vague poorly-grounded ideological reasons (because they drank the Stallman kool-aid or whatever) and then try to embarrassingly justify the missteps of their idols by deferring to said vague ideology to deflect from the elephant in the room.
I have a whole Twitter thread on how the modern FSF's policies actually hurt user freedom while being completely inconsistent with their own ideals, if you'd perhaps consider opening up your mind to criticism of them: https://twitter.com/marcan42/status/1377899929209774081
The FSF is a political organization. Supporting it for political reasons is wholly appropriate. Disparaging the people who do so with put-downs and insults instead of engaging in reasoned debate is not.
The FSF has long been shifting from being a political organization to being a religious organization. Its policies are no longer consistent with their own views of user freedom. Read the thread for more info. They are literally promoting things that hurt user freedoms - things like promoting ignorance of security flaws, promoting hiding proprietary firmware (because it's only a problem if it looks like a blob), promoting actively crippling hardware (thus reducing its usefulness with present and future free software), and more. Those who blindly support them without taking a deep look into whether their actual policies are consistent and rational and further their goals are no different from religious fanatics who stand by ancient scripture while ignoring thousands of years of progress.
I support user freedom. I support users having control over their own devices, knowledge of what software runs on them, and knowledge of what choices are available so they can make an informed decision. I spend most of my time working on reverse engineering and porting Linux to proprietary hardware (Apple M1) so users have the freedom to run whatever OS they want. I cannot support the FSF, an organization which supports censoring security vulnerability warnings when fixing them involves updating proprietary firmware (that already exists anyway); that supports burying proprietary firmware in non-introspectable read-only memory because then they can "pretend it's hardware", then selling the result as "fully libre" (even though the same device with the same firmware visible and accessible to the user would be much more free and transparent, allowing introspection, auditing, and replacement with a free alternative); that supports physically destroying hardware components that presently require a proprietary driver to function, ignoring the possibility that free software drivers might appear in the future. These are real FSF policies and actions. This is what you are supporting when you support the FSF. It's not user freedom any more. It's religious nonsense like "blobs in /lib/firmware are bad, we can't have any blobs be out in the clear".
> that supports burying proprietary firmware in non-introspectable read-only memory because then they can "pretend it's hardware", then selling the result as "fully libre" (even though the same device with the same firmware visible and accessible to the user would be much more free and transparent, allowing introspection, auditing, and replacement with a free alternative);
Totally agree, they have been completely swallowed by their own ideology. This shows that their actions are contradicting the same things FSF says publicly: that reverse-engineering is important.
"The most notable thing about this project is it allows proprietarization while GNU coreutils doesnt"
That's a GitHub issue title filed by an FSF director on a project whose goal is to rewrite the basic POSIX core utilities in Rust, for security and maintainability. Because they MIT licensed them. They are literally telling a new competing project that they are worthless and an attack on the FSF due to their license choice.
The author uses "we" a lot but who are this "We"? For sure there is a large number of Linux users that are not part of his "we". I consider myself someone that is into the technical stuff but at the same time I care about the philosophy and meaning of "Free Software".
> OK, now have a look at the 120 lines long source of GNU yes. And compare now that behemoth against the implementation in OpenBSD, the one from sbase, or the one from busybox. Like… why does GNU yes does so many memory operations for a task that is implemented so easily by everyone else?
But from the style of the author, it is clear that it is the case of a knowledgeable well informed educated person. GNU clearly has its problems, but badmouthing it like that is hard to differ from malice.
Does GNU yes need such great performance, though? I'm curious to hear where it matters. Why would we need 10 billion yeses a second, and why would 10 million yeses a second not be good enough?
GNU yes is elaborate and fast because Unix's yes is conceptually simple[0]: the HN link the GP shared cover how GNU programs were made to deliberately differ from Unix versions to avoid copyright issues.
I'm curious as well. On the other hand, does it need... whatever it is that GNU yes's decriers are after? The choice to focus on /usr/bin/yes as a case study is at least as bad as the GNU advocates' retort ("GNU yes is 10GiB/s fast").
When I ask for a very simple command to do something, I expect it to be the best possible implementation of that command. `grep` is pretty close to absolutely optimum (ripgrep can beat it, sometimes, maybe).
I don't see why the standard for composable utilities should allow for inefficiency.
Wrong thread? How is this a response to the previous comment? Arguing that you want /usr/bin/yes to "be the best possible implementation" in terms of efficiency is a weird way of being in "violent agreement" (more like "aggressive agreement", but whatever). If writing a defensive response, you should actually defend the thing on the other side. Namely, the response here should defend the position that it's more important for yes to be simple than it needs to be efficient. That's what the previous comment challenges potential respondents to come up with.
You don't take a position. That's my point. What is wrong with yes being capable of doing 10Gib/S? Nothing. It's optimum for a utility which has that sole function.
It's not a bad retort, it's the end of the story: there is a superior utility, it's implemented.
You're either still misreading and believing that you're arguing against a position that is at odds with yours, or you know it's not at odds but have weird ideas about how argument works.
My position is: either articulate an argument for why it's bad for /usr/bin/yes to be implemented the way GNU yes is implemented or drop the argument entirely.
Your comments manage to be in disagreement with those on the other side of this, and yet still adopt an argumentative/defensive tone with respect to what I've written. I consider this to be an even bigger waste of time than the people (like OP) who are upset about GNU yes's implementation.
I can defend my position—which is and always has been from the beginning of this (annoying) argument that anyone who says that GNU yes's implementation is bad—or that "10GiB/s" is pointless feature for something like /usr/bin/yes—needs to be able to defend the decision to scrutinize "something like /usr/bin/yes" in the first place. For whatever dumb reason, you think it's a good idea to argue about this, even while apparently believing yourself that the way GNU yes is implemented is a good thing. This makes no sense.
(Are you under the impression that "X is at least as bad as Y" is the same thing as saying "Y is bad"—and that that means I need to defend the position that Y is bad? It doesn't. Maybe learn to read English better, especially before leaping so confidently into trying to argue with someone over what you're reading. Another bonus tip: not only is it important to read in complete sentences, but it's important to write in complete sentences, too, and to make sure that the sentences follow logically one after the other.)
I don't know. And actually I don't care. A program that fits in a single file with 120 lines of readable code is a price I'm willing to pay for close to best possible performance.
But that is not why I commented. I commented because I wanted to illustrate that the author questioned something that can be easily answered with a simple google search. I'm sure the author is qualified enough to answer the question, but instead left in the post giving most readers a impression that the complexity is there out of incompetence of GNU developers.
EDIT: I should not assume you asked in bad faith, but as the sibling commented. The answer to your question can be easily found in the link I posted. Making your behavior strangely similar to the author of the post: making questions that can be easily answered while giving a bad impression of others. Please, avoid that. It may work in many places but will fire back at you sometimes.
I don't find it so questionable to ask questions that the right google search (out of an infinite number; we're not all equally deeply immmersed in this stuff) can answer. But if we disagree about that I guess I'll stop talking to you. The actual question we're debating will have to wait. I've had to deal with enough ornery people in my life who had the answer like, right in their head but would still insist on putting others through pain for it. These days I try to foster a workplace where anybody struggling with a question the person next to them knows the answer to is wasting the team's time.
(Skills are not the same as factual knowledge here.)
Sorry for giving the wrong impression. My problem with the question is not that it is easy to answer. The problem is that it is easy to answer WHILE giving a bad impression of others.
I don't understand what you mean. Why can't we just debate the question without speculating about motivations of the people raising/answering the question?
I think I misread and misjudged you. I'm sincerely sorry for that.
Maybe it is a sensible subject for me or I've been living with people asking rhetorical redundant questions as a passive agressive form for disqualifying others.
I read your question as: "I can't see any reason this performance is needed, maybe it is really incompetence or pedantism from GNU developers." But I think it is prejudice on my part.
Thank you! And yes, I admit you were partly right. Questions often have some component of rhetorical quality. It's not always yes/no on whether it's a rhetorical question or not. But either way, I find (based on experience teaching) that the best way to answer a question is just by treating it as genuine. Past a point if I decide someone's acting in bad faith I just disengage.
Anyways, back to the original question, I do believe that a lot of programmers (I don't buy OP's separation between GNU and non-GNU) over-engineer things without regard to the costs created by over-engineered code. My default assumption is that that's happening in yes. If it's happening in such a small program where it's so obvious, the same tendencies are likely to be at work in more complex programs like cat and awk and cc. Happy to debate that :)
Thanks! Glad you didn't think I was in bad faith after the semi-rhetorical question and disengage.
That whole thread makes me sad. Another, more subtle way IP is ruining the world. But that's neither here nor there on this debate.
Just to flesh out my current thinking a bit more:
* I'd really like a study to understand how much complexity old GNU tools have gained in the past 2 decades. Because that would not be due to copyright concerns. But I'm not sure I have the necessary skills. (I'd like this study for non-GNU as well, but I assume copyright is not a concern there so the study doesn't have to set crisp time bounds.)
* I share some of Drew DeVault's frustration in https://drewdevault.com/2020/09/25/A-story-of-two-libcs.html. I've viscerally felt the immediate sense of disempowerment he describes when cracking open an open source package. _However_ I vehemently oppose his cavalier lack of concern for locales. Internationalization is important. ASCII is just as effective at disempowering others.
> Thanks! Glad you didn't think I was in bad faith after the semi-rhetorical question and disengage.
You are welcome! Considering the karma I got from that answer, I don't think I'm the only one who misread you. If you want a piece of advice: starting a question like that with "Genuinely curious: ..." is my strategy to prevent this kind of misunderstanding.
> * I share some of Drew DeVault's frustration in https://drewdevault.com/2020/09/25/A-story-of-two-libcs.html. I've viscerally felt the immediate sense of disempowerment he describes when cracking open an open source package. _However_ I vehemently oppose his cavalier lack of concern for locales. Internationalization is important. ASCII is just as effective at disempowering others.
As Drew DeVault's says, it was his code which was broken. I know glic code is complex, but it is battle tested, mature and highly robust. I'd bet there are good reasons for that complexity. If I was really curious as to why, I'd try asking devs directly to avoid making them look bad by making public statements about it.
Now, I looked in scdoc code. The solution was to cast the parameters to "unsigned char". Considering the code
uint32_t ch;
isalnum(ch);
compiles without warning (even with -Wall -Wextra) on my machine, I'd consider this a potential vulnerability for programs using this function. I can't see why glibc developers don't turn isalnum into a macro that automatically casts its parameter (when its address is not taken). I'll consider asking them this weekend.
Regarding "genuinely curious": I've seen some real passive-aggressive comments that say that :)
Regarding "asking devs directly": I often find it challenging even to figure out where to ask. Going from a program to its package, dealing with interactions between packages, etc. All of which goes back to simplicity as a core goal. "Good reasons for complexity" isn't sufficient, IMO. Complexity requires having a place to explain the reasons, and making it easy for people to find the place. Considering that we're all volunteers when it comes to open source and this sort of documentation is difficult, I'd often rather err on the side of not having a feature rather than having a more complex feature.
On the other hand, there are fields like internationalization or power management on laptops that are irreducibly complex. Not sure where I'm going with all this. Open source today serves businesses more than people, IMO. But it also serves people indirectly through businesses. This is a difficult topic. Your comment made it seem like there's an easy thing Drew or I could do. I'm not sure that's the case.
> _However_ I vehemently oppose his cavalier lack of concern for locales. Internationalization is important. ASCII is just as effective at disempowering others.
I also think internationalization is important, and that ASCII is mostly good at disempowering others. But at the same time I hate the way C locales work. They're a ridiculously terrible way of dealing with the problem, and an endless source of bugs.
And for that matter, what should `isalnum(六)` return? (in case HN mangles that, I put the Japanese "Roku" kanji there, i.e. the numeral 5). I'd say it should return `true` no matter what locale you're set to, since it's a numeral. But glibc returns `false` unless your locale is set to Japanese when `isalnum` is called, and musl returns `false` always. Yet they inconsistently return `true` for `isalnum(5)`, no matter the locale.
The selling point of various GNU userland tools and libraries in the 80's was better performance, removal of obscure bugs and limitations and then maybe some useful extensions.
Case in point is that most people have no idea of how POSIX defines "text file" (ie. ends with \n, no line is longer than LINE_MAX), and what happens when you inout something that technically is not text file into POSIX utility that expects text file is undefined. GNU tools do something sensible or at least produce an meaningful error message, traditional Unix userspace usually does not (lines get truncated, last line sillently ignored, some buffer somewhere overflows and in better case you get SIGSEGV...).
Premature optimization is mostly an issue if I want to build a big machine and spent the first weeks on optimize and polish a tiny, not too relevant cog.
But `yes` isn't such a thing, it's a standalone tool. Here the UNIX philosophy "Write programs that do one thing and do it well" comes rather at play.
Further, the optimizations it makes are not really complicated nor expensive to maintain. A page aligned buffer twice the most common page size is standard for basically everything where something is written or memory moves around.
One certainly can argue that it is Ok doing it more simply as it still is more than fast enough for the common use case.
But arguing that a bread and butter optimization, namely one that adds just a few dozen lines of code total while increase throughput by three orders of magnitudes, should not be made, or that it even points to a complexity problem, is not holding any ground IMO.
> I want cultish followings to end within the FOSS community. I swear I’ll always work against that.
> And we like things following the standards already put in place. Because we don’t give a damn about helping the FSF do their thing: We care about tech. We care about and love the legacy of UNIX, its philosophy, and we want to adhere to it as strictly as possible.
Sorry for breaking it up to you, but this kind of "standards must be respected 11!!!!1&!1" mindset is the actual cult-like behaviour, commonly found in communities such as cat-v.org, and with people clinging to plan 9 like it's anything beyond a failed experiment.
GNU and shells like bash (I use zsh which is I guess even "worse" in terms of feature bloat) are dominant because they provides a ton of super useful features to make work faster. Likewise you bash against glibc's bloat, but most software perform faster when using glibc than other "less bloated" libc implementations like musl ; the bloat is here for a reason and that reason is performance (which matters infinitely more than whatever ideological purity BS someone can find). Who cares if it's not strictly POSIX compliant: it makes computing a better experience and that's all that matters, and if tomorrow someone manages to create an entirely different non-posix platform that manages to improve on that, I'll happily jump there and I hope you do too.
I think the difference is with bash & zsh is down to whether you choose to use them as your interactive shell or whether you choose to write shell scripts in them.
I had assumed that this was about shell script portability. Bash and zsh have a ton of super useful features to make work faster, but the features for shell programming (writing shell scripts that other people will use) are not so useful. Yes, I'm aware of stuff like [[, I don't think that [[ is very useful. I have dash as /bin/sh and #!/bin/sh in all my scripts, and seriously, it's not a problem.
There're many features that are useful for programming: variable expansion, pattern replacement, substing expansion, and others. Not only that but some bash built-ins essentially replace external processes leading to your script being faster and more portable (ironically).
Bash is not alone in having built-ins that make it faster. Generally speaking, Bash is measurably slower than other shells. If you convert make your shell scripts portable and change #!/bin/bash to #!/bin/sh, in my experience, this usually makes them faster, too (on systems where sh is not bash). This is the driving factor for Debian's switch from Bash to Dash, which resulted in noticeable startup speed improvements:
(Note that Systemd has since obviated most of that work.)
You may be writing very different shell scripts than the ones which I encounter. I'm not sure what you mean by "variable expansion", because variable expansion is portable. Something like 98%+ of all pattern replacement I see is something simple like replacing a suffix or prefix of a string. For example,
infile=input.png
outfile="${infile%.png}.jpg"
You can typically use sed for the complicated stuff.
Above a certain level of complexity, I'd likely rewrite the script in Perl or Python, anyway. Or possibly Go.
Yes, supposing it exists and you've permissions for it, you can use sed. Do that a dozen times and the calls to an external process add up to a significant time. But the real advantage is being easier to write and work with.
I could equally ask, "What level of Bash features do you deem necessary for what you do? I rarely encounter a shell script that cannot be trivially modified to work on POSIX /bin/sh."
At least historically speaking, Bash was noticeably slower than Dash. It's the slowest shell. If you are doing things like, say, build scripts in a CI environment where you need to execute shell scripts thousands of times or more, the 4x difference can make a real impact on user experience. Same thing during startup, back before the transition to Systemd, when Debian changed the /bin/sh alias to point to Dash instead of Bash to speed up system startup.
At some point, shell scripts reach a level of complexity where most people choose to rewrite them in a higher-level language, often Python or Perl, or maybe even Go. My personal experience is that the Bash-isms, although kind of nice, don't really shift that threshold... like, although Bash has arrays, but if I need to manipulate arrays, I switch to Perl or Python. If I just need to pull elements out of one array, I can do that portably.
It seems like if one want to be a bit more focused on compatibility, just using python with compatibility added for v2 vs v3 is easier than using shell scripts for the multiple different operating system, platforms, shells and versions that exist today. Shells seems much more coupled with the operating system and decision that people have made on that level.
Yes, above a certain complexity, you'd probably want to write your script in Perl or Python.
Python has an enormous overhead compared to shell scripts, though. If you're writing some simple wrapper in a build system that's going to be invoked thousands of times for a single build, Python could easily end up being a choke point.
If you're really concerned about portability, I would probably write it in Perl :-)
Perl is basically a souped up shell language, with built-in Awk & Sed, faster start-up time than Bash, and it's installed nearly everywhere.
Perl vs Python on the script side has always seemed to me to be a bit up to taste, through I have noticed on the sysadmin side that in general things has moved away from Perl towards Python. I also notice that in terms of job listing, there seems to be a lack of new perl programmers in order to support older systems.
In terms of build systems, what I have most experience with would be with Debian and their build system, which pulls in the required dependencies. Compatibility in this space is less about zsh vs bash and more in line with architecture difference like RISC vs AMD64.
> Who cares if it's not strictly POSIX compliant: it makes computing a better experience and that's all that matters
Whilst I broadly agree with your point, I'd suggest that "better experience" can be extremely "eye of the beholder" - especially if it interrupts a long established workflow / toolkit.
For many, "long established workflow" now means "Linux with GNU userland". But if deviations from POSIX behavior by the GNU tools cause problems for a flow, that's what the POSIXLY_CORRECT environment variable is for.
> Sorry for breaking it up to you, but this kind of "standards must be respected 11!!!!1&!1" mindset is the actual cult-like behaviour, commonly found in communities such as cat-v.org, and with people clinging to plan 9 like it's anything beyond a failed experiment.
That's a bit of a non sequitur: Plan 9 is very much not a standard, and indeed it derives a great deal of its power from not implementing the POSIX standard.
Nothing's worse than the cult of POSIX. Half a century ago some people gathered and figured out the lowest common denominator of all the unixes of the time. Now it's 2021 and we're still supposed to restrict ourselves to this "standard" system in the name of portability. POSIX has so few features that restricting yourself to it is masochism. Every unix adds their own extensions, they would be almost unusable if they didn't. So why is GNU criticized for doing the same?
The author says GNU is just some optional userland. I find that amusing since in every traditional unix the POSIX userland is deeply integrated with the system. He talks about glibc but the BSDs and even Windows ship with their own libc that I can't ever get rid of because they're the only supported way to interface with the kernel.
Linux is the only operating system that actually frees us from this cult. Unlike other systems, the kernel/userspace interface is stable and defined at the lowest possible level: processor instruction set architecture.
Only on Linux is GNU truly just some optional userland. You can throw it in the trash if you want. You can make your own. Who says you gotta have little commands like cp, mv, grep, sed, whatever? Who says people have to use some "standard" shell from the 80s? You can make a graphical userland if you want. A 100% Rust or Lisp userland. Even no userland at all.
People complain about systemd but I actually have a lot of respect for it. The developers had the balls to trash all this sacred POSIX stuff and make something new that actually uses Linux kernel features. The resulting system is better for it.
increasing my donation to FSF. we need more education on why free software and the GPL matters. Everyday is an attack on the computing freedoms that people take for granted.
> Everyday is an attack on the computing freedoms that people take for granted.
Just to be clear, Ariadna Vigo is not attacking computing freedoms; pretty much the opposite. I agree with her position on nearly everything except for her stance about the GNU operating system. This is a technical detail in the grand scheme of things. Just like tabs vs. spaces, for which it is fun to disagree in an over the top way.
We must support the FSF for pragmatic reasons, but we can still acknowledge that the "pure unix" philosophy is a great and beautiful idea. Also, cat-v and suckless are fun websites full of interesting stuff.
> And we like things following the standards already put in place. Because we don’t give a damn about helping the FSF do their thing: We care about tech. We care about and love the legacy of UNIX, its philosophy, and we want to adhere to it as strictly as possible.
Who is this 'we'? I don't care about the UNIX philosophy (I'm an emacs user). First thing I do on bsd systems is install the GNU userland. BSD tools follow the POSIX spec like a cult refusing to add useful extensions. Look how hopelessly broken posix sh is. I work quite often with busybox on small embedded systems and the limitedness of its vi implementation makes me irrationally angry.
Be wary of posts and articles which talk about an universal "we" as if everyone thought the same.
I, for example, care more about the philosophical aims of the FSF and RMS than about the finer technical points. That Linux is a very useful OS for me is a bonus. So "we" do not just care about tech; "we" also care about users rights and freedom.
> Why are we vendor locked into GNU? Why are we still pretending that’s good or not as evil as the vendor lock-in measures taken by Google, MS, or Apple? Landmining your userland and C Standard Library with extensions is exactly what MS does. When MS does it is bad, so why is it acceptable or tolerated when GNU does it?
Although you can pay the FSF for copies is (I think) possible calling GNU a vendor similar to "MS or Apple" is too much of a stretch. When "MS or Apple" contributes back agreeing to the license, out of goodwill or because it helps the development, it is a nice behavior in their part; otherwise they are just being themselves.
As for why GNU is a different case with regards to extensions... I can think of a few reasons:
- The extensions are useful for a reasonable part of the users.
- GNU is free software, so you can easily use and benefit from the extensions.
- GNU is portable and its license allows even proprietary systems to include it without any problem.
- Some of these extensions can be disabled.
- Some of these extensions do not break portability if not used.
- Some of these extensions helped to shape and evolve what people expect from a UNIX-like system. A few GCC extensions influenced the ISO-C standard.
Ok, portability may suffer, but that is the price some groups choose to pay. Considering the gained benefits, I'd say it is a fair price.
Works on Android (Toybox userland) and macOS (BSD userland, but you might need to prefix LC_ALL=C to get `tr` to not hate the binary input) at least. And it's more flexible, as you can customize the charset and password length arbitrarily, which you can't with base64.
True, but it's also not GNU anything. Most modern UNIX-like kernels implement some form of /dev/urandom or /dev/random. The actual shell tools used are POSIX.
I don't think POSIX even defines any functions/system calls for getting cryptographically secure random numbers, so if you're literally restricting yourself to POSIX constructs, then indeed what OP asks for is impossible :)
I always wanted the /dev/zero driver to use the device minor number to define the number that got used instead of zero, so you could mknod /dev/seven to make an infinite source of beeps.
Why don't you submit a patch to the kernel so that this beautiful idea becomes reality? It is free software: here you can easily be the change you want to see in the world.
You may need to be careful because the major number is shared with /dev/null, but I doubt these numbers are hardcoded in many places.
They are definitely hardcoded in many places (e.g. embedded systems, early bring-up - udev style stuff is used for most things these days, but the core /dev/zero, /dev/null, /dev/console stuff is hardcoded everywhere). This would be a kernel-userspace ABI break, which is a big no-no.
You could add a new major that does this (keeping the old numbers for compatibility), but I think you'd have a hard time convincing people that this is a worthwhile enough idea; hard-coded new major numbers are in short supply.
I don't want an only-POSIX-compliant OS or userland, because POSIX is not a good standard. POSIX is "let's standardize the lowest common denominator across popular commercial Unices". Almost everything in the GNU coreutils is there for a decent reason - in many instances because of severe problems with the POSIX spec (just think about all the POSIX tools handling record-shaped data, but not supporting NUL separated records) or because it's just handy to have.
As a Linux user I often get annoyed when I try to use MacOS and some of the commands differ. For example, the find command on Linux, er GNU, doesn't require a path, but MacOS does.
I always blamed MacOS/BSD for just being different, but now I know that it's GNU that is different. I'm not sure what to think about it though--I do like the idea of compatibility with a standard, but I also don't like standards to hold back innovation.
Some of it is GNU being different, but you also have to understand the difference between BSD and AT&T System V UNIX. Many GNU tools are closer to the standard AT&T behavior than BSD. BSD was the original nonstandard userland.
Old UNIX was never really that beholden to standards. In practice, the de facto and sometimes "de jure" standards were just based on whatever was the most common variant of UNIX, which for a while was Solaris (it's oversimplifying, but POSIX is kind of like "standardized Solaris"), but is becoming GNU/Linux.
Commercial unixes "based on" SysVR2/3/4 had plenty of "non-standard" behavior and extensions, it was how you differentiated your product. You support POSIX compatibility to enable portability if you need it, but you differentiate with your extensions or "weird behavior". GNU was no different. The BSDs just have their own tradition, one that's older than POSIX.
POSIX allows implementations to have extensions, and almost all implementations have extensions.
There are good reasons for many of those extensions. Many options implement capabilities that probably should be in the standard, but currently aren't. For example, POSIX doesn't have easy-to-use secure options for handling file names; attackers can insert newlines in filenames, yet POSIX lacks options like find -print0 and xargs -0, so you either spend a great deal of additional time writing complicated code that's likely to be insecure, or you simply use extensions as almost everyone actually does. Even the BSDs and busybox include those examples, by the way. Also, long option names are excellent for making scripts easier to understand, and really should be in the
POSIX standard for systems that are not highly memory constrained.
I try to make my shell scripts portable when that is reasonable, but sometimes that is simply far too difficult. The minimalism of POSIX is an advantage on small systems, or in other constrained environments, but most of us today are not trying trying to write software for a PDP-7.
In most cases the scarcest resource is developer time. In that case having rich functionality and extensions that improve readability is more important. I think the market is being perfectly reasonable, minimalism is less important than developer efficiency in most cases. Your mileage may vary.
> Why are we vendor locked into GNU? Why are we still pretending that’s good or not as evil as the vendor lock-in measures taken by Google, MS, or Apple? Landmining your userland and C Standard Library with extensions is exactly what MS does. When MS does it is bad, so why is it acceptable or tolerated when GNU does it?
Because POSIX (and partiall also ISO C) is just ex-post stadardization of features that appeared first as vendor extensions, and also does not cover many areas entirely. Useful extensions became part of standard, and also many programs just need to use non-standardized parts of API (e.g. some Linux-specific parts).
> I could’ve talked about Android and embedded Linux. I didn’t want to, though. But me not talking about them doesn’t mean they also show that the FSF telling half truths when it comes to what GNU is.
> If you thought I was going to go the “Android route” of argumentation… You know… saying that Android shows that Linux can be used without GNU… Sorry! I’m way more sophisticated than that!
Sophisticated in what sense? That when you search the Android source tree you find FSF code?
Finding a few files containing (BSD-licensed!) code that originated in the FSF doesn't sound like a particularly compelling argument that people have to use their code. Of course people will use their code when it does the job and the license fits, it'd be silly not to.
FWIW, here's a search for FSF-copyright code that excludes build generator output (autotools/automake/libtool/etc) and gcc/bison/etc related stuff (which many other projects which are not themselves FSF-authored use), as well as licenses themselves:
193 hits, and a lot of that is ffi and libbacktrace stuff. Exclude those two projects, and you're down to 47 hits, most of which are random things like headers, test shell scripts, and some translations. I think it's fair to say that Android does not include any significant amount of GNU/FSF code.
(Keep in mind that the Android codebase is massive and vendors a huge amount of third-party projects, so it's actually quite remarkable that there is so little FSF code in here - part of that speaks to how little FSF people contribute to other projects!)
Only tangentially related, but POSIX certification doesn’t require that you don’t have extensions, at least I don’t think so. I could’ve sworn macOS supported long options on its BSD coreutils.
The author didn't really bother researching much the historical points. Someone can read the initial GNU bulletins to see that Stallman was planning a kernel before those tools. It isn't that he chose (as said) to write the userland first but did while waiting for TRIX to be released as free, something that never happened.
Of course when Linux became a viable they decided to abandon their plan and concentrate resources on userland. Kinda opposite to the usual of similar projects being co-developed leading to fragmentation. If there's an already capable kernel, why write another?
Another problematic point is the "look how much lines it takes, such a bloat" for `yes`. And then proceed to call it convoluted for pointless reasons. I guess being[1] about 100x (you read that correctly) faster is a pointless reason if a binary program you use rather develop can be written in fewer lines of code. Those GNU devs really should be more lazy.
Also there's "portability". If you write a program that is utilizing Linux' specific calls then your program isn't portable to other Unix-like systems. Same way that if you write `/usr/bin/env bash` at your script it won't be portable to other shells. You can of course install some POSIX utils implementation and use those, and you'll limiting yourself, but at least you'll have portability to systems you aren't using.
I suppose this is part of the new generation of people getting into software (not saying Ariadna is one of them, but it wouldn't surprise me). She is right that in the Spanish world (and in Latin America in particular) RMS and the FSF are well-respected, with little of the controversy that sparked in the English-speaking world -- thankfully!
The article raises interesting points, but the whole rant against GNU and FSF is weird. GNU (yes, and Linux) and FSF is why many of us are in this business. They played a huge role in the career and passions of many of us -- I feel entitled to say "us" just like the author of TFA carelessly speaks about "we" -- and misunderstanding the history of how we got there doesn't help.
To me it feels as if a new generation of tech minded people have started taking things for granted, "why do we even need all those movements fighting for users' freedom, if I am free now?". It's a bit depressing.
The different approaches and goals of Linus Torvalds and RMS are both valid, and only in hindsight can we say one triumphed and the other didn't, but the article makes it look as if one of the two was obviously wrongheaded.
So some interesting points, but it could do without the whole rant against GNU and the FSF, and without the accusations of cultishness.
Only - we're not free. You have a point about young people not being to appreciate not having freedoms perhaps. But our generation has already dropped the ball. We're non-free when it comes to most hardware, web services, mobile software. This is because people don't really care about their freedoms. They settle for "pragmatic" benefits of open source software. People enjoy freedoms themselves, mind you, but they the problem is that they think that anything beyond that there be conspiracy theorists or something. Along with Stallman's social awkwardness, the culprit is Linus' siding with TiVoization and against GPLv3 in my opinion.
Anyway, It's not over, and knowing how markets tend to monopolies, we'll be fighting forever. Which won't be a problem once numbers are on our side. Right now we're in a stale mate with much of the front being taken by proprietary software. That is not likely to change until the free software community gets a better face, but agenda-pushing codes of conduct won't do that. We need a serious discussion about software freedoms
I'm in the English-speaking world and I wasn't aware of any particular controversy with the FSF other than issues regarding RMS himself and his personal conduct. Is that what you're referring to?
Mostly yes. But also the feeling that FSF is "out of touch" or that RMS is a weirdo is much less prevalent in Latin America, where (in general, the people/organizations I know of at least) view them with more admiration than in the English-speaking world.
There is certainly controversy around RMS, as you say.
But I think more generally, wrt FSF there's not so much controversy [1] as disappointment in the FSF failing to stay relevant. Copyleft was a solution to the 80/90'ies software world. The FSF tried with the AGPL to answer the 'threat' of SaaS, but largely the world has shunned it except as a poison pill license to upsell a proprietary version. Similarly with the GPLv3 the FSF tried to answer to the threat of Tivoization, but again, largely failed to gain adherents in the low level components, like the Linux kernel, where this would have mattered.
[1] Well, the FSF taking RMS back on its board did produce a tsunami of corporate (financial) contributors leaving ship.
The "failure to stay relevant" is an odd comment that I have indeed read many times (mostly from English-speakers; as I said, in Latin America the FSF is admired).
It's odd because it's worded in a way that sounds as if the FSF wasn't necessary now (or maybe even never), instead of thinking that the FSF is needed more than ever, now that there's a war against general purpose computing and against privacy and users' rights, and that if the GPL3 and AGPL didn't work -- which is possible! -- then how do we fix that?
Instead it's either apathy and even scorn against the FSF, which is alarming to me.
> Instead it's either apathy and even scorn against the FSF, which is alarming to me.
It's not just FSF, there's also ethical software types trying to destroy OSI from within. We are seeing so many attempts to dismantle important organizations now it leaves me baffled.
> To me it feels as if a new generation of tech minded people have started taking things for granted
Indeed. Most fundamentally, they take their computing freedom for granted. There are powerful forces who want to take it away from us. There are powerful and rich people out there who think it's too dangerous for mere citizens to have access to unrestricted computers.
People focus on Stallman's software licenses as if that's what he's all about. They barely matter. People ignore the big picture: computing freedom. It's not just important, it's fundamental. Without this freedom, there is no hacking as we know it. There is no way to write and run our own software. What good is free software if a corporation or government signature is required to run it?
Stallman saw this coming before most if not all of us.
> Most fundamentally, they take their computing freedom for granted. There are powerful forces who want to take it away from us. There are powerful and rich people out there who think it's too dangerous for mere citizens to have access to unrestricted computers.
To an extent, I agree. I wonder whether the general move in the open source community will turn out to be a Faustian bargain. Are open source developers just a bunch of suckers being taken advantage of to provide building blocks for various proprietary business models? OTOH I'm thinking that more open source code out there is a public good, and we should rejoice in that regardless of 'proprietary free riders'.
> People ignore the big picture: computing freedom. It's not just important, it's fundamental.
No. What's fundamental is freedom for people. Computing freedom matters only as a means to that end.
> What's fundamental is freedom for people. Computing freedom matters only as a means to that end.
I agree, and I'm going to guess the person you're replying to also agrees. In fact, RMS would also agree. It's just that we are narrowly focusing on a single aspect of freedom in this conversation, but yes: freedom to live, to work, to education, to health, are all more important :)
> Are open source developers just a bunch of suckers being taken advantage of to provide building blocks for various proprietary business models?
Perhaps. The whole point of the GPL is to give free software developers advantages and create leverage we can use against those who would exploit our labor. Permissive licenses allow that exploitation.
In the end, it's irrelevant. All this software, all this positioning for advantages and leverage, none of it will matter if the machine itself is taken away from us.
We'll never be truly free until computer manufacturing is as democratized as software development. Ideally we should be able to make computers at home.
> No. What's fundamental is freedom for people.
Absolutely. Computing freedom is a subset of personal freedom.
> We'll never be truly free until computer manufacturing is as democratized as software development. Ideally we should be able to make computers at home.
If we had a few NGOs that were committed to freedom, were transparent and produced good hardware it would be a more reasonable option. Sure I can dream about the future where I can 3D print all electronics at home, but that computer wouldn't run very good and we all know this.
> If we had a few NGOs that were committed to freedom
NGOs are still centralized operations that can easily be targeted and corrupted. We need decentralized manufacturing. Anyone should be able to make functioning computers and it should be impossible to stop or even regulate it.
> the future where I can 3D print all electronics at home, but that computer wouldn't run very good
It would be better than nothing. They may have relatively low performance but they would still be computers that are guaranteed to be free and completely under our control. The fact they exist at all would provide significant protections against attempts at controlling us.
The well regards for GNU and RMS in the Latin world is due to the strong philosophy against closed software, specially from MS. During most of the 90's and beginnings of 2000's, M$ would force their expensive licenses in governmental systems and would make everything to avoid interoperability, even within versions of their own systems.
End result was that countries would spend millions just to be able to share documents across agencies. Things started changing when KDE/Slackware, Ubuntu, Firefox, OpenOffice and other software systems started to become user-friendly, with nicer and intuitive interfaces. From this point on, governments started teaching GNU/Linux in schools and universities, explaining the benefits, the rights, etc, and then switching to open-sourced and free-software systems.
These things were debated ad nauseam in the 90s, I started to feel nostalgic. To me it seems like a semantic issue and, fortunately, we are free people and can use either name.
As for the "horrible implementation" part... People nowadays don't realize how it worked in practice. Many extensions, like in GCC, appeared because people needed them (and at least in the case of compilers they're clearly marked as such). Also, at the time the GNU utilities were superior to native tools. Many Solaris admins started by installing GNU tools to make the system more usable, for example. If GNU was really so terrible, why would they do that? It's very easy to criticize it now, but it was a huge effort by thousands of people and I'm vert grateful to all of them.
The fact that the post does not link to https://www.gnu.org/gnu/gnu-linux-faq.en.html, makes it immediately suspect. That is a necessary resource to consult when talking about the GNU/Linux name.
155 comments
[ 3.7 ms ] story [ 220 ms ] threadHurd or seL4...but obviously no one is interested in such an Kernel for servers, mobile or the desktop.
From MY perspective, the OSS OS community was in a much healthier state in the ~05's then it is now (Linux, BSD, Solaris, Minix etc) where all maintained and gave plenty of selection of ~OSS operating-systems, now it's just that Linux.
Middle layer is networking, file systems, ACL, monitoring, graphics and other acceleration in a rich os. Or monolithic deterministic real time blobs. Or small dedicated OS partitions for, security or similar hardware jailed applications.
Top layer is containers/jails inside the rich OSs.
Then fuchsia is a candidate for the bottom layer and for the security OSs and maybe for specialized OCI payloads.
Edit: Linux is not a OS but a Kernel...got it? Your should know the difference as a Windows/UNIX person ;)
By the way I am a Windows/UNIX person, note the UNIX part.
The FSF is not the Linux Foundation
Fucshia is already shipping on the Nest, lets see what happens next.
Excluding Google (Fuschia) and Huawei (HarmonyOS).
That's something the article leaves out: while GNU and Linux are many times seen as a unit, there's a rich history of installing the GNU tools on other Unixes, because of the extra functionality they offer.
There is, of course, the argument against portability. At my last job, as the only Linux user on my team, I more than once wrote shell scripts that only worked for me - as soon as someone on macOS ran them, they'd run into a missing feature on `sed`, or a different calling convention on `date`. Which is why, at my current job, I'm very begrudgingly a macOS user - it makes me a better coworker.
Besides the lack of portability to other Unix systems isn't a real problem. The operating system itself is portable! /s
If you follow the FSF's actions closely, you will find they're incredibly insecure about people switching away from their software to alternatives. Typically the alternatives are not GPLed but rather under a permissive BSD-like license, and they latch onto this to deride any new competitors [1]. But those competitors don't typically exist because people hate the GPL; they exist because -surprise- GNU software isn't the be-all end-all, and it's a fairly common pattern for GNU maintainers to be reluctant to change at best, or actively hostile to third parties at worst (anyone remember glibc's Ulrich Drepper?).
Then there's how Stallman vetoed GCC having a useful AST output mode (a requirement to build smart IDEs and other development tools - yes, including such features in emacs) because he was scared of third party proprietary extensions. That's one reason why clang took off - its extensibility and flexibility, which the FSF was always against GCC having. The FSF (and particularly Stallman) hates clang, again reaching for licensing and moral arguments, because they just can't accept that some people may have written technically superior software to theirs, and done so with a more permissive license. [2]
In the end, it's hard to see the FSF's response to these things are anything but controlling and attempting to stifle competition. And we absolutely need competition for a healthy free software ecosystem. There is no value in trying to represent GNU as some kind of indispensable component of a Linux-based OS - they aren't, and pretending they are hurts us.
[1] https://github.com/uutils/coreutils/issues/1781
[2] https://gcc.gnu.org/legacy-ml/gcc/2014-01/msg00247.html
Well, if chrome is the primary set of userspace tools, "chromebook" might be a reasonable name.
I'm not sure how to differentiate android/linux (or even Chrome/linux) from UNIX-philosophy/linux, whether UNIX-philosophy ends up being busybox or GNU or something else.
A fact that termux people refuse to accept, hence why it doesn't run on the latest versions, unless they now finally accepted that POSIX isn't part of the official NDK stable APIs and have started calling into JNI.
Plenty of Linux software won't be compilable just with these APIs, unless they are game based,
https://developer.android.com/ndk/guides/stable_apis
FWIW, Termux works perfectly fine on the latest version of Android - what they can't do is push updates through the Play Store, because the minimum required API level now imposes security restrictions (no running executables from your data directory) that break it. But that doesn't make Android not a Linux system; you can still adb shell into it and run commands and shell scripts and copy new binaries in and run them.
This is not unlike Linux container systems, which also break various apps. It comes with the territory of containerizing applications that you have to break insecure legacy APIs; this is not unique to Android.
Regular users don't have any idea what adb is, and it requires developer mode to be enabled.
> The FSF (and particularly Stallman) hates
> FSF's response to these things are […] controlling and attempting to stifle competition
The “hate” seem to be all on your part.
I have a whole Twitter thread on how the modern FSF's policies actually hurt user freedom while being completely inconsistent with their own ideals, if you'd perhaps consider opening up your mind to criticism of them: https://twitter.com/marcan42/status/1377899929209774081
I support user freedom. I support users having control over their own devices, knowledge of what software runs on them, and knowledge of what choices are available so they can make an informed decision. I spend most of my time working on reverse engineering and porting Linux to proprietary hardware (Apple M1) so users have the freedom to run whatever OS they want. I cannot support the FSF, an organization which supports censoring security vulnerability warnings when fixing them involves updating proprietary firmware (that already exists anyway); that supports burying proprietary firmware in non-introspectable read-only memory because then they can "pretend it's hardware", then selling the result as "fully libre" (even though the same device with the same firmware visible and accessible to the user would be much more free and transparent, allowing introspection, auditing, and replacement with a free alternative); that supports physically destroying hardware components that presently require a proprietary driver to function, ignoring the possibility that free software drivers might appear in the future. These are real FSF policies and actions. This is what you are supporting when you support the FSF. It's not user freedom any more. It's religious nonsense like "blobs in /lib/firmware are bad, we can't have any blobs be out in the clear".
Totally agree, they have been completely swallowed by their own ideology. This shows that their actions are contradicting the same things FSF says publicly: that reverse-engineering is important.
That's a GitHub issue title filed by an FSF director on a project whose goal is to rewrite the basic POSIX core utilities in Rust, for security and maintainability. Because they MIT licensed them. They are literally telling a new competing project that they are worthless and an attack on the FSF due to their license choice.
> They are literally telling a new competing project that they are worthless and an attack on the FSF due to their license choice.
If that’s your interpretation of that quote, I’m not sure I can help you.
> OK, now have a look at the 120 lines long source of GNU yes. And compare now that behemoth against the implementation in OpenBSD, the one from sbase, or the one from busybox. Like… why does GNU yes does so many memory operations for a task that is implemented so easily by everyone else?
The answer can be seen here: https://news.ycombinator.com/item?id=14542938 https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...
But from the style of the author, it is clear that it is the case of a knowledgeable well informed educated person. GNU clearly has its problems, but badmouthing it like that is hard to differ from malice.
[0]: https://www.gnu.org/prep/standards/standards.html#Reading-No...
I don't see why the standard for composable utilities should allow for inefficiency.
It's not a bad retort, it's the end of the story: there is a superior utility, it's implemented.
My position is: either articulate an argument for why it's bad for /usr/bin/yes to be implemented the way GNU yes is implemented or drop the argument entirely.
Your comments manage to be in disagreement with those on the other side of this, and yet still adopt an argumentative/defensive tone with respect to what I've written. I consider this to be an even bigger waste of time than the people (like OP) who are upset about GNU yes's implementation.
> as bad as the GNU advocates' retort ("GNU yes is 10GiB/s fast").
If you don't want to defend your position, don't take it.
(Are you under the impression that "X is at least as bad as Y" is the same thing as saying "Y is bad"—and that that means I need to defend the position that Y is bad? It doesn't. Maybe learn to read English better, especially before leaping so confidently into trying to argue with someone over what you're reading. Another bonus tip: not only is it important to read in complete sentences, but it's important to write in complete sentences, too, and to make sure that the sentences follow logically one after the other.)
But that is not why I commented. I commented because I wanted to illustrate that the author questioned something that can be easily answered with a simple google search. I'm sure the author is qualified enough to answer the question, but instead left in the post giving most readers a impression that the complexity is there out of incompetence of GNU developers.
EDIT: I should not assume you asked in bad faith, but as the sibling commented. The answer to your question can be easily found in the link I posted. Making your behavior strangely similar to the author of the post: making questions that can be easily answered while giving a bad impression of others. Please, avoid that. It may work in many places but will fire back at you sometimes.
(Skills are not the same as factual knowledge here.)
Maybe it is a sensible subject for me or I've been living with people asking rhetorical redundant questions as a passive agressive form for disqualifying others.
I read your question as: "I can't see any reason this performance is needed, maybe it is really incompetence or pedantism from GNU developers." But I think it is prejudice on my part.
Sorry.
EDIT: grammar.
Anyways, back to the original question, I do believe that a lot of programmers (I don't buy OP's separation between GNU and non-GNU) over-engineer things without regard to the costs created by over-engineered code. My default assumption is that that's happening in yes. If it's happening in such a small program where it's so obvious, the same tendencies are likely to be at work in more complex programs like cat and awk and cc. Happy to debate that :)
That whole thread makes me sad. Another, more subtle way IP is ruining the world. But that's neither here nor there on this debate.
Just to flesh out my current thinking a bit more:
* I'd really like a study to understand how much complexity old GNU tools have gained in the past 2 decades. Because that would not be due to copyright concerns. But I'm not sure I have the necessary skills. (I'd like this study for non-GNU as well, but I assume copyright is not a concern there so the study doesn't have to set crisp time bounds.)
* I share some of Drew DeVault's frustration in https://drewdevault.com/2020/09/25/A-story-of-two-libcs.html. I've viscerally felt the immediate sense of disempowerment he describes when cracking open an open source package. _However_ I vehemently oppose his cavalier lack of concern for locales. Internationalization is important. ASCII is just as effective at disempowering others.
You are welcome! Considering the karma I got from that answer, I don't think I'm the only one who misread you. If you want a piece of advice: starting a question like that with "Genuinely curious: ..." is my strategy to prevent this kind of misunderstanding.
> * I share some of Drew DeVault's frustration in https://drewdevault.com/2020/09/25/A-story-of-two-libcs.html. I've viscerally felt the immediate sense of disempowerment he describes when cracking open an open source package. _However_ I vehemently oppose his cavalier lack of concern for locales. Internationalization is important. ASCII is just as effective at disempowering others.
As Drew DeVault's says, it was his code which was broken. I know glic code is complex, but it is battle tested, mature and highly robust. I'd bet there are good reasons for that complexity. If I was really curious as to why, I'd try asking devs directly to avoid making them look bad by making public statements about it.
Now, I looked in scdoc code. The solution was to cast the parameters to "unsigned char". Considering the code
compiles without warning (even with -Wall -Wextra) on my machine, I'd consider this a potential vulnerability for programs using this function. I can't see why glibc developers don't turn isalnum into a macro that automatically casts its parameter (when its address is not taken). I'll consider asking them this weekend.EDIT: possible nice finding: https://news.ycombinator.com/item?id=24597461
EDIT2: part of the reason: "unsigned char" can't hold EOF.
Regarding "asking devs directly": I often find it challenging even to figure out where to ask. Going from a program to its package, dealing with interactions between packages, etc. All of which goes back to simplicity as a core goal. "Good reasons for complexity" isn't sufficient, IMO. Complexity requires having a place to explain the reasons, and making it easy for people to find the place. Considering that we're all volunteers when it comes to open source and this sort of documentation is difficult, I'd often rather err on the side of not having a feature rather than having a more complex feature.
On the other hand, there are fields like internationalization or power management on laptops that are irreducibly complex. Not sure where I'm going with all this. Open source today serves businesses more than people, IMO. But it also serves people indirectly through businesses. This is a difficult topic. Your comment made it seem like there's an easy thing Drew or I could do. I'm not sure that's the case.
I also think internationalization is important, and that ASCII is mostly good at disempowering others. But at the same time I hate the way C locales work. They're a ridiculously terrible way of dealing with the problem, and an endless source of bugs.
And for that matter, what should `isalnum(六)` return? (in case HN mangles that, I put the Japanese "Roku" kanji there, i.e. the numeral 5). I'd say it should return `true` no matter what locale you're set to, since it's a numeral. But glibc returns `false` unless your locale is set to Japanese when `isalnum` is called, and musl returns `false` always. Yet they inconsistently return `true` for `isalnum(5)`, no matter the locale.
Case in point is that most people have no idea of how POSIX defines "text file" (ie. ends with \n, no line is longer than LINE_MAX), and what happens when you inout something that technically is not text file into POSIX utility that expects text file is undefined. GNU tools do something sensible or at least produce an meaningful error message, traditional Unix userspace usually does not (lines get truncated, last line sillently ignored, some buffer somewhere overflows and in better case you get SIGSEGV...).
But `yes` isn't such a thing, it's a standalone tool. Here the UNIX philosophy "Write programs that do one thing and do it well" comes rather at play.
Further, the optimizations it makes are not really complicated nor expensive to maintain. A page aligned buffer twice the most common page size is standard for basically everything where something is written or memory moves around.
One certainly can argue that it is Ok doing it more simply as it still is more than fast enough for the common use case. But arguing that a bread and butter optimization, namely one that adds just a few dozen lines of code total while increase throughput by three orders of magnitudes, should not be made, or that it even points to a complexity problem, is not holding any ground IMO.
> And we like things following the standards already put in place. Because we don’t give a damn about helping the FSF do their thing: We care about tech. We care about and love the legacy of UNIX, its philosophy, and we want to adhere to it as strictly as possible.
Sorry for breaking it up to you, but this kind of "standards must be respected 11!!!!1&!1" mindset is the actual cult-like behaviour, commonly found in communities such as cat-v.org, and with people clinging to plan 9 like it's anything beyond a failed experiment.
GNU and shells like bash (I use zsh which is I guess even "worse" in terms of feature bloat) are dominant because they provides a ton of super useful features to make work faster. Likewise you bash against glibc's bloat, but most software perform faster when using glibc than other "less bloated" libc implementations like musl ; the bloat is here for a reason and that reason is performance (which matters infinitely more than whatever ideological purity BS someone can find). Who cares if it's not strictly POSIX compliant: it makes computing a better experience and that's all that matters, and if tomorrow someone manages to create an entirely different non-posix platform that manages to improve on that, I'll happily jump there and I hope you do too.
I had assumed that this was about shell script portability. Bash and zsh have a ton of super useful features to make work faster, but the features for shell programming (writing shell scripts that other people will use) are not so useful. Yes, I'm aware of stuff like [[, I don't think that [[ is very useful. I have dash as /bin/sh and #!/bin/sh in all my scripts, and seriously, it's not a problem.
https://wiki.ubuntu.com/DashAsBinSh
(Note that Systemd has since obviated most of that work.)
You may be writing very different shell scripts than the ones which I encounter. I'm not sure what you mean by "variable expansion", because variable expansion is portable. Something like 98%+ of all pattern replacement I see is something simple like replacing a suffix or prefix of a string. For example,
You can typically use sed for the complicated stuff.Above a certain level of complexity, I'd likely rewrite the script in Perl or Python, anyway. Or possibly Go.
At least historically speaking, Bash was noticeably slower than Dash. It's the slowest shell. If you are doing things like, say, build scripts in a CI environment where you need to execute shell scripts thousands of times or more, the 4x difference can make a real impact on user experience. Same thing during startup, back before the transition to Systemd, when Debian changed the /bin/sh alias to point to Dash instead of Bash to speed up system startup.
https://unix.stackexchange.com/questions/148035/is-dash-or-s...
https://wiki.ubuntu.com/DashAsBinSh
At some point, shell scripts reach a level of complexity where most people choose to rewrite them in a higher-level language, often Python or Perl, or maybe even Go. My personal experience is that the Bash-isms, although kind of nice, don't really shift that threshold... like, although Bash has arrays, but if I need to manipulate arrays, I switch to Perl or Python. If I just need to pull elements out of one array, I can do that portably.
Python has an enormous overhead compared to shell scripts, though. If you're writing some simple wrapper in a build system that's going to be invoked thousands of times for a single build, Python could easily end up being a choke point.
If you're really concerned about portability, I would probably write it in Perl :-)
Perl is basically a souped up shell language, with built-in Awk & Sed, faster start-up time than Bash, and it's installed nearly everywhere.
In terms of build systems, what I have most experience with would be with Debian and their build system, which pulls in the required dependencies. Compatibility in this space is less about zsh vs bash and more in line with architecture difference like RISC vs AMD64.
Whilst I broadly agree with your point, I'd suggest that "better experience" can be extremely "eye of the beholder" - especially if it interrupts a long established workflow / toolkit.
That's a bit of a non sequitur: Plan 9 is very much not a standard, and indeed it derives a great deal of its power from not implementing the POSIX standard.
Nothing's worse than the cult of POSIX. Half a century ago some people gathered and figured out the lowest common denominator of all the unixes of the time. Now it's 2021 and we're still supposed to restrict ourselves to this "standard" system in the name of portability. POSIX has so few features that restricting yourself to it is masochism. Every unix adds their own extensions, they would be almost unusable if they didn't. So why is GNU criticized for doing the same?
The author says GNU is just some optional userland. I find that amusing since in every traditional unix the POSIX userland is deeply integrated with the system. He talks about glibc but the BSDs and even Windows ship with their own libc that I can't ever get rid of because they're the only supported way to interface with the kernel.
Linux is the only operating system that actually frees us from this cult. Unlike other systems, the kernel/userspace interface is stable and defined at the lowest possible level: processor instruction set architecture.
https://man7.org/linux/man-pages/man2/syscalls.2.html
https://man7.org/linux/man-pages/man2/syscall.2.html
https://github.com/torvalds/linux/blob/master/Documentation/...
Only on Linux is GNU truly just some optional userland. You can throw it in the trash if you want. You can make your own. Who says you gotta have little commands like cp, mv, grep, sed, whatever? Who says people have to use some "standard" shell from the 80s? You can make a graphical userland if you want. A 100% Rust or Lisp userland. Even no userland at all.
People complain about systemd but I actually have a lot of respect for it. The developers had the balls to trash all this sacred POSIX stuff and make something new that actually uses Linux kernel features. The resulting system is better for it.
Just to be clear, Ariadna Vigo is not attacking computing freedoms; pretty much the opposite. I agree with her position on nearly everything except for her stance about the GNU operating system. This is a technical detail in the grand scheme of things. Just like tabs vs. spaces, for which it is fun to disagree in an over the top way.
We must support the FSF for pragmatic reasons, but we can still acknowledge that the "pure unix" philosophy is a great and beautiful idea. Also, cat-v and suckless are fun websites full of interesting stuff.
Who is this 'we'? I don't care about the UNIX philosophy (I'm an emacs user). First thing I do on bsd systems is install the GNU userland. BSD tools follow the POSIX spec like a cult refusing to add useful extensions. Look how hopelessly broken posix sh is. I work quite often with busybox on small embedded systems and the limitedness of its vi implementation makes me irrationally angry.
Be wary of posts and articles which talk about an universal "we" as if everyone thought the same.
I, for example, care more about the philosophical aims of the FSF and RMS than about the finer technical points. That Linux is a very useful OS for me is a bonus. So "we" do not just care about tech; "we" also care about users rights and freedom.
Although you can pay the FSF for copies is (I think) possible calling GNU a vendor similar to "MS or Apple" is too much of a stretch. When "MS or Apple" contributes back agreeing to the license, out of goodwill or because it helps the development, it is a nice behavior in their part; otherwise they are just being themselves.
As for why GNU is a different case with regards to extensions... I can think of a few reasons:
Ok, portability may suffer, but that is the price some groups choose to pay. Considering the gained benefits, I'd say it is a fair price.This command generates a strong password. How do you do it with only posix?
But uuencode exists in the POSIX standard. https://pubs.opengroup.org/onlinepubs/9699919799.2018edition...
I don't think POSIX even defines any functions/system calls for getting cryptographically secure random numbers, so if you're literally restricting yourself to POSIX constructs, then indeed what OP asks for is impossible :)
You may need to be careful because the major number is shared with /dev/null, but I doubt these numbers are hardcoded in many places.
You could add a new major that does this (keeping the old numbers for compatibility), but I think you'd have a hard time convincing people that this is a worthwhile enough idea; hard-coded new major numbers are in short supply.
head -c 20 /dev/urandom | md5sum
Criticizing GNU for providing features is absurd.
Users need features, not phylosophycal purity.
Software needs less politics, not more.
I always blamed MacOS/BSD for just being different, but now I know that it's GNU that is different. I'm not sure what to think about it though--I do like the idea of compatibility with a standard, but I also don't like standards to hold back innovation.
Commercial unixes "based on" SysVR2/3/4 had plenty of "non-standard" behavior and extensions, it was how you differentiated your product. You support POSIX compatibility to enable portability if you need it, but you differentiate with your extensions or "weird behavior". GNU was no different. The BSDs just have their own tradition, one that's older than POSIX.
There are good reasons for many of those extensions. Many options implement capabilities that probably should be in the standard, but currently aren't. For example, POSIX doesn't have easy-to-use secure options for handling file names; attackers can insert newlines in filenames, yet POSIX lacks options like find -print0 and xargs -0, so you either spend a great deal of additional time writing complicated code that's likely to be insecure, or you simply use extensions as almost everyone actually does. Even the BSDs and busybox include those examples, by the way. Also, long option names are excellent for making scripts easier to understand, and really should be in the POSIX standard for systems that are not highly memory constrained.
I try to make my shell scripts portable when that is reasonable, but sometimes that is simply far too difficult. The minimalism of POSIX is an advantage on small systems, or in other constrained environments, but most of us today are not trying trying to write software for a PDP-7.
In most cases the scarcest resource is developer time. In that case having rich functionality and extensions that improve readability is more important. I think the market is being perfectly reasonable, minimalism is less important than developer efficiency in most cases. Your mileage may vary.
Because POSIX (and partiall also ISO C) is just ex-post stadardization of features that appeared first as vendor extensions, and also does not cover many areas entirely. Useful extensions became part of standard, and also many programs just need to use non-standardized parts of API (e.g. some Linux-specific parts).
> If you thought I was going to go the “Android route” of argumentation… You know… saying that Android shows that Linux can be used without GNU… Sorry! I’m way more sophisticated than that!
Sophisticated in what sense? That when you search the Android source tree you find FSF code?
https://cs.android.com/android/platform/superproject/+/maste...
FWIW, here's a search for FSF-copyright code that excludes build generator output (autotools/automake/libtool/etc) and gcc/bison/etc related stuff (which many other projects which are not themselves FSF-authored use), as well as licenses themselves:
https://cs.android.com/search?q=copyright.*free%5C%20softwar...
193 hits, and a lot of that is ffi and libbacktrace stuff. Exclude those two projects, and you're down to 47 hits, most of which are random things like headers, test shell scripts, and some translations. I think it's fair to say that Android does not include any significant amount of GNU/FSF code.
(Keep in mind that the Android codebase is massive and vendors a huge amount of third-party projects, so it's actually quite remarkable that there is so little FSF code in here - part of that speaks to how little FSF people contribute to other projects!)
https://www.youtube.com/watch?v=OPgo6s1lBbw
Of course when Linux became a viable they decided to abandon their plan and concentrate resources on userland. Kinda opposite to the usual of similar projects being co-developed leading to fragmentation. If there's an already capable kernel, why write another?
Another problematic point is the "look how much lines it takes, such a bloat" for `yes`. And then proceed to call it convoluted for pointless reasons. I guess being[1] about 100x (you read that correctly) faster is a pointless reason if a binary program you use rather develop can be written in fewer lines of code. Those GNU devs really should be more lazy.
Also there's "portability". If you write a program that is utilizing Linux' specific calls then your program isn't portable to other Unix-like systems. Same way that if you write `/usr/bin/env bash` at your script it won't be portable to other shells. You can of course install some POSIX utils implementation and use those, and you'll limiting yourself, but at least you'll have portability to systems you aren't using.
[1]: https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...
Needs correction.
The article raises interesting points, but the whole rant against GNU and FSF is weird. GNU (yes, and Linux) and FSF is why many of us are in this business. They played a huge role in the career and passions of many of us -- I feel entitled to say "us" just like the author of TFA carelessly speaks about "we" -- and misunderstanding the history of how we got there doesn't help.
To me it feels as if a new generation of tech minded people have started taking things for granted, "why do we even need all those movements fighting for users' freedom, if I am free now?". It's a bit depressing.
The different approaches and goals of Linus Torvalds and RMS are both valid, and only in hindsight can we say one triumphed and the other didn't, but the article makes it look as if one of the two was obviously wrongheaded.
So some interesting points, but it could do without the whole rant against GNU and the FSF, and without the accusations of cultishness.
Anyway, It's not over, and knowing how markets tend to monopolies, we'll be fighting forever. Which won't be a problem once numbers are on our side. Right now we're in a stale mate with much of the front being taken by proprietary software. That is not likely to change until the free software community gets a better face, but agenda-pushing codes of conduct won't do that. We need a serious discussion about software freedoms
But I think more generally, wrt FSF there's not so much controversy [1] as disappointment in the FSF failing to stay relevant. Copyleft was a solution to the 80/90'ies software world. The FSF tried with the AGPL to answer the 'threat' of SaaS, but largely the world has shunned it except as a poison pill license to upsell a proprietary version. Similarly with the GPLv3 the FSF tried to answer to the threat of Tivoization, but again, largely failed to gain adherents in the low level components, like the Linux kernel, where this would have mattered.
[1] Well, the FSF taking RMS back on its board did produce a tsunami of corporate (financial) contributors leaving ship.
It's odd because it's worded in a way that sounds as if the FSF wasn't necessary now (or maybe even never), instead of thinking that the FSF is needed more than ever, now that there's a war against general purpose computing and against privacy and users' rights, and that if the GPL3 and AGPL didn't work -- which is possible! -- then how do we fix that?
Instead it's either apathy and even scorn against the FSF, which is alarming to me.
It's not just FSF, there's also ethical software types trying to destroy OSI from within. We are seeing so many attempts to dismantle important organizations now it leaves me baffled.
Indeed. Most fundamentally, they take their computing freedom for granted. There are powerful forces who want to take it away from us. There are powerful and rich people out there who think it's too dangerous for mere citizens to have access to unrestricted computers.
People focus on Stallman's software licenses as if that's what he's all about. They barely matter. People ignore the big picture: computing freedom. It's not just important, it's fundamental. Without this freedom, there is no hacking as we know it. There is no way to write and run our own software. What good is free software if a corporation or government signature is required to run it?
Stallman saw this coming before most if not all of us.
To an extent, I agree. I wonder whether the general move in the open source community will turn out to be a Faustian bargain. Are open source developers just a bunch of suckers being taken advantage of to provide building blocks for various proprietary business models? OTOH I'm thinking that more open source code out there is a public good, and we should rejoice in that regardless of 'proprietary free riders'.
> People ignore the big picture: computing freedom. It's not just important, it's fundamental.
No. What's fundamental is freedom for people. Computing freedom matters only as a means to that end.
I agree, and I'm going to guess the person you're replying to also agrees. In fact, RMS would also agree. It's just that we are narrowly focusing on a single aspect of freedom in this conversation, but yes: freedom to live, to work, to education, to health, are all more important :)
Perhaps. The whole point of the GPL is to give free software developers advantages and create leverage we can use against those who would exploit our labor. Permissive licenses allow that exploitation.
In the end, it's irrelevant. All this software, all this positioning for advantages and leverage, none of it will matter if the machine itself is taken away from us.
We'll never be truly free until computer manufacturing is as democratized as software development. Ideally we should be able to make computers at home.
> No. What's fundamental is freedom for people.
Absolutely. Computing freedom is a subset of personal freedom.
If we had a few NGOs that were committed to freedom, were transparent and produced good hardware it would be a more reasonable option. Sure I can dream about the future where I can 3D print all electronics at home, but that computer wouldn't run very good and we all know this.
NGOs are still centralized operations that can easily be targeted and corrupted. We need decentralized manufacturing. Anyone should be able to make functioning computers and it should be impossible to stop or even regulate it.
> the future where I can 3D print all electronics at home, but that computer wouldn't run very good
It would be better than nothing. They may have relatively low performance but they would still be computers that are guaranteed to be free and completely under our control. The fact they exist at all would provide significant protections against attempts at controlling us.
End result was that countries would spend millions just to be able to share documents across agencies. Things started changing when KDE/Slackware, Ubuntu, Firefox, OpenOffice and other software systems started to become user-friendly, with nicer and intuitive interfaces. From this point on, governments started teaching GNU/Linux in schools and universities, explaining the benefits, the rights, etc, and then switching to open-sourced and free-software systems.
As for the "horrible implementation" part... People nowadays don't realize how it worked in practice. Many extensions, like in GCC, appeared because people needed them (and at least in the case of compilers they're clearly marked as such). Also, at the time the GNU utilities were superior to native tools. Many Solaris admins started by installing GNU tools to make the system more usable, for example. If GNU was really so terrible, why would they do that? It's very easy to criticize it now, but it was a huge effort by thousands of people and I'm vert grateful to all of them.
Huh? Torvalds never intended to create an operating system, just a kernel, and that is true to this day.