What problem is this warning trying to solve? Are these two symlinks too much maintenance burden? Or is the check in the code hurting the code quality? Is the extra check at startup ruining performance?
I'm usually in favour of having one way to do things but in this case, with this much legacy it just doesn't seem worth it.
This is free software. It is legitimate to consider convenience for the maintainers. If they don't want to maintain two symlinks then they are empowered to make that call.
If anyone thinks it is a big enough problem that they want to fork the software they can, or the distros can maintain their own symlinks. But I think in this case the simple answer is if the maintainer doesn't want it in the source tarball then it isn't going to be there and that is more than sufficient a justification.
Complaining is probably more reasonable than asking for a justification here.
I don't know when society in general just said "Fuck It" to the idea of stewardship; but we now have it ingrained that people in positions of trust and power (whether volunteered, elected or appointed) are not morally or ethically beholden or responsible to the communities they have taken it upon themselves to represent.
At least we used to pay lip service to that ideal.
They didn't necessarily "choose communities to represent". As the maintainer and author of various open source libraries and tools that are used by many thousands, in most cases it's just that ... I'm one of the few willing to spend the time on it, and it's usually useful for myself as well. I don't really "represent" any community or anyone.
That said, I certainly wouldn't have put in this change myself, because I wouldn't like to inconvenience anyone. But that's just basic good manners that you should have in every-day life to random strangers.
Well, if I write some software for my own use, put it on the internet "because why not?", and lots of people start using then that's nice. But ... I don't think putting anything on the internet automatically imparts any kind of responsibility towards "the community", which usually means "people who download and use your software, the overwhelming majority never give anything back in the form of code, bug reports, money, or anything else".
But like I already said in my previous comment, I wouldn't have made this change myself. I actually strongly disagree with it. But I can also accept that other people have a different attitude, and that's okay too, even if I personally don't really care much for the particular attitude.
Funny you mention OpenBSD, because OpenBSD is very much "by the developers, for the developers" and has a fairly decent "fuck off" attitude once people start making demands (which I don't think is a negative per se).
That's a very different thing than joining GNU. If your package gets adopted by thousands of peoples that's having this community thrust upon you.
I still feel you have an obligation to transfer to someone willing to shoulder the responsibility if this happens, an obligation incurred by publishing it in the first place.
Jim Meyering made the change, who has been maintaining these things from before half the people here were born.
Even when taking over maintainership for popular packages later on, people often aren't exactly breaking down the door for it. So my philosophy is simple: "if you do the work, you get to decide". I may like or dislike these decisions, and at times I may even rant about how stupid a certain decision is, but in the end ... the people doing the work get to decide. The alternative of being beholden to a vaguely defined "community" of armchair quarterbacks is much worse, IMO.
Are you suggesting that a 15 year deprecation path is a fuck it attitude towards stewardship?
Distros are free to've replaced the symlinks with wrapper scripts in the meantime. Granted, I expect Arch will go ahead & let the symlinks disappear. But I can't imagine what you think of Arch's stewardship
I expect this should have about as little impact as the usr merge many distros have gone through
I'm replying to the attitude in the comment I'm replying to, if I meant this as a comment on GNU's approach to fgrep/egrep, it would have been top level.
The original deprecation seems ... petty? Not sure why it's a priority, but whatever.
I know there are scripts still in use that I wrote more than a decade ago that might do weird things now and I wish the maintainers best of luck.
One major issue with humans is that sacrifice begins to be expected and often is not rewarded. When a job not only becomes thankless, or near enough, but also expected as the default, it becomes hurtful to continue doing it.
Are the stewards being provided fair compensation? How do we even talk about what is fair compensation when non-monetary compensation has become difficult to even discuss (often due to past instances being extreme disproportionate or of a form that is no longer tolerable).
The simplest way to put it is that if you can't find a steward you aren't paying enough and trying to use appeals to morals or ethics to get people to accept lower pay no longer holds as much weight when being moral or ethical no longer provides the same level of non-monetary benefits.
> One major issue with humans is that sacrifice begins to be expected and often is not rewarded. When a job not only becomes thankless, or near enough, but also expected as the default, it becomes hurtful to continue doing it.
Oh, most definitely. The anger and vitriol I see directed towards maintainers, or volunteers of any type, who are "stepping back for personal reasons" is horrifying.
My father was a very active volunteer in his community ... the number of people who were mad at him when he stepped away after his heart problems was startling. Conversely, the number of people who volunteered to help him and my mother with shopping etc. when COVID hit was heart warming.
> At least we used to pay lip service to that ideal.
When was that? For as long as I can recall, a core mantra of free/libre software has been that it was provided "AS IS" without warranty of any kind. Decades ago the dominant response I recall was one of gratitude and a little amazement that ad-hoc communities of volunteers were making real software that wasn't just academic but on par with commercial offerings. Some of those communities chose to adopt a user-friendly posture because they wanted people to like them but plenty did not and just did their own thing. As long as they could attract contributors they kept going.
Personally what concerns me is this growing expectation that volunteering to maintain an open source project also means you are "morally or ethically beholden or responsible" to anyone who uses it. In practice that seems to mean maintainers must respond to user requests or end up on the receiving end of a great deal of vitriol. It's no wonder so many volunteer maintainers who have internalized this responsibility are burning out, and how many more potential maintainers are dissuaded by seeing what is happening to the current maintainers.
Well, concurrent with this decline in stewardship has been a decline in graciousness towards volunteers, so I see your point.
I remember when I was younger, every volunteering experience I had was a delight; people thanking me for my time, getting me free coffee "just because", etc. Some of my more recent volunteering experiences have been less pleasant.
This is "back in my day"-levels of lazy armchair criticism. Nothing has changed. People will always make decisions that you disagree with, and they are more likely than not doing it in a good faith attempt to benefit the community. These sorts of inflammatory and hyperbolic comments help nobody, and are just childishly over-the-top. egrep going away is not evidence of moral or ethical bankruptcy, just wow.
It can't be that much work to maintain a symlink. The idea that it could be less work to remove a standard feature that has been part of Unix for several decades, has no connection with reality.
There is no reasonable explanation for this decision except that somebody thought that having both egrep and grep -E was "ugly" according to their own personal sensibilities.
And this is what I hate from the argument "you can't expect the maintainers to always support the feature". No, I don't. But I do expect them NOT to regularly remove/break the features I have contributed!
E.g. I can't count the number of times I have submitted _the_ _same_ _GUI_ _fixes_ to certain popular browser over the decades. Because apparently "they" have to rewrite the chrome of the mobile version from scratch every handful of years. "They" is in quotes because it's, in fairness, never the same person or even the same group of people. It's a CADT.
And in this case, to remove what apparently are two small shell scripts which for sure cost more to remove than to preserve....
What 'maintenance' goes into symlinks that already exist? The only thing one needs to do is nothing at all. It is in fact more 'maintenance' to delete them.
What exactly do you imagine the maintenance being? egrep and fgrep are already shell scripts, and those scripts haven't been edited since 2015. If not needing to change something for 7 years counts as a maintenance burden, sign me up.
Unfortunately, this seems symptomatic of GNU's attitude: we've come up with something silly and it's going to screw some people over badly in the least agreeable moment, but we're too stubborn and proud to admit it. Take it, or buy a Windows 11 license.
At a guess, it isn't actually the symlinks, it's the argument parsing they're trying to simplify.
Old programs have stupidly complex argument parsing. If you can pass your arguments as "cmd foo bar path", "cmd -f bar path", "cmd -fbar path", "cmd path --foo=bar" or "fcmd bar path" and "cmdf path bar", it can be really convenient for users who can structure their commands in the way that makes the most sense for them.
But it can be really frustrating to maintainers who are maintaining -- and testing! -- a thousand lines of bespoke argument parsing for a ten line function.
It's really, really tempting to define a simple syntax for argument parsing, turn it into a library, and reduce your pile of shell commands to a few lines of argument configuration and a function call.
Why aren't these split into separate packages? If a distro wants to drop them, they can still be installed. If some distro wants to include them with a warning that's also fair game.
Because they're the same thing. What all is about are the `xgrep` commands being symlinks to `grep`. Though I guess you can have packages that just add the symlink.
>If some distro wants to include them with a warning that's also fair game.
Some distros already do what is recommended in release notes. Rather being symlinks they're wrapper scripts. E.g. in Nix the `fgrep` and `egrep` are just `exec ${nixpkgs.gnugrep}/bin/grep -F "$@"` and `-E` respectively.
I cannot edit anymore, so I add this: This is the case in a distribution (Arch) that has effectively taken the maintainer's advice into account, not the reverse.
So I was wrong, and let's not be unfair to the maintainer.
True, the symlink part is wrong. Guess I should've checked the code. But in similar vein, a distro can just have packages adding the scripts.
edit: But was not always wrong. Just terribly outdated. Searching the log I found the commit [5cb71b0] with the message:
Add patch from
Paul Eggert <> to comply with ridiculous
guidelines (don't act differently if invoked as egrep or fgrep)
which made the change from creating symlinks to creating scripts. The code continued to adjust behavior according filename (in contrast to what someone would expect based on the commit message). Then few years afterwards in [d25bebd] the scripts and the symlink behavior were dropped for actual binaries, with in-source comment:
/* We build specialized legacy "egrep" and "fgrep" programs.
No program adjusts its behavior according to its argv[0].
No scripts are provided as an alternative. Distributors
are free to do otherwise, but it is their burden to do so. */
It also funnily added the following prints, quite similar to what they're doing now:
Invocation as `egrep' is deprecated; use `grep -E' instead.
Invocation as `fgrep' is deprecated; use `grep -F' instead.
The scripts returned about a decade later (or else few years ago) in [b639643]. The commit message mentioned the reasoning:
Although egrep's and fgrep's switch from shell scripts to
executables may have made sense in 2005, it complicated
maintenance and recently has caused subtle performance bugs.
Go back to the old way of doing things, as it's simpler and more
easily separated from the mainstream implementation. This should
be good enough nowadays, as POSIX has withdrawn egrep/fgrep and
portable applications should be using -E/-F anyway.
1 line. Fixed. Done. Btw needs to be run as root ( as signified by # prompt above)
EDIT: see other comment in this thread from FreeBSD user that on FreeBSD grep, egrep, and fgrep are all separate but identical (copies) of the same file. So this isn’t quite such a silly solution as some might think.
Which has lower cognitive load if everyone starts doing it?
Explicitly specifying switches, or argv magic? I'd argue, the switches are. In the abscence of the symlinking, that is how the tool functionality would have to be driven anyway.
Argv magic now runs into a problem if another program sharing the name ever comes into existence on the path. It's also completely unergonomic in a sense, because short of looking at the source, you have no way of knowing what argv transforms implementations support are, and in order to use them, you must explicitly pollute the Symbol namespace with a denormalized util. Also, the argv magic does require one extra shell to do the transform from !grep to grep -!, Which is technically more overhead. On the other hand, fgrep and egrep are ironically easier to grep/sed for as opposed to grep/ -[F|E]
One tool, one manual, one name, one argv0.
I am not the Grand Poobah of the Internet, however, even if their hat is in my possession, so I understand that it is likely that the fgrep/egrep convention is probably deeply entrenched, and likely to spawn a new holy war on par with Emacs/Vim. Tabs/spaces, etc...
Yes, very useful to bypass aliases if need be, quickly edit a script ("vi =my-script"), etc. I don't think bash has it; you need to use where/whence/which/command/whatever (I can never remember, why are there so many?!)
Sure now that I’ve finished rebuilding all my dependencies to use 64-but time_t and migrated all my services to IPv6 I can finally take some time to finish the great *grep deprecation :)
There is so much outdated info on the internet in various forms that it is hard even to realize what is "proper modern way" of doing things unless you really are into Linux \ config stuff.
I am casual user so I can get around system but modern ways always surprise me when I finally find out about it.
And for all you know "grep -E" isn't supported on some system people use. This is not really a concern if you're just writing script for yourself (which are really >90% of scripts; portability often isn't really a concern), but knowing it will work on all systems – new and old – is pretty hard, and sometimes it does matter. Does it work on NetBSD? HP-UX? Solaris? Last year autoconf changed the `..` command substitution syntax to the "new" $(..) syntax and someone complained it broke on their ancient Solaris system.
So ... people will stick with what works, like "egrep".
I learned about the deprecation from ShellCheck [1] that warns if using egrep instead of `grep -E`. That tool deprogrammed many of my bad habits. I had never seen any discussions about it otherwise.
That's a cool tool, it's neat seeing Haskell in the wild.
Unfortunately (or not?) I tend to use actual programming languages to make my tools rather than doing shell scripting, so I don't have anything interesting to put in there, but I'll keep a bookmark of it around.
I for one didn't know they were properly deprecated.
Though I've not used either since uni over two decades ago¹ on a somewhat off-standard Unix a few of the machines ran, so until this thread I can't say I was remembering that they existed at all.
----
[1] using grep -E when needed since²
[2] and presumably grep -F too though I don't remember ever actually doing that
Most people shouldn’t have been using this in the first place. If you’ve been taught to use these in the last ~10 years then someone, somewhere, has failed horribly.
What actually happened here is that POSIX skipped it. It never entered a standard, even though it was (literally!) in every OS and available to everyone. But no one cared that it wasn't in some arbitrary standard, because it was always there. For half a century!
I dare say that "fgrep" and "egrep" have more active users (both interactive and scripted) than "awk" or "ed" or "od" or "bc", all of which are still around.
The blog makes two main points: 1) adding new error messages causes compatibility problems; 2) some people are used to typing "egrep".
On the first point the author only gives hypothetical examples. I feel the argument might have been more compelling if we could see some concrete examples of things that break with GNU Grep 3.8.
As for the second point, I find it less convincing than the first one. If it's just the muscle memory then an alias would be an acceptable workaround. And I doubt that "everyone in the world" would want such an alias, as the author suggests.
It seems pretty simple, piping bash commands into other bash commands and other text stream juggling is a pretty typical use of these commands and so changing what stream is output can change the behavior of consumers of the output of these functions.
I haven’t done anything with fgrep and egrep before but piping grep into another grep for more complex classes of text search is something i use a lot.
Automatically monitoring the stderr from cron jobs for unusual outputs is a prudent measure, and its plausible that this change will increase the burden of false positives (it certainly will not reduce it.)
But if you’re monitoring the output it usually means you are in a position to fix problems which means you can likely update the script in question to use the new warning-less invocation.
If I had been woken up in the middle of the night or had a vacation interrupted on account of this, I would not be entertaining warm and grateful thoughts toward whoever thought it was a good idea.
Wouldn't you test before upgrading packages in production? And usually you'd want to schedule any upgrades so that the next day or two has coverage from someone who can deal with any issues that arise.
I have yet to work at a place that didn’t have systems running mission-critical shell scripts with little to no SDLC on boxes that got periodic “yum update -y”s. There seems to be a difference in oversight of software we write & “the operating system”.
Should we do better? Absolutely! Will this burn people if vendors don’t take care? Also absolutely!
I don't know if I have sympathy for this argument.
Your script ostensibly handles (at least logs) errors and warnings right? Do you exhaustively handle every single error and warning in a unique and different way or do you have a catchall "If non-0 return code then fail"? How does introducing new output to stderr affect that?
All the replies so far are missing the point: it is prudent to monitor for unusual events, including previously-unseen messages, over and above the explicit handling of specific errors. It is also prudent to not wait until morning to investigate.
Your infrastructure probably is crap, as very few people get to build it themselves from scratch. That does not mean one should cheerfully accept additional unnecessary or pedantic complications.
It would also be prudent to investigate each and every change to any of the software you use, in order to anticipate problems, but unnecessary and pedantic changes increase the burden there, as well.
It is, however, very unusual to do so and then try to parse the output. Aside from compilers, what other CLI tools make any guarantees wrt what they print to stderr?
With regard to the first point, the examples may be hypothetical, but they are also very plausible.
When a change has little or no objective benefit, I feel the burden of demonstrating that it is harmless falls on those making the change.
As has been pointed out elsewhere, this is free software and the maintainers are free to do whatever they like. That does not stop others having an opinion about it, especially when it is in the form of constructive criticism.
Sure, but it would still be nice to have at least one such example. Looking at the rest of the discussion thread here on HN as of now it's still only hypotheticals.
Using that model, we can prove conclusively that, since a behavior has changed (a warning printed), it might cause problems. Therefore, we cannot prove that it cannot cause problems. What we would really like, though, is an actual problem shown to exist. Just like in mathematics; it’s one thing to prove that it’s impossible to prove something could not exist, but another thing entirely to show it existing.
You are overlooking something here: I never said anything about proof. I explicitly wrote 'convincing evidence' because proof is too demanding!
It's rather amusing how you have flipped from saying "you can't prove a negative" to an argument for the certainty of observable effects and the probability of consequences! (you wrote might cause problems, but everyone can see that's an unrealistic understatement of the implications of the argument you are using.)
The NASA managers prior to the Challenger crash thought that what they really wanted was something showing them an actual problem existed. Erring on the side of caution is generally prudent, even in relatively small matters.
> everyone can see that's an unrealistic understatement of the implications of the argument you are using
If nobody can show an actual existing problem, or even an example of reasonable code someone could have written which would be impacted by a the printed warning, then yes, I would think that I was charitable when I wrote “might cause problems”.
What does 'charitable' mean here? Generous towards what person or point of view? As you say you have conclusively proved that there is a non-zero probability of there being problems, the use of 'might' is already trying to persuade that this possibility is next to zero.
Yes, of course. If the case is reasonably likely to occur in the real world and have real world impact, that is. And I would assume that the GNU grep developers would agree with me.
I think that is a very reasonable position to hold. It would only be the rejection of plausible cases, on the basis of no actual case having been uncovered, that I would take issue with. When assessing the downside of a proposal, there should not be much, if any, difference between how highly plausible and certain consequences are assessed.
If we could prove there would be no downside, then plausible problems could be ignored as merely hypothetical, and there would be no need to posit offsetting benefits. The question the GNU grep developers might want to consider is whether the supposed upside will have sufficient material consequences for their purposes.
But it changes the behavior of the script in the UI. It can cause things like cron to send mail. It can cause other things wrapped around the script that are capturing both STDOUT and STDERR from the script to capture extra content. Any tool that's monitoring STDERR and expecting it to be empty may consider that an erroneous run, which may impact other scripted decisions. It's a breaking change in multiple circumstances, even if you don't consider extraneous warnings shown to a user manually running a script a breaking change.
Does that code look like something you'd log into a system and manually run on a regular basis? Does it maybe instead look like one layer of a legacy automation stack absorbed into other tools?
> On the first point the author only gives hypothetical examples
I have scripts everywhere, some of them 20 years old or more, that use fgrep. For years and years it was a "best practice" thing if you were checking for a fixed string (so that you didn't accidentally match on a "." or whatever by forgetting it was a regex).
You have two options: The first option is to simply replace "fgrep" with "grep -F" everywhere in all your scripts, which is correct but is more work than your other option, which is to add your own "fgrep" script somewhere in your path.
OK, are you going to call up all my former employers to tell them to audit the scripts I wrote for them in the late 90's?
I really don't think people understand the impact here. It's not it's just a bunch of angry geriatric graybeards yelling at the modern world. It's that there is decades of uncounted, unrecognized, untraceable software written using these old conventions that are suddenly changing.
It's just a terrible idea. Linux cares about conforming to syscall interfaces for binaries compiled 20 years ago, but somehow you think it's OK to break scripts that have worked fine for 50 (fifty!) years?
Either a system is frozen and static, in which case it will not receive this version of GNU grep, and there is no problem. On the other hand, if a system receives updates, the system needs both minor and major changes all the time, to keep up with its ever-changing environment. This is the jungle in which we live. Linux syscalls are important to keep, since it’s hard to change a compiled binary. But it’s easy to change a shell script.
And don’t exaggerate. This won’t, in all likelihood, “break” your scripts.
> This won’t, in all likelihood, “break” your scripts.
Previously:
> The first option is to simply replace "fgrep" with "grep -F" everywhere in all your scripts, which is correct but is more work than your other option, which is to add your own "fgrep" script somewhere in your path.
script.sh: line 100: fgrep: command not found
seems like evidence of a broken script to me. The fact that it can be fixed doesn't make it not broken.
We were discussing the warning printed by fgrep and egrep, not their removal. I was suggesting that you put a version of fgrep/egrep in your path which does not show the warning.
That wasn't my take. I thought we were discussing the deprecation itself. It's true that nothing is broken yet[1]. But it's clear that "broken" is where we're going, and I don't think you or the GNU maintainers have thought through the implications correctly.
[1] At least, nothing that isn't sensitive to junk on stderr -- bourne scripts are not always as tolerant as you'd like.
We were explicitly discussing GNU grep 3.8, which does not remove anything, only add warnings. And the remote possibility of breakage due to warnings is why I qualified my assertion with “in all likelihood”.
The shell happens to be a programming tool and not just an interactive command interpreter. There are whole books on how to write shell scripts portably across various Unix-y platforms. Many of the examples in those books will now throw warnings on systems using the GNU tools.
If a book on writing portable shell scripts across *nix platforms depends on commands not specified by POSIX, I don't think those books are doing their job very well.
POSIX.2 wasn't a standard until 1992. Perhaps it's the standard at fault for specifying the -E and -F flags rather than specifying the tools that existed in v7 in 1979. Authors and publishers didn't just pause for 13 years to wait and see.
scripts will be break. Strange outputs will appear. Devops will rip the gowns and douse ashes on their heads. Patches will arrive and a years later someone will make a joke about fgrep like the jokes about ed, the GNU ed line editor.
Clearly that will never be done. Outputting this to stdout would be ridiculous and break everyone and everything using it. Might as well just outright remove egrep and fgrep. Ridiculous comment.
Funny, I've never ever used egrep or fgrep in the 20 odd years I've been operating systems, always preferred explicit flags to aliases or argv[0] evil.
I get the annoyance, for both the GNU maintainers and distro maintainers/sysadmins. I'm not too wedded to either outcome, but my take is that we should avoid global mutable state:
- If you're relying on random globals (like the path /usr/bin/egrep) to (a) exist and (b) behave in a certain way, then don't mutate them. Stick with known-good versions, and treat updates like any other code change (review, test, etc.). This is the usual case for legacy systems. In other words, don't blindly run `apt-get -y upgrade` on systems which are meant to be stable.
- Alternatively, don't use globals: install grep in its own directory (e.g. via the `--prefix` argument of `./configure; or using a sledgehammer like chroot), and treat that location as a capability (as per capability-security). Programs/scripts which need that grep must be given its location explicitly (either for direct calls, or by having it prepended to their $PATH). If you want to use an updated version elsewhere, just install that to a different location; no need to touch the existing setup.
(Shout-out to Nix for doing this via ./configure and $PATH. Runner-up prizes for chroots/jails/zones/containers/VMs/etc.)
Both of your suggestions are basically equivalent to "think before you upgrade" and/or "don't upgrade". This is the second elephant in the room with all these distros that "don't use globals" and/or statically link everything (or do an analogue to that, like nix). There's very little benefit for desktops. So the upgrade to dependency X breaks component Y, and you are forced not to update X, or at least, Y's copy of X. Great. What do you do now? Swim away from upstream? Stay on outdated components? The situation is as unatenable long-term as it is on a regular distro...
The first elephant in the room is that generally you _do_ want most dependencies to be global state. Is a situation where every program is using its own version of the graphics toolkit, with different theming issues or even different themes altogether, really ideal for a desktop situation? What about libevent -- so that your mouse's wheel has one acceleration curve in some programs and some other speed in some other programs ? Or what about ibus, where generally having different client versions running simultaneously means your entire input system stops working ?
Even grep is likely something that you'd prefer to be global state, lest the grep exec() by a Python interpreter have different features than the one launched by your main shell.
> The situation is as unatenable long-term as it is on a regular distro.
I may be misremembering, but isn't this pretty much the default user experience on rolling distributions like Arch Linux? -- You consult the wiki to see what changes you need to make are (or maybe see what breaks after an update), make the change, and get on with your desktop experience.
Idk, but Arch Linux is a bad model, in my experience. I had inherited a system that ran Arch Linux, but hadn't been updated in a while. Imagine my surprise when I needed to update something, and couldn't. A part of the upgrade path (or whatever you want to call it) had been removed. So now I have an Arch Linux system that hasn't been updated for way too long, and we need to consider abandoning the product that runs on that machine, or invest in porting it to some modern environment. It's hidden behind a firewall with access for only 2 IP addresses, so it's unlikely to get hacked, but it's most undesirable.
If you like that model, fine, but don't force it onto the whole world, unless you can commit more resources to it than to Arch Linux, and basically keep all upgrade paths alive forever.
> If you're relying on random globals (like the path /usr/bin/egrep) to (a) exist and (b) behave in a certain way, then don't mutate them.
It's a bit old now, but one of the principles of Twelve Factor Applications is to vendor-in all of your dependencies.[1]
> A twelve-factor app never relies on implicit existence of system-wide packages. It declares all dependencies, completely and exactly, via a dependency declaration manifest. Furthermore, it uses a dependency isolation tool during execution to ensure that no implicit dependencies “leak in” from the surrounding system. The full and explicit dependency specification is applied uniformly to both production and development.
This scenario, with `fgrep` and `egrep` releasing a potentially breaking change, is exactly why this principle exists. If your software depends on "whatever fgrep happens to be lying around at the moment", your application might break the next time you build and deploy a new image. If you're pinned to a specific version, however, you're protected.
You're protected, but it also means you can never upgrade anything. An application can never know what future versions of a dependency it can work with (because a new version might contain a breaking change), so it will always require versions that existed at the time of its own creation.
But those versions might contain vulnerabilities, and the new version might fix those while being fully compatible in every other way, in which case you really do want the upgrade.
Those dependencies should really declare when they change existing behaviour of the previous versions, and only then should your application refuse the upgrade.
No idea if there is a system that works that way; I can imagine it could get hideously complex.
> You're protected, but it also means you can never upgrade anything.
You're protected, and it means you have to test your system _before_ upgrading.
Also, I think I'm working with a much broader definition of what "a dependency" is than you are. I consider `Alpine Linux 3.16.2` to be an atomic thing; I don't try to track the state of every single binary that comes along for the ride.
But if I depend on something to be executable from my application, like `ffmpeg 5.1.2` or whatever, I _would_ version lock that, and install it myself as part of the build script.
Then, when `Alpine 3.17` or `ffmpeg 5.2` comes out, I can bump to those versions, run our integration tests, and verify nothing broke.
> You're protected, but it also means you can never upgrade anything.
No, it just means that:
- Upgrading a dependency is a change, which should be treated like a code change (review, testing, etc.). I stated as much above.
- If, for some reason, you're stuck using an old version, that only affects the relevant part of the system. For example, scripts which rely on some old behaviour can remain pinned to that version; whilst other parts of the system can use an upgraded dependency. Also, just as importantly, parts of the system which don't need something (like grep) have no access to any version of it.
I went over to Nix cold turkey for my new work laptop. Took about a week or two, to get it where I wanted it. The real issue I hit is that Nix is so alien to working with "normal" unix stuff... like stuff that assumes /usr/bin/grep exists etc, I had issues, with the existing code-base I was hired to work on.
So, I went back to Fedora. But I'd goto Nix again in a heartbeat. It is a great system, and if you start a company with it day 1... there will be a bit of heartburn as you start, but I can't see regretting it.
This reminds me of the last time daylight savings time was changed (in the US). A bunch of hassle and software breakage.. for what? Just to get back to the same place we already were (working software)? What a waste.
I think it's a bit silly to get rid of such established aliases. Surely they aren't a real maintenance burden.
Though I can't say I entirely hate it, for the (admittedly absurd) reason that egrep = grep -E, fgrep = grep -F, but pgrep != grep -P! It's an awkward incongruity that's easy to get tripped up on occasionally.
While I hated the decision of adding warnings without much notice (which, in the case of such widely used CLI tools, is the equivalent of a breaking change), I also found an easy solution that would prevent my scripts from spitting out lots of unneeded warnings.
alias egrep='grep -E'
alias fgrep='grep -F'
Now GNU developers can keep doing whatever they're doing, and I can keep doing whatever I used to do.
> While I hated the decision of adding warnings without much notice (which, in the case of such widely used CLI tools, is the equivalent of a breaking change)
15 years isn't “much notice”? I had already stopped using those back then because, as noted in the article, they weren't standardized and so you had to work about portability across Unix installations.
It's also worth noting that this is only a breaking change if you are using the non-standard names in a context where you are trapping output. For the vast majority of people using a shell script which doesn't use the common name, they will at some point upgrade, see the warning, spend 30 seconds making the change, and never think about it again. If you're that sensitive to the extra work, presumably you also do some testing before installing new upstream releases.
EDIT: it was actually 17 years ago that the warning was added about egrep/fgrep:
I’ve been using grep -E for a long time, I thought I remember it being from a warning from egrep or something.. can’t really remember for sure, though.
Either way, I don’t see what the big deal is, just add an
alias egrep='grep -E'
If you’re worried about your non-interactive shell scripts,
shopt -s expand_aliases; alias egrep='grep -E'
and you can move on with your life.
EDIT: I must’ve been warned by shellcheck
EDIT2: here’s another one liner with one caveat being updates (also needs to be run as root)
EDIT3: just realized for most this will only ever come up in scripts because many distros already add alias egrep='grep -E' to your shell aliases (~/.bash_aliases, ~/.zshrc, etc). Thus you may only need the shopt -s expand_aliases.
The problem is that we don't just live on our own machines. We hop around to different boxes and just being able to type what you are used to matters. My dotfiles and other tools don't go or even work everywhere.
We shouldn't change the oldest parts of our OSes without a really good reason.
I find this reasoning interesting since it's why I _stopped_ using egrep/fgrep around the turn of the century: because those aren't standard, it wasn't uncommon to find that you depended on some behaviour which wasn't available in the version installed on some random server but it worked when you used grep.
I don't remember why I stopped using the shorter `egrep` and only use `grep -E`, but I suspect this was the reason. I used to work on a variety of BSD and Linux servers.
I'm fine with this clean-up and simplification. Eventually, there will be more future users of `grep` than past users.
Yeah, I worked with enough older systems (SunOS, Solaris, AIX, HPUX, FreeBSD/NetBSD/OpenBSD, etc.) that I don't remember which ones caused me to do that, either. I'm really glad it's no longer common to have things like hand-compiled installs of GNU utilities, especially since not every sysadmin was diligent about updating all of them.
I switched to just grep, but it's still part of the same line of thinking. I really, really, really dislike it when something that used to be reliable changes or is removed. I understand and accept that things change, but when it's as simple as a symlink or what have you I really do not see the cost-benefit tradeoff for sunsetting something.
It’s a matter of principle. If you wrote a script yesterday (or a year ago) and didn’t include that line, you need to re-release it today. That’s stupid.
Also, you’re being very sanctimonious about this but what if you didn’t see this HN story. Would you know to do that for the next script you distribute?
Frankly I can count on two hands the number of times I’ve used egrep in my life (same for grep -E). As long as I can remember I’ve relied on sed, awk, or find -regex for filtering output with regex. I only say this because I can’t relate to those that are upset about the change.
That said, to answer your questions directly: hopefully you aren’t auto updating anywhere in production or important, so this won’t effect anything until the box has grep 3.8. Ubuntu 20.04 uses grep 3.4, for example.
And the warning prints to stderr, so honestly, I'm having a difficult time seeing this being an actual problem for more than 0.1% of users.
I’m not unsympathetic to those that it adversely affects, but I genuinely haven’t seen anyone point out any severe consequences of the change. I’m not saying it’s impossible, if someone has a real example (sorry, saying you might get a page on the weekend for a warning printed to stderr is a stretch), I’m all ears.
Everyone has their own story, and one’s person’s experience can be very different from another person’s experience. I used egrep a whole lot, dozens of times for the automated test setup I have for my open source project. I had to spend most of an hour this morning updating that code to no longer use egrep—a non-trivial task. Here’s the amount of hassle breaking egrep has given me:
This is just one open source project. I’ve seen fgrep in use for well over 25 years, back on the SunOS boxes we used at the time. egrep apparently has been around for a very long time too. Just because it didn’t get enshrined in a Posix document—OK, according to Paul Eggert it was made obsolete by Posix in 1992, but apparently no one got the telegram and it’s been a part of Linux since the beginning and is also a part of busybox—doesn’t mean it’s something which should be removed.
I’m just glad I caught this thread and was able to “update” my code.
fgrep and egrep have been indicated as obsolescent by the Single Unix Specification and POSIX since the late 1980's.
The man page for GNU grep says:
In addition, the variant programs egrep, fgrep and rgrep are the same
as grep -E, grep -F, and grep -r, respectively. These variants are
deprecated, but are provided for backward compatibility.
"Deprecated" should be read as "maybe ain't gonna be around one day". "Use interactively, or in your personal setup, but don't put into shipping code".
So if you wrote a script yesterday with these things, you're just one of those people who don't look into specs or man pages.
The scripts of coders who don't read documentation are going to tend not to be portable or reliable anyway. That doesn't mean we should break things left and right to trip people up, but in this case the writing has been on the wall for a very long time.
I don't think aliases help in one common case listed in the article. That is, scripts run out of cron, that don't read initialization files. And that case will email on unexpected stderr writes.
Python has it's own problems. Not the least of which being it being hundreds of times slower than coreutils.
I'll take learning Shell and a handful of coreutils, over praying the sysadmin has successfully navigated/allowed me to navigate all the Python footguns.
Also, good luck doing anything embedded where space is at a premium. Not all base systems have/need Python. Everything needs a shell, however.
Hehe, wasn’t the Python 2 to 3 incompatibility debacle a thousand times worse than [fe]grep? Python2 is still lingering in places even after Python 2 was removed.
" there's a difference between avoiding fossilization and the kind of minimal, mathematical purity that we see GNU Grep trying to impose here. Unix has long since passed the point where it had that sort of minimalism in the standard commands. Modern Unix has all sorts of duplications and flourishes that aren't strictly necessary, and for good reasons."
Well, really, the type of purity many of the original Unix hands desired was that each tool did one thing well, not that one tool did everything well. Some folks would say that putting the -E and -F flags in GNU grep in the first place instead of using egrep and fgrep was the wrong direction. Now GNU is wanting to further consolidate the hydra.
What’s wrong with removing the commands and adding shell aliases instead? That sounds perfectly reasonable to me.
eg. In bash, they can be expressed as:
alias fgrep=grep -F
alias egrep=egrep -E
This sounds like a push for purity - similar to what happened in Python 3 with the move from print “xyz” (special keyword) to print(“xyz”) (standard function).
The new function requires three additional keystrokes every time it is used.
I cannot edit anymore, so I add this:
This is the case in a distribution (Arch) that has effectively taken the maintainer's advice into account, not the reverse.
I do wish that there was a standard pipeline for warnings, so stuff like this didn't have to go into error (bad) or output (worse). Powershell has such a feature but of course since it's not at the OS/Posix level then support is spotty.
Why is it bad for it to go stderr? It's supposed to be used for all kinds of app meta-output, not just errors. If you want to actually check if the command failed, that's what exit codes are for.
I did run into some Node.js code that assumed that anything printed out to stderr is a fatal error - but that's just people making wrong assumptions, not using the interface as intended.
353 comments
[ 2.9 ms ] story [ 289 ms ] threadI'm usually in favour of having one way to do things but in this case, with this much legacy it just doesn't seem worth it.
If anyone thinks it is a big enough problem that they want to fork the software they can, or the distros can maintain their own symlinks. But I think in this case the simple answer is if the maintainer doesn't want it in the source tarball then it isn't going to be there and that is more than sufficient a justification.
Complaining is probably more reasonable than asking for a justification here.
Yes, but the convenience of... creating 2 symlinks and adding the flag based on argv[0]?
Sorry, I don't buy it. Wanting to die on such a tiny hill has GNU written all over it though
At least we used to pay lip service to that ideal.
That said, I certainly wouldn't have put in this change myself, because I wouldn't like to inconvenience anyone. But that's just basic good manners that you should have in every-day life to random strangers.
But like I already said in my previous comment, I wouldn't have made this change myself. I actually strongly disagree with it. But I can also accept that other people have a different attitude, and that's okay too, even if I personally don't really care much for the particular attitude.
Funny you mention OpenBSD, because OpenBSD is very much "by the developers, for the developers" and has a fairly decent "fuck off" attitude once people start making demands (which I don't think is a negative per se).
I still feel you have an obligation to transfer to someone willing to shoulder the responsibility if this happens, an obligation incurred by publishing it in the first place.
Though, I don't feel too strongly about this.
Even when taking over maintainership for popular packages later on, people often aren't exactly breaking down the door for it. So my philosophy is simple: "if you do the work, you get to decide". I may like or dislike these decisions, and at times I may even rant about how stupid a certain decision is, but in the end ... the people doing the work get to decide. The alternative of being beholden to a vaguely defined "community" of armchair quarterbacks is much worse, IMO.
Distros are free to've replaced the symlinks with wrapper scripts in the meantime. Granted, I expect Arch will go ahead & let the symlinks disappear. But I can't imagine what you think of Arch's stewardship
I expect this should have about as little impact as the usr merge many distros have gone through
The original deprecation seems ... petty? Not sure why it's a priority, but whatever.
I know there are scripts still in use that I wrote more than a decade ago that might do weird things now and I wish the maintainers best of luck.
Are the stewards being provided fair compensation? How do we even talk about what is fair compensation when non-monetary compensation has become difficult to even discuss (often due to past instances being extreme disproportionate or of a form that is no longer tolerable).
The simplest way to put it is that if you can't find a steward you aren't paying enough and trying to use appeals to morals or ethics to get people to accept lower pay no longer holds as much weight when being moral or ethical no longer provides the same level of non-monetary benefits.
Oh, most definitely. The anger and vitriol I see directed towards maintainers, or volunteers of any type, who are "stepping back for personal reasons" is horrifying.
My father was a very active volunteer in his community ... the number of people who were mad at him when he stepped away after his heart problems was startling. Conversely, the number of people who volunteered to help him and my mother with shopping etc. when COVID hit was heart warming.
When was that? For as long as I can recall, a core mantra of free/libre software has been that it was provided "AS IS" without warranty of any kind. Decades ago the dominant response I recall was one of gratitude and a little amazement that ad-hoc communities of volunteers were making real software that wasn't just academic but on par with commercial offerings. Some of those communities chose to adopt a user-friendly posture because they wanted people to like them but plenty did not and just did their own thing. As long as they could attract contributors they kept going.
Personally what concerns me is this growing expectation that volunteering to maintain an open source project also means you are "morally or ethically beholden or responsible" to anyone who uses it. In practice that seems to mean maintainers must respond to user requests or end up on the receiving end of a great deal of vitriol. It's no wonder so many volunteer maintainers who have internalized this responsibility are burning out, and how many more potential maintainers are dissuaded by seeing what is happening to the current maintainers.
I remember when I was younger, every volunteering experience I had was a delight; people thanking me for my time, getting me free coffee "just because", etc. Some of my more recent volunteering experiences have been less pleasant.
There is no reasonable explanation for this decision except that somebody thought that having both egrep and grep -E was "ugly" according to their own personal sensibilities.
E.g. I can't count the number of times I have submitted _the_ _same_ _GUI_ _fixes_ to certain popular browser over the decades. Because apparently "they" have to rewrite the chrome of the mobile version from scratch every handful of years. "They" is in quotes because it's, in fairness, never the same person or even the same group of people. It's a CADT.
And in this case, to remove what apparently are two small shell scripts which for sure cost more to remove than to preserve....
In which case, time to fork it.
I don't fucking care who you are, you do not break grep and keep a privileged position on my machine.
Conflicts: grep
Provides: grep
Old programs have stupidly complex argument parsing. If you can pass your arguments as "cmd foo bar path", "cmd -f bar path", "cmd -fbar path", "cmd path --foo=bar" or "fcmd bar path" and "cmdf path bar", it can be really convenient for users who can structure their commands in the way that makes the most sense for them.
But it can be really frustrating to maintainers who are maintaining -- and testing! -- a thousand lines of bespoke argument parsing for a ten line function.
It's really, really tempting to define a simple syntax for argument parsing, turn it into a library, and reduce your pile of shell commands to a few lines of argument configuration and a function call.
There’s no parsing. GNU’s egrep and fgrep are trivial shell scripts:
Because they're the same thing. What all is about are the `xgrep` commands being symlinks to `grep`. Though I guess you can have packages that just add the symlink.
>If some distro wants to include them with a warning that's also fair game.
Some distros already do what is recommended in release notes. Rather being symlinks they're wrapper scripts. E.g. in Nix the `fgrep` and `egrep` are just `exec ${nixpkgs.gnugrep}/bin/grep -F "$@"` and `-E` respectively.
https://git.savannah.gnu.org/cgit/grep.git/tree/src/egrep.sh
Such an insanity...
They are wrapper scripts that precisely enable to obey to the advice the warning is giving, yet the warning ends up advising not using the wrapper...
So I was wrong, and let's not be unfair to the maintainer.
edit: But was not always wrong. Just terribly outdated. Searching the log I found the commit [5cb71b0] with the message:
which made the change from creating symlinks to creating scripts. The code continued to adjust behavior according filename (in contrast to what someone would expect based on the commit message). Then few years afterwards in [d25bebd] the scripts and the symlink behavior were dropped for actual binaries, with in-source comment: It also funnily added the following prints, quite similar to what they're doing now: The scripts returned about a decade later (or else few years ago) in [b639643]. The commit message mentioned the reasoning: [5cb71b0]: https://git.savannah.gnu.org/cgit/grep.git/commit/?id=5cb71b...[d25bebd]: https://git.savannah.gnu.org/cgit/grep.git/commit/?id=d25beb...
[b639643]: https://git.savannah.gnu.org/cgit/grep.git/commit/?id=b63964...
- Thousands of users have to update their scripts, habits, shell configs; or
- The GNU Grep maintainers spend essentially zero minutes "maintaining" a few lines of code to automatically use -E or -F based on argv[0].
It seems to me the second is obviously the better option.
EDIT: see other comment in this thread from FreeBSD user that on FreeBSD grep, egrep, and fgrep are all separate but identical (copies) of the same file. So this isn’t quite such a silly solution as some might think.
Which has lower cognitive load if everyone starts doing it?
Explicitly specifying switches, or argv magic? I'd argue, the switches are. In the abscence of the symlinking, that is how the tool functionality would have to be driven anyway.
Argv magic now runs into a problem if another program sharing the name ever comes into existence on the path. It's also completely unergonomic in a sense, because short of looking at the source, you have no way of knowing what argv transforms implementations support are, and in order to use them, you must explicitly pollute the Symbol namespace with a denormalized util. Also, the argv magic does require one extra shell to do the transform from !grep to grep -!, Which is technically more overhead. On the other hand, fgrep and egrep are ironically easier to grep/sed for as opposed to grep/ -[F|E]
One tool, one manual, one name, one argv0.
I am not the Grand Poobah of the Internet, however, even if their hat is in my possession, so I understand that it is likely that the fgrep/egrep convention is probably deeply entrenched, and likely to spawn a new holy war on par with Emacs/Vim. Tabs/spaces, etc...
As far as I can tell this is the entire maintenance burden:
This is the case in a distribution (Arch) that has effectively taken the maintainer's advice into account, not the reverse.
Isn't 15 years more than enough time to handle the deprecation?
I’m just joking :)
I am casual user so I can get around system but modern ways always surprise me when I finally find out about it.
So ... people will stick with what works, like "egrep".
[1] - https://www.shellcheck.net/
Unfortunately (or not?) I tend to use actual programming languages to make my tools rather than doing shell scripting, so I don't have anything interesting to put in there, but I'll keep a bookmark of it around.
Though I've not used either since uni over two decades ago¹ on a somewhat off-standard Unix a few of the machines ran, so until this thread I can't say I was remembering that they existed at all.
----
[1] using grep -E when needed since²
[2] and presumably grep -F too though I don't remember ever actually doing that
https://medium.com/@rualthanzauva/grep-was-a-private-command...
What actually happened here is that POSIX skipped it. It never entered a standard, even though it was (literally!) in every OS and available to everyone. But no one cared that it wasn't in some arbitrary standard, because it was always there. For half a century!
I dare say that "fgrep" and "egrep" have more active users (both interactive and scripted) than "awk" or "ed" or "od" or "bc", all of which are still around.
On the first point the author only gives hypothetical examples. I feel the argument might have been more compelling if we could see some concrete examples of things that break with GNU Grep 3.8.
As for the second point, I find it less convincing than the first one. If it's just the muscle memory then an alias would be an acceptable workaround. And I doubt that "everyone in the world" would want such an alias, as the author suggests.
I haven’t done anything with fgrep and egrep before but piping grep into another grep for more complex classes of text search is something i use a lot.
Should we do better? Absolutely! Will this burn people if vendors don’t take care? Also absolutely!
Your script ostensibly handles (at least logs) errors and warnings right? Do you exhaustively handle every single error and warning in a unique and different way or do you have a catchall "If non-0 return code then fail"? How does introducing new output to stderr affect that?
This is something that should be caught before it gets to that point SPECIFICALLY so you aren't getting woken up in the middle of the night.
Your infrastructure probably is crap, as very few people get to build it themselves from scratch. That does not mean one should cheerfully accept additional unnecessary or pedantic complications.
It would also be prudent to investigate each and every change to any of the software you use, in order to anticipate problems, but unnecessary and pedantic changes increase the burden there, as well.
your scripts will continue to produce the expected output. the side effects, otoh, will change indeed.
Everything about this, even this comment I'm writing right now, is a waste of time.
When a change has little or no objective benefit, I feel the burden of demonstrating that it is harmless falls on those making the change.
As has been pointed out elsewhere, this is free software and the maintainers are free to do whatever they like. That does not stop others having an opinion about it, especially when it is in the form of constructive criticism.
It's rather amusing how you have flipped from saying "you can't prove a negative" to an argument for the certainty of observable effects and the probability of consequences! (you wrote might cause problems, but everyone can see that's an unrealistic understatement of the implications of the argument you are using.)
The NASA managers prior to the Challenger crash thought that what they really wanted was something showing them an actual problem existed. Erring on the side of caution is generally prudent, even in relatively small matters.
If nobody can show an actual existing problem, or even an example of reasonable code someone could have written which would be impacted by a the printed warning, then yes, I would think that I was charitable when I wrote “might cause problems”.
If we could prove there would be no downside, then plausible problems could be ignored as merely hypothetical, and there would be no need to posit offsetting benefits. The question the GNU grep developers might want to consider is whether the supposed upside will have sufficient material consequences for their purposes.
Does that code look like something you'd log into a system and manually run on a regular basis? Does it maybe instead look like one layer of a legacy automation stack absorbed into other tools?
I have scripts everywhere, some of them 20 years old or more, that use fgrep. For years and years it was a "best practice" thing if you were checking for a fixed string (so that you didn't accidentally match on a "." or whatever by forgetting it was a regex).
Any of these options seem reasonable to me.
I really don't think people understand the impact here. It's not it's just a bunch of angry geriatric graybeards yelling at the modern world. It's that there is decades of uncounted, unrecognized, untraceable software written using these old conventions that are suddenly changing.
It's just a terrible idea. Linux cares about conforming to syscall interfaces for binaries compiled 20 years ago, but somehow you think it's OK to break scripts that have worked fine for 50 (fifty!) years?
And don’t exaggerate. This won’t, in all likelihood, “break” your scripts.
Previously:
> The first option is to simply replace "fgrep" with "grep -F" everywhere in all your scripts, which is correct but is more work than your other option, which is to add your own "fgrep" script somewhere in your path.
seems like evidence of a broken script to me. The fact that it can be fixed doesn't make it not broken.[1] At least, nothing that isn't sensitive to junk on stderr -- bourne scripts are not always as tolerant as you'd like.
For now.
- If you're relying on random globals (like the path /usr/bin/egrep) to (a) exist and (b) behave in a certain way, then don't mutate them. Stick with known-good versions, and treat updates like any other code change (review, test, etc.). This is the usual case for legacy systems. In other words, don't blindly run `apt-get -y upgrade` on systems which are meant to be stable.
- Alternatively, don't use globals: install grep in its own directory (e.g. via the `--prefix` argument of `./configure; or using a sledgehammer like chroot), and treat that location as a capability (as per capability-security). Programs/scripts which need that grep must be given its location explicitly (either for direct calls, or by having it prepended to their $PATH). If you want to use an updated version elsewhere, just install that to a different location; no need to touch the existing setup.
(Shout-out to Nix for doing this via ./configure and $PATH. Runner-up prizes for chroots/jails/zones/containers/VMs/etc.)
The first elephant in the room is that generally you _do_ want most dependencies to be global state. Is a situation where every program is using its own version of the graphics toolkit, with different theming issues or even different themes altogether, really ideal for a desktop situation? What about libevent -- so that your mouse's wheel has one acceleration curve in some programs and some other speed in some other programs ? Or what about ibus, where generally having different client versions running simultaneously means your entire input system stops working ?
Even grep is likely something that you'd prefer to be global state, lest the grep exec() by a Python interpreter have different features than the one launched by your main shell.
I may be misremembering, but isn't this pretty much the default user experience on rolling distributions like Arch Linux? -- You consult the wiki to see what changes you need to make are (or maybe see what breaks after an update), make the change, and get on with your desktop experience.
If you like that model, fine, but don't force it onto the whole world, unless you can commit more resources to it than to Arch Linux, and basically keep all upgrade paths alive forever.
It's a bit old now, but one of the principles of Twelve Factor Applications is to vendor-in all of your dependencies.[1]
> A twelve-factor app never relies on implicit existence of system-wide packages. It declares all dependencies, completely and exactly, via a dependency declaration manifest. Furthermore, it uses a dependency isolation tool during execution to ensure that no implicit dependencies “leak in” from the surrounding system. The full and explicit dependency specification is applied uniformly to both production and development.
This scenario, with `fgrep` and `egrep` releasing a potentially breaking change, is exactly why this principle exists. If your software depends on "whatever fgrep happens to be lying around at the moment", your application might break the next time you build and deploy a new image. If you're pinned to a specific version, however, you're protected.
[1] https://12factor.net/dependencies
But those versions might contain vulnerabilities, and the new version might fix those while being fully compatible in every other way, in which case you really do want the upgrade.
Those dependencies should really declare when they change existing behaviour of the previous versions, and only then should your application refuse the upgrade.
No idea if there is a system that works that way; I can imagine it could get hideously complex.
You're protected, and it means you have to test your system _before_ upgrading.
Also, I think I'm working with a much broader definition of what "a dependency" is than you are. I consider `Alpine Linux 3.16.2` to be an atomic thing; I don't try to track the state of every single binary that comes along for the ride.
But if I depend on something to be executable from my application, like `ffmpeg 5.1.2` or whatever, I _would_ version lock that, and install it myself as part of the build script.
Then, when `Alpine 3.17` or `ffmpeg 5.2` comes out, I can bump to those versions, run our integration tests, and verify nothing broke.
No, it just means that:
- Upgrading a dependency is a change, which should be treated like a code change (review, testing, etc.). I stated as much above.
- If, for some reason, you're stuck using an old version, that only affects the relevant part of the system. For example, scripts which rely on some old behaviour can remain pinned to that version; whilst other parts of the system can use an upgraded dependency. Also, just as importantly, parts of the system which don't need something (like grep) have no access to any version of it.
When you can get Nix to do what you want, it's wonderful.
When you hit a barrier (e.g. a program you want isn't packaged), that's when Nix is very hard to use, and requires learning.
So, I went back to Fedora. But I'd goto Nix again in a heartbeat. It is a great system, and if you start a company with it day 1... there will be a bit of heartburn as you start, but I can't see regretting it.
And I'm going out on a limb here...
Don't support user hostile maintainers who force breaking changes to a decades old API just because they felt like it.
Though I can't say I entirely hate it, for the (admittedly absurd) reason that egrep = grep -E, fgrep = grep -F, but pgrep != grep -P! It's an awkward incongruity that's easy to get tripped up on occasionally.
alias egrep='grep -E' alias fgrep='grep -F'
Now GNU developers can keep doing whatever they're doing, and I can keep doing whatever I used to do.
15 years isn't “much notice”? I had already stopped using those back then because, as noted in the article, they weren't standardized and so you had to work about portability across Unix installations.
It's also worth noting that this is only a breaking change if you are using the non-standard names in a context where you are trapping output. For the vast majority of people using a shell script which doesn't use the common name, they will at some point upgrade, see the warning, spend 30 seconds making the change, and never think about it again. If you're that sensitive to the extra work, presumably you also do some testing before installing new upstream releases.
EDIT: it was actually 17 years ago that the warning was added about egrep/fgrep:
https://git.savannah.gnu.org/cgit/grep.git/commit/?id=0b4859...
I've resorted to right-clicking a command window and using findstr. =(
I'm sure there's also a powershell equiv but I haven't taken the time to walk the PS object tree to find it.
Either way, I don’t see what the big deal is, just add an
If you’re worried about your non-interactive shell scripts, and you can move on with your life.EDIT: I must’ve been warned by shellcheck
EDIT2: here’s another one liner with one caveat being updates (also needs to be run as root)
EDIT3: just realized for most this will only ever come up in scripts because many distros already add alias egrep='grep -E' to your shell aliases (~/.bash_aliases, ~/.zshrc, etc). Thus you may only need the shopt -s expand_aliases.We shouldn't change the oldest parts of our OSes without a really good reason.
I'm fine with this clean-up and simplification. Eventually, there will be more future users of `grep` than past users.
Also, you’re being very sanctimonious about this but what if you didn’t see this HN story. Would you know to do that for the next script you distribute?
That said, to answer your questions directly: hopefully you aren’t auto updating anywhere in production or important, so this won’t effect anything until the box has grep 3.8. Ubuntu 20.04 uses grep 3.4, for example.
And the warning prints to stderr, so honestly, I'm having a difficult time seeing this being an actual problem for more than 0.1% of users.
I’m not unsympathetic to those that it adversely affects, but I genuinely haven’t seen anyone point out any severe consequences of the change. I’m not saying it’s impossible, if someone has a real example (sorry, saying you might get a page on the weekend for a warning printed to stderr is a stretch), I’m all ears.
Emphasis on "I", which says strictly nothing of the rest of CLI users out there.
We're very glad to learn that it is not a problem for you, but it brings very little to the conversation.
Worth reading the guidelines [0].
[0] https://news.ycombinator.com/newsguidelines.html
https://github.com/samboy/MaraDNS/commit/afc9d1800f3a641bdf1...
This is just one open source project. I’ve seen fgrep in use for well over 25 years, back on the SunOS boxes we used at the time. egrep apparently has been around for a very long time too. Just because it didn’t get enshrined in a Posix document—OK, according to Paul Eggert it was made obsolete by Posix in 1992, but apparently no one got the telegram and it’s been a part of Linux since the beginning and is also a part of busybox—doesn’t mean it’s something which should be removed.
I’m just glad I caught this thread and was able to “update” my code.
The man page for GNU grep says:
"Deprecated" should be read as "maybe ain't gonna be around one day". "Use interactively, or in your personal setup, but don't put into shipping code".So if you wrote a script yesterday with these things, you're just one of those people who don't look into specs or man pages.
The scripts of coders who don't read documentation are going to tend not to be portable or reliable anyway. That doesn't mean we should break things left and right to trip people up, but in this case the writing has been on the wall for a very long time.
I'll take learning Shell and a handful of coreutils, over praying the sysadmin has successfully navigated/allowed me to navigate all the Python footguns.
Also, good luck doing anything embedded where space is at a premium. Not all base systems have/need Python. Everything needs a shell, however.
Well, really, the type of purity many of the original Unix hands desired was that each tool did one thing well, not that one tool did everything well. Some folks would say that putting the -E and -F flags in GNU grep in the first place instead of using egrep and fgrep was the wrong direction. Now GNU is wanting to further consolidate the hydra.
eg. In bash, they can be expressed as:
alias fgrep=grep -F
alias egrep=egrep -E
This sounds like a push for purity - similar to what happened in Python 3 with the move from print “xyz” (special keyword) to print(“xyz”) (standard function).
The new function requires three additional keystrokes every time it is used.
Then one could simply do a `ln -s grep egrep` and be done with it.
Thanks for pointing it out though, I was not aware!
I did run into some Node.js code that assumed that anything printed out to stderr is a fatal error - but that's just people making wrong assumptions, not using the interface as intended.