Your reading of the title is grammatically correct. "written in Rust" is a modifier, and modifiers apply to the noun that comes just before it -- in this case, "existing software". The title implies that existing software written in Rust should be replaced.
Why? Most of these "existing software" are tried and tested tools that have stood the test of time. Why rewrite them and introduce potential bugs? Write something new!
Most of this are not actual rewrites, but "write-a-news", so they are not drop in replacements. Like ripgrep for example: it servers the same task, but is in some ways improvement over "tried and tested" grep.
Because they can be rewritten by removing legacy cruft, in a language that prevents the numerous security issues they had (and still have), and optimizing several things besides...
Does anybody miss Sendmail? That had also "stood the test of time".
Sendmail never stood the test of time and is pretty much abandoned now, replaced with Postfix. Guess in what language Postfix was written...
The idea is that language will not solve poorly designed software, but a developer that knows what he is doing like Wietse did with Postix. And Postfix stood the test of time pretty well, considering how complex it is.
Caveat: Saying “Just write it in rust” won’t change the fact that writing secure software is hard.
The only thing that will actually prevent security holes is attentive software engineers deliberately choosing to learn tools and practice habits to avoid security holes by design. The professional judgement of a lot of those engineers is leading them to choose to learn rust right now.
> Caveat: Saying “Just write it in rust” won’t change the fact that writing secure software is hard.
Absolutely. It's just that statistically the memory safety bugs have been the dominant percent of all the bugs, so rewriting core tools in a memory-safe language does make sense.
And those are bugs from from people writing core stuff, like you know, the people who should know Rust better than you and me. People writing foundational code that your program will run on top of.
Of special danger is the last CVE, that was rushed and backported into the last release. But nobody talks about it. I can imagine why.
I can bet that the "but the bug happened within unsafe code" excuse will be the next Rust meme.
Then he's the exception, at least anecdotally in my career and life. Most C/C++ programmers I've known were ready to defend their code quality with their fists. They were in complete denial.
Heartbleed happen because the open source software used by everyone were left getting dust, with low maintenance and despite its popularity, nobody was paying or donating nothing to keep the project going.
Even if Rust was invented in the 80's, and such a project was in it, the same could happen in one of those unsafe{} blocks a Rust project like this might need to use (and we are forgetting all the ordinary bugs even in the safe portions of the code).
Sure, being in C might be a part of the problem, and this is a good space for tools like Rust to occupy more space, but there's a bigger picture or else Linux would be breaking all the time, and its a solid, big and complex piece of software that works pretty well because there's a lot of economical incentives to keep it going.
> with low maintenance and despite its popularity, nobody was paying or donating nothing to keep the project going.*
There probably is a Wikipedia page describing this phenomena but I'll settle for calling it "the commodity effect": when something is very widely used then everybody assumes it has been improved to perfection and that it has no flaws (this is not exclusive to software engineering, it happens in many areas of life).
> the same could happen in one of those unsafe{} blocks a Rust project like this might need to use
Sure but they are much easier to search for and audit (like with e.g. `cargo-geiger`). That by itself is a win. Whereas C/C++ needs a special class of linters and scanners, a good chunk of which are paid (a deal-breaker for many software devs not because they're poor but because they don't want to pay for them out of their pocket, and their employer is blind to see the added value and buy those tools for them).
> and we are forgetting all the ordinary bugs even in the safe portions of the code
Only people with agenda forget about those. I worked with Rust in 3 companies in total so far and I've only seen serious, diligent, hard-working devs with attention to detail and a slight case of efficiency mania. :) And they were very mindful of potential bugs and we wrote a lot of tests of different kinds like acceptance/integration, unit, property, and probably others.
No normal smart developer forgets that memory safety bugs are just one class of all the bugs. Turns out however that those are the most widespread bugs in general so IMO it is sensible to use a language that eliminates them by the mere virtue of compiling your program -- and not using unsafe{} in certain ways.
As another poster said, time and attention are limited resources. By removing memory safety bugs from the picture our brains are now free to pay proper attention to the more subtle bugs.
Answering to another point separately because my other comment grew large enough already:
> or else Linux would be breaking all the time, and its a solid, big and complex piece of software that works pretty well because there's a lot of economical incentives to keep it going.
It is solid... as far as we know. Truth is, nobody is fuzzing the Linux kernel 24/7 except maybe the worst possible actors like national intelligence agencies or malevolent hacking groups -- and they of course would never share and help fix a zero-day because it can help them make money by exfiltrating sensitive information and/or breaking in protected networks.
Don't get me wrong, I admire the Linux kernel devs. They are a standing tribute to all ideas of free and open source and open project management. But Linus himself said security isn't the first priority of the kernel which already means that certain potential problems in the code are being overlooked in favor of speed and stability.
(There was a story some weeks ago on HN showing that there was a pretty nasty exploitable bug in the Linux kernel but I lost the link.)
Writing something new in rust is harder than rewriting because you need to think a lot about your memory model up front. In other languages you can leak all the memory you want and then (maybe) figure enough of it out when you've already shipped/already deployed. In Rust you need to do the up front work, so instead of releasing a leaky program, you don't release a program at all. If there is an existing program you can make assumptions more readily.
Am I the only one who doesn't care what a tool / software package is written in, provided it does the job? If a Rust port is superior then sure, I'll use it, but I won't use it because it was written in Rust.
Language per se isn't important, but the majority of security vulnerabilities are still caused by non-memory-safe languages. So I'd regard that as a reason not to use a given software package.
Half those applications are going to have other classes of new bugs simply because it’s new code and they’ve had less people audit the code. Plus some of those original tools were already written in safe languages like Haskell.
That’s not true at all. 2 of the 3 biggest security vulnerabilities I’ve had to deal with in my career were completely unrelated and wouldn’t have been prevented had software been written in Rust instead.
Also half the software mentioned in that gist should never be used in security-focused applications anyway (if your depending on ‘cat’ or ‘awk’ to be bug free for your application to be hardened then you’re already doing it wrong)
I use cat and awk in my build system. I can imagine bugs in either that I wouldn't notice in general that would compromise my application. They are off the wall enough that I don't think they have ever happened, but my application is an embedded system that has the ability to kill people so I'm a little paranoid. (not so much that I don't use awk in my build system where it is a useful tool)
My point was if you’re exposing your build system to untrusted individuals inputting untrusted data then you’ve already lost the game. Rust won’t save you from a RCE bug because you’ve already granted them RCE from the beginning.
An overflow bug in awk is only exploitable if someone can craft input into awk. And if you’re allowing people to do that then you’ve already given them access to remotely run code without them needing a bug.
> My point was if you’re exposing your build system to untrusted individuals inputting untrusted data then you’ve already lost the game.
This is how the vast majority of build systems are configured. If I have owned a dev and can push code to a branch I can likely execute code in their build environment. If they then parse that build output with something like awk, that's attack surface.
Is it the first thing I'd try to attack? Nope. But I do wonder what the cumulative impact of every binary on your system being memory safe would be. I'd definitely feel better.
> If I have owned a dev and can push code to a branch I can likely execute code in their build environment.
But then you’re also executing that code as a trusted user. Ie the real hack is owning the dev.
> This is how the vast majority of build systems are configured.
No. The vast majority of build systems are either locked to private repositories, and/or a subset of developers, and/or run in an ephemeral sandbox.
> But I do wonder what the cumulative impact of every binary on your system being memory safe would be.
I’d already answered that. It’s negligible because if someone can exploit a buffer overflow in awk then they can already run arbitrary code in awk (remember, awk is a programming language). So if you’re not already taking precautions about who is running what on your build system and how running code is sandboxes then it’s not going to take a buffer overflow bug to RCE it.
> But then you’re also executing that code as a trusted user. Ie the real hack is owning the dev.
That's like saying the real hack is sending a phishing email, ignoring all privesc, lateral movement, etc.
> No. The vast majority of build systems are either locked to private repositories, and/or a subset of developers, and/or run in an ephemeral sandbox.
I didn't say otherwise. I said that the majority allow untrusted execution. If you consider your engineers laptops to be trusted you're thinking about networks the way people did ~10+ years ago.
> It’s negligible because if someone can exploit a buffer overflow in awk then they can already run arbitrary code in awk
This isn't true, but even if it were you can just replace awk with anything else that's not turing complete.
> I said that the majority allow untrusted execution. If you consider your engineers laptops to be trusted you're thinking about networks the way people did ~10+ years ago.
i never said anything about trusting engineers laptops.
Let me put it another way, are you running your build pipelines bare metal, allowing anyone to instantiate the pipeline against a branch of code that anyone in the public has written? Or is that build pipeline running against a private repository, or triggered by trusted users, or does it run inside a temporary VM or container that gets destroyed afterwards? That’s where you harden CI/CD. I’m not suggesting fixing buffer overflows aren’t worthwhile (security should be multilayered after all) but it’s missing the real risk since you’ve already got code remotely executing. Thus your threat model should be about ensuring you minimise the blast radius of that code.
To use your colourful
language: if your build pipelines are running bare metal then you’re thinking about infra like people did ~10 years ago.
> This isn't true, but even if it were you can just replace awk with anything else that's not turing complete.
But you’re still not addressing the real issue. If you can’t trust the the code executing and you’re not sandboxing that code then you’ve already lost the fight. You can’t give unrestricted access to execute code remotely and then moan “we wouldn’t have been exploited if everything was written in Rust”. The bone headed design was allowing unrestricted remote code execution and not the buffer overflow bug in ‘cat’.
> That's like saying the real hack is sending a phishing email, ignoring all privesc, lateral movement, etc.
It’s really not. What I’m taking about is the equivalent of saying “phishing attacks are always going to happen so you need MFA et al to harden against the array of of ways one can manipulate the system”
You’re entire security model seems to be focused around fixing every fucking executable against every possible class of bug and that’s insanely impractical. Whereas I’m suggesting one shouldn’t implicitly trust remote code in build pipelines to begin with.
> 2 of the 3 biggest security vulnerabilities I’ve had to deal with in my career were completely unrelated and wouldn’t have been prevented had software been written in Rust instead.
The last study I saw was that 52% of security vulnerabilities were still basic memory safety vulnerabilities. Memory safety isn't the only thing, but it's still the biggest thing.
> (if your depending on ‘cat’ or ‘awk’ to be bug free for your application to be hardened then you’re already doing it wrong)
People run cat or awk on log files (where an attacker could easily craft particular data patterns) all the time. Maybe they shouldn't, but they do.
> The last study I saw was that 52% of security vulnerabilities were still basic memory safety vulnerabilities. Memory safety isn't the only thing, but it's still the biggest thing.
My point wasn’t that memory safety isn’t a big issue. It’s that it’s not the only cause of vulnerabilities. Assuming your 52% figure is accurate (and I’m happy to take your word on that) that still means that roughly half of all vulnerabilities are not memory safety. Which means your figure actually just reinforces my point.
> People run cat or awk on log files (where an attacker could easily craft particular data patterns) all the time. Maybe they shouldn't, but they do.
I think there’s enough obscurity there hidden from the attacker that such usage should be safe from all but the most determined of attacker. And even then, they’d likely need some other attack to probe for usage about what systems are listening to what logs and using which POSIX utils that you could argue the attacker already has the access they need without going to the pain of crafting an overflow bug in a log file.
Im not saying those bugs don’t exist nor shouldn’t be addressed, I’m just being pragmatic about their exploitability.
The bigger issue is CI/CD pipelines. Often they’re define in the same mono-repo as the source itself and they’ll have execution rights automatically because that’s literally their function as a build and test pipeline. But because said pipelines are already visible and already executable, you need to have greater controls around that pipeline to prevent abuse otherwise it’s already game over even without an overflow bug.
> Assuming your 52% figure is accurate (and I’m happy to take your word on that) that still means that roughly half of all vulnerabilities are not memory safety.
Sure, but that half is every other category of bug put together. I guess if your model is that every other class of bug will be significantly increased by a rewrite then maybe that logic makes sense, but that seems unlikely to me.
You said "Half those applications are going to have other classes of new bugs simply because it’s new code and they’ve had less people audit the code", and when I pointed out that such bugs would most likely not be security bugs you responded by saying that many security bugs aren't memory safety bugs. You may not have intended to say what you said, but what I'm disagreeing with is what you said, not a strawman, and even if you were right your condescension would still be unwarranted.
And yes, I did say that at the start of the conversation. Sorry I’d forgotten about that part because this conversation had taken a tangent on discussing other specifics in security.
I do stand by my comments, a rewrite does introduce new bugs and not all security bugs are buffer overflows. Rust doesn’t protect you against bugs like Shellshock. They protect against bugs like Heartbleed. But people frequently forget about the former when focusing on the latter. So absolutely we need to be considered when replacing battle tested software with an entirely new code. It’s not an unfair meme when people state this. If it were an easy and risk free upgrade then we’d have already done so in one of the other safer languages that predate Rust. A lot of what’s changed between then and now isn’t that languages have gotten better, it’s simply that Rust is trendier than any of the safer languages before them. And that’s a pretty awful reason to reason to rush a complete rewrite of the entire stack.
Source: I’m an old fart who’s been writing safe code in safe languages for 30+ years.
"Try reading the thread again" is extremely condescending.
> Rust doesn’t protect you against bugs like Shellshock. They protect against bugs like Heartbleed. But people frequently forget about the former when focusing on the latter. So absolutely we need to be considered when replacing battle tested software with an entirely new code. It’s not an unfair meme when people state this.
Like I said, over half of security bugs are the kind of bug that Rust protects you against, whereas the other half (apparently more like a quarter, per side thread) is every other category of bug put together. So unless your model is that every other category of bug will be significantly increased by a rewrite, the rewrite wins out on net.
> If it were an easy and risk free upgrade then we’d have already done so in one of the other safer languages that predate Rust. A lot of what’s changed between then and now isn’t that languages have gotten better, it’s simply that Rust is trendier than any of the safer languages before them.
Per another side thread, I do agree that the reason people are rewriting in Rust now when they weren't rewriting in OCaml 10 years ago is mostly trendiness. But I disagree about which of those two decisions is the mistake.
> Like I said, over half of security bugs are the kind of bug that Rust protects you against, whereas the other half (apparently more like a quarter, per side thread) is every other category of bug put together. So unless your model is that every other category of bug will be significantly increased by a rewrite, the rewrite wins out on net.
Rewrite does increase other types of bugs though. But the more specific point I’m making is that the domain matters massively here. A buffer overflow bug in ‘cat’ isn’t equivalent to a buffer overflow bug in ‘openssh’. If you’re worried about RCEs in ‘cat’ then the fix isn’t to rewrite ‘cat’, it’s to fix the calling pipeline. Eg is the containerisation service secure? Using Rust as a blunt hammer to fix all security bugs demonstrates a real lack understanding about how to build hardened systems in the first place.
This is the point I’m making. Not that Rust shouldn’t be used as part of the solution but rather that people need to stop bleating on about Rust being “the” solution. People love to quote vulnerabilities in Chrome and OpenSSL as a reason everything should be rewritten in Rust but fail to understand that coreutils are called in entitled different circumstances and thus have an entirely different security model.
By the way, your blog link needs updating in your HN profile (to github.io). Some really interesting content on there and I really enjoyed your writing style too. Thanks for publishing your thoughts :)
(The actual number is more than that, it’s roughly 70%. This number was reported by at least Microsoft and Chrome, among others. That roughly the same number has popped up across many large companies helps us gain confidence that it’s roughly correct, though obviously it’s not a law.)
The domain matters a lot here though. If you’re talking about bugs in Chrome then yeah, you’d expect it to be weighted towards buffer overflows. But if you’re taking about bugs in a build script someone has written, then I’d wager human error (given commands and have shells have a pretty esoteric syntax) are more of a problem — and certainly easier to exploit - than a buffer overflow in awk.
Don’t get me wrong, I’m all for writing stuff in safer languages (I personally haven’t touched C in years for that reason). But simply throwing Rust at the problem without assessing the risks is just as careless. In the case of most POSIX base utils, the risk isn’t buffer overflows, it’s people exposing internal tools to untrusted external users.
Take shellshock for example, that’s a prime example for just how careless people can be. And while there was patches to fix that vulnerability in Bash, people shouldn’t have been writing software that passed untrusted data to Bash in the first bloody place.
My point is: Security is a multi-layered problem. So by all means let’s pivot towards newer, safer, languages of it can remove an entire class of bugs. But let’s not pretend that rewriting the entire of POSIX in Rust is either practical nor the end of the discussion. And if we acknowledge it’s not, then we must also acknowledge there’s already things we can do to harden against our C++ utils today while those C++ utils are still a necessity.
This is why the Rust argument fucks me off. People expect too much from it and use it as an argument whenever other countermeasures should have been in place irrespective of the vulnerability that was inevitably used.
I too am not convinced that Haskell tools have to be replaced by Rust ones. In my eyes Rust's main applicability in this situation would be to replace C/C++ tools.
As for other kinds of bugs, yeah, that's inevitable, but I still prefer to have the memory safety bugs eliminated from the get go so I'd still say it's a worthy sacrifice to walk the road to safe tools one more time just for the memory safety.
So yeah, no need to rewrite tools in already memory safe languages but when it comes to C/C++ tools then yes, we need to rewrite them.
To note that the parent post was not referring to rewrites, but to the language a package was written in first place, which is an important distinction.
I don't disagree - IMO 95% of what's written in Rust should have been written in OCaml years ago. But often those languages aren't used, for whatever reason. If Rust is what gets us the actual big rewrite into memory-safe languages, I'll take it.
OCaml's community has been pretty elitistic up until just some few short years ago. Stuff like "build your project", "build docs", "generate parts of your project" have been a dark territory there for a long time where people invent their own stuff and just look down on many others who want an experience similar to Elixir's `mix` or Rust's `cargo`.
This is changing, happily, but OCaml I always viewed as kind of an elitistic intellectuals club. Happy to be proven more and more wrong lately, though!
I am looking forward, eagerly, to OCaml 5.0 and having an amazingly fast compiler with solid multicore runtime system support! It's very likely I might abandon Rust for OCaml when that time comes (8 - 12 months).
I think you are definitely pointing at something that has been a weakness of OCaml for a long time (but as you say is happily changing). But I wouldn't characterise it as elitism; IME the OCaml community is very friendly.
Instead, I think it comes from the fact that when it was first developed in the 90s, it was viewed in the context of C (and this attitude has carried over somewhat to the modern day where it makes much less sense). For example, building non-trivial projects with just the compiler is very painful from most modern perspectives, but it's very similar to what you have to do for C.
What you say is fair. It seems that OCaml's niche with time moved from competing with C to competing with Haskell and Rust -- at least from where I am standing. Maybe some members of the community aren't OK with that goalpost moving. That would be understandable.
But to be fair, I like OCaml more than Rust but I got very spoiled by both Elixir's and Rust's tooling -- both are excellent enablers of productivity.
Once OCaml overcomes this barrier (and introduces multicore) I am definitely going in, neck deep! :)
Yes, I think everyone would agree that OCaml has gone from competing with the predominant high-level language of 1996 (C) to the predominant languages of today (definitely not C).
Another factor is that creating a language (Rust, Go etc.) from scratch lets you have nice unified tooling to an extent that is probably just not possible with languages with baggage. So I doubt OCaml will ever manage to be quite as seamless as those, but IMO it's already gone from significantly worse tooling than e.g. Python/Java to significantly better, and is still improving all the time!
Which security vulnerabilities have been found and exploited in the tools listed? I mean your point is valid, broadly speaking, but those usually apply to consumer software like, what Rust was initially designed for, browsers.
Ditto. And I'm one of Rust's recent happy converts – it's probably the language I've learned that scores the highest ever when it comes to the unweighted sum of
* joy of coding
* (human) efficiency of coding
* performance of idiomatic code
* interoperability with C
* deployability
I'm totally sold on Rust. Yet: Why would I want to replace tried and tested tools with new versions just because they're written in Rust?
I burned myself on this when I was young and naive: Having just learned Haskell, I tried to use Haskell replacements for everything. It was incredibly stupid and counterproductive (the biggest real-world pain was clinging to darcs over git for a long time at the beginning of the DVCS wars – not because of darcs' arguable good features, but because it is written in Haskell; so stupid).
It seems the new culture around re-writing these tools in rust starts with a few premises:
* Old tools are well established, not looking for major feature additions or changes
* Hard to add new features to old C-based system tools due to mature codebase and being written in C
* Using Rust to reach a low-bug/maintainable state more quickly than a re-write in C or C++
* Adding a bunch of quality of life features that make it worth switching
The big downside with these tools, of course, is that they are re-writes of tools that have been standard Linux/Unix system tools for a long time and can be generally expected to be present on any system for the purpose of shell scripts, services etc. Using any of these tools in a shell script requires the user to install it, along with the whole dependency graph that rust typically comes with (at least when building from source which is typically the case with rust).
I agree with you about the ubiquity of the standard tools but this also has downside: bigger size of the deployment containers.
Nowadays the distroless movement and the general zeal towards minimizing the overhead of a Docker image have the positive side-effect of making us less reliant on the older tools. So we can opt for newer ones (although it's very hard to beat 120KB of a classic old tool to 4MB of a new Golang/Rust tool still).
As for rewriting in Rust, the main point for me is always the memory safety. When it comes to writing programs, there's a ton of very nice languages out there already.
You're probably not the only one. However, I will always prefer CLI tools written in C/Go/Rust/anything native because Python dependency management is a shitshow and its startup time is way too slow.
But you're bringing up a good argument for preferring a binary over a script there - portability, external dependencies, performance are important factors.
I don't believe performance will be significantly different if you're comparing a C vs a Rust tool though.
IME Rust/Go/Python/Node tools tend to have better ergonomics / feature sets compared to C tools. This can be because the C tool is simply older, but I also think it's because writing and maintaining extra quality of life features is a major pain in C, so they tend not to get added. The same can also apply to performance optimisation in some cases (e.g. multithreading is much easier to add without introducing bugs in Rust than in C).
Rust and Go thus occupy a sweet spot for CLI tools, because they have the advantage of high-level maintainable code AND being fast, easily installable binary executables.
I could not care less about the hype squad screaming endlessly about every single side project that is written in Rust because of the sake of the language. If it is useful and it makes me money then fine, other than that I ignore it.
If the tool that is being developed doesn't help me make money, then I do not care or I won't use it.
As for choosing the best programming language? I do not care unless it is the one that makes me the most money. in some use cases, it is definitely not Rust.
> Why even go to a site like HN if your goal is purely to make money?
Yet this site seems to be hosted by a company whose goal is purely about making money by investing in companies they find which are likely to give huge returns. i.e A VC fund.
So rewriting it in Rust for the sake of Rust is irrelevant to them and doesn't give any advantage whatsoever and won't get their attention.
Yes. Many wannabe entrepreneurs come here hoping to gain "street cred" and get their "insightful" comments and projects noticed by YC, hoping to parlay that into funding or hoping a launch on HN will do for them what it did for others (like Dropbox.) Believe it or not Hacker News is serious business, there's a reason the staff is so guarded about releasing any code or hints about their algorithm that could be gamed.
I absolutely care if the BT stack of my phone is written in a safe language, because many of the security flaws that make many, many Android devices permanently unsafe (due to end of support) are caused by memory unsafety.
On the other hand, I have the suspicion that Rust is voiced a bit too much by a loud minority - I like Rust as much as I don't like Golang, nonetheless I think that for the majority of non-performance critical projects, Golang may be a better fit.
One reason to care is because some languages (go, Rust) are statically linked by default which is quite useful for system tools that can be disabled entirely by messing with shared dependencies.
If I know what it's written in I can infer a lot. If it's written in Python I know I don't want to call it in a loop, I have to care about my interpreter version if it isn't packaged with one, etc. If it's written in Rust I can infer that it'll start up quickly and likely perform well.
I get the point of wanting to use safer languages but I feel this list somewhat misses the point and looks more like some misguided worship to a single language. For example a lot of complaints that can be made about C and C++ don’t apply to Haskell yet this list parades a Rust counterpart to Shellcheck as if it’s automatically better just by the fact it’s written in Rust (frankly, I’d rather trust the more mature Shellcheck).
And a lot of those projects are just someone’s pet project, often written as a task for learning Rust, and certainly likely to have numerous new bugs that haven’t yet been found just by virtue of being a ground up rewrite.
As I said, I’m fully in favour of using newer and safer languages but we need to be careful not to get carried away with thinking anything new is better simply because it’s new.
Especially since one huge strength of Rust is its C interoperability, one would imagine that incrementally replacing or extending parts of the legacy tools would be a much more sound approach. But of course, the point you make does answer this:
> And a lot of those projects are just someone’s pet project, often written as a task for learning Rust, and certainly likely to have numerous new bugs that haven’t yet been found just by virtue of being a ground up rewrite.
Many of the replacements are indeed not meant to actually replace the tools. They're learning projects. That's great, but it's a bit scary that people wanna use them for anything else.
> Especially since one huge strength of Rust is its C interoperability, one would imagine that incrementally replacing or extending parts of the legacy tools would be a much more sound approach.
When it is unlikely that the maintainers of existing tool written in C would accept pull requests that add Rust as a dependency (and I imagine this would usually be the case), why wouldn't you start anew? It also allows easier experimentation on the architecture of the original tool, and breaking compatibility. For example, ripgrep isn't a drop-in replacement for grep and the author never intends it to be, so forking grep probably wouldn't have made it easier to develop.
I think the difference between a learning project and a "real" project is mostly how many people end up using it.
JS most of the time works inside some of the biggest and most strictly reviewed software packages in the world -- the browsers. JS needn't be concerned with memory safety, only not to allow (too big of a) memory leak.
Rust solves another problem. Most of the security vulnerabilities out there come from buffer underflows or overflows.
Rewriting tools people rely on every day in a language that prevents those is not failing to learn from JS.
As a fan of the language I also don't get this. I mean I welcome wider adoption of Rust for various reasons, but this whole "it's written in Rust!" checkbox filling doesn't have any connection to what matters in reality.
If you're rewriting something anyway, or creating something new then Rust should be considered. But a lot of the value of old software is not the set of features listed in the Readme. Not every area comes with its own "don't roll your own crypto" warning, but almost everything has its share of edge cases and quirks that were already handled in existing tools.
Of course it is possible. If you get your hands on the binary itself there's nothing that a good reverse-engineer will not be able to do.
Point however is, if you have a SaaS which uses Rust, the chances for triggering a buffer underflow / overflow exploit are zero.
As for other aspects of security, Rust makes no special effort there. The main win for using Rust is to eliminate the most widespread bugs (the memory safety ones) from the get go. Everything else is still fair game and has to be paid proper attention to.
> the chances for triggering a buffer underflow / overflow exploit are zero
Doesn't the mere existence of unsafe make this untrue? From a quick google I can see at least one[1] so the chances are definitely a lot higher than zero
The GP is either being incorrectly absolutist, or left out an important qualifier: "In safe Rust, the chances of <insert memory safety problem here> are zero."
But that is also absolutist. Instead, you might say, "In safe Rust, the chances of <insert memory safety problem here> are zero, modulo bugs in the compiler."
But that's not quite the fully story either. So instead, you might say, "In safe Rust, the chances of <insert memory safety problem here> are zero, modulo bugs in the compiler or any dependencies or any other uses of 'unsafe' in your application."
And maybe that's still not quite it. So let's try again: "In safe Rust, the chances of <insert memory safety problem here> are zero, modulo bugs in the compiler, any dependencies or any other uses of 'unsafe' in your application and any other platform specific tricks for arbitrarily rewriting memory used by your process."
That probably covers it. But it's a mouthful. It's probably best to avoid saying the chances of anything are zero. Instead, the key value add of Rust is that it flags areas of code that permit introduction of UB for easier auditing, and also enables building safe---for all inputs---abstractions for others to use. But humans are fallible, so there is and always will be an opportunity to fuck something up somewhere. What Rust gives you over something like C or C++ is a way to make assumptions that are more fine grained than, "I trust that every line of code written in the transitive dependency chain is free of UB."
What I simply meant was "it's quite hard to introduce memory non-safety in Rust in a normal everyday work". But your explanation is definitely much more precise.
Right, I figured as much. There is so much misunderstanding about what 'unsafe' is and does that it's really really important to be precise about it when your audience isn't Rust programmers.
I suppose that depends on your definition of normal everyday work though, if you're developing device drivers or something that needs to write into shared mmap'd memory or whatever `unsafe` is something you're going to need all the time (from my 0 rust experience understanding?)
Thanks, that last bit clears it up for me. I've never touched Rust but write C++ everyday so don't really know much about what is guaranteed vs. what is less likely due to borrow checking/compiler verifaction
I imagine what they're alluding to is that it seems like most of the security bugs are memory-safety related. The number is about 70%, and has been replicated independently at a number of places.
Calling unsafe libraries automatically makes the calling code unsafe by definition. You cannot have a 100% safe language that also interacts with its (unsafe) environment in any way.
The language needs a way to express the lack of safety of an external library in order to clearly isolate it from the safe code sections. In Rust that's the unsafe block.
This is an accurate view. The "Awesome" meme is indeed a bit dangerous in this case :)
Something important to be noted that RIIR does not represent the attitude of the Rust community - actually, the instances I've read until now were from people who had little or no experience.
I think this distinction is important because there are some negative attitudes attributed to the community that I personally haven't found (e.g. "screaming" how good Rust is; committed/professional Rust developers are already busy screaming at the BCK ;)).
If a common question in a job interview is "have you written any OSS projects that are used by others?" and a positive answer is expected, then this is where we end up.
Thousands of new developers, eager to learn Rust and get a job, rewriting anything they can get their hands on and pushing it out for the community to use.
To clarify: I'm in no way opposed to experimentation, learning and weekend projects.
But there current "github-centered" development culture makes it very difficult to tell if authors are invested in their FOSS projects are going to maintain them or not.
As the churn keeps increasing we can end up with ecosystems collapsing.
In comparison, the library maintained since 2003 by the Nebraska person is less worrying. https://xkcd.com/2347/
Also, the "don't write code, just find a library that does what you want to do and plumb it in" mentality means that people approach a problem by searching Github, find these half-finished experiments, and ship them to prod.
I agree, I see the same thing happening in Go - there are endless pointless libraries being developed and published for no reason except as filler in people's Github accounts for recruiters to look at.
I get it pretty often too. Having said that, I haven't typically had much to show and I've usually been offered the job anyway so I'm not sure it's typically given that much weight.
I've asked that question. I use it as a way to offer an alternative demonstration of work to developers who are early in their careers. It's not about seeing the code as much as it is wanting to see that a person has taken software through the full lifecycle. I'd rather see an app in an App Store or a live site on the web, or even just talk about an in-house project that they've managed over a period of years.
That makes a lot of sense. Most of my github projects are from prior to my first job, which I was quite proud of at the time, but are somewhat less demonstrative of my skill level now over 10 years later.
I don't think that list misses the point, a lot of what's in here is actually a better/more modern alternative. Starting with "An experimental container runtime" isn't a great idea, but bat, tokei, dust, fd, skim, exa, fnm, hyperfine, tealdeer and just are all serious projects. I do think the list could be curated a bit harder (and ripgrep should be added, fortunately there's already a pull request).
> And a lot of those projects are just someone’s pet project, often written as a task for learning Rust, and certainly likely to have numerous new bugs that haven’t yet been found just by virtue of being a ground up rewrite.
I'd say 6 of them are not as serious as the others, but even them have 5 contributors or more. Is this what you mean by "a lot", 6 out of 34 (at the time of writing this) ?
“A lot” is certainly a subjective term and thus open to interpretation. One might argue that the entirety of the collection isn’t “a lot” since it only amounts to 34. Others might argue that 17% is “a lot” as it’s that means near enough 1 in 5 projects isn’t mature and that’s a pretty poor signal to noise ratio.
If the repo advertised itself as a curated list of interesting Rust alternatives to standard tools then I probably wouldn’t have commented. But it said “replacements” and that suggests a higher level of maturity and community support.
My take is that was just poor wording and they should be considered alternatives or potential future replacements in progress. Once you give up on that word, the page makes more sense.
More sense in one regard but less sense in another. If you’re curating a list of alternative software that is acknowledged not to be mature in all instances, then the arguments for making it Rust specific become moot since it’s no long a curated list of more secure software.
> For example a lot of complaints that can be made about C and C++ don’t apply to Haskell yet this list parades a Rust counterpart
I think there's an obvious case for being aware of rewrites from high-level memory-safe languages (not just Haskell, of course) to Rust, although the case is obviously different from rewriting C/C++ code. Besides the obvious improvement in performance (Rust being on par with C/C++ itself) it gives us a better idea of whether Rust is actually missing features from these languages in a way that meaningfully impacts maintainability.
> I’m fully in favour of using newer and safer languages...
And the ONLY way C/C++, that cost BILLONS on bugs alone, is replaced is that people rewrite things on something better (and Rust IS better).
And this start for fun.
And it could get new bugs? yeah... but it also NOT continue the old bugs that C/C++ cause, some that will be erased of existence (like nulls) for the rest of the life of the codebase.
And this is the point. Logic bugs eventually are squashed, but language design mistakes are nearly eternal for the life of the codebase.
I have been doing it for 20 years, and all the time Pascal had null-safe strings and C-like performance. And the strings even had mutability xor aliasing with copy on write
Imagine an alternative reality where Borland did not went greedy, Java 1.0 would just be like Go, while .NET Native would just be improved VB 7 and a new language C# (just a kind of "managed C++") making COM developer friendly without C++ boilerplate that persists to this day.
Are you implying that Rust is only / mostly used as a replacement for unsafe C/C++, and Rust programs should always be evaluated in the context of this choice?
The safety issue or comparison against C/C++ was not mentioned in the submission. So I don't follow why the protest against replacements for Haskell programs, based on a safety argument.
I mostly agree with this sentiment but I think it's not that necessary to point out. Yes, some of these are abandonware but many are written by the same set of strong Rust programmers. I think you'd be hard pressed to argue that ripgrep or fd is not a valuable contribution to the OSS ecosystem.
The more interesting question is - why is that? Sure, Rust is memory safe, but I suspect the real reason people built high quality software in Rust is because cargo is good and because minimizing copies is part of the culture around the language. This means that libraries are cheap to use, and as a result programming feels productive, because you don't have a reason to reinvent the wheel (see arrays in C, or various Boost libraries that have been moved into stdlib C++). For example, compare the cost of Python's click to Rust's clap. Or look at serde.
I find it more interesting to ask the question "why do people spend so much hobby time writing interesting programs in Rust?".
Nah, the thread is about people redoing the whole car, adding airbags, swapping out the carburetor, installing power steering, and maybe throwing in a new stereo and a paint job.
sed vs. sd, for example https://github.com/chmln/sd The semi-lovable but reliable old jalopy overhauled to match modern practices.
> I find it more interesting to ask the question "why do people spend so much hobby time writing interesting programs in Rust?".
My theory is that people that like Rust want to work with Rust, and for that the best way is to prove that it's a good <language> replacement in <space> multiple times. By making modern and good versions of classic CLI tools, they prove that it can replace C in that space. By making good new CLI tools, they prove that it can replace Go (that replaced Python/Ruby) in that space. Some people are trying to replace JS on the frontend (Yew), some people are trying to use it in the backend (Rocket, Actix). Some people working on chromium were (I don't know if they still work on it) trying to prove that it can replace C++/a GC'd language while having the benefits of both. There are lots of examples like that.
Rust often ended up as the most loved language on the stackoverflow survey. I don't know if it's intentional or not, but people in the community seems to understand than marketing is important, and proving your claims is too. And the best way to prove your claims is to deliver working software that's at least as good or better than what was previously made in <language> in <space>, to prove that you could use Rust here.
Of course as you said, cargo really helps here. I'd say other big players are the culture and the type system. The type systems acts as a first layer of documentation and helps ship better code. The culture is also heavily focused on documentation and welcoming newcomers. Compiler errors are a joy to read, rustfmt is a nice addition, clippy is also very good, and people tend to be very nice and welcoming in the community.
What I like with this explanation is that is also in part explains the rewrite it in Rust (RIIR) obsessed people: they see how the community act, fail to understand that delivering working code is the most important part because it's less visible than the "pure marketing" talk, and so blindly repeat the "pure marketing" talk.
> I find it more interesting to ask the question "why do people spend so much hobby time writing interesting programs in Rust?".
Probably for the same reason people spend so much hobby time writing interesting programs in other languages: because they either already have experience in that language and like it or they’re looking for an excuse to learn a new language.
Personally I don’t find it an interesting question to second guess why people have hobbies. Sometimes people enjoy challenges because they’re easy, but sometimes people enjoy challenges because they’re hard. My wife relaxes by watching TV. I relax by writing OSS. Everyone is different.
It's unlikely that this evidence exists, and if the 'clone everything to get past the borrow checker' attitude is commonplace then we get stuck on what 'high quality' means.
It's way too hard to build evidence for something like that. Virtually all research into PL effectiveness ends up not account for the absurd number of variables.
One metric might be how long it takes a developer to build it from scratch given all the esoteric builds in C/C++ land. Another might be lines per cve in a given language. Another still might be time to fix cves per language.
I'm thinking rust would come out on top of those, but would be interesting to measure!
> I find it more interesting to ask the question "why do people spend so much hobby time writing interesting programs in Rust?".
Maybe just as a learning experience or to be able to say "I know $hypedLanguage." Nothing against rust on my side, but at least for me, time spent on writing something in a language new to me is in no relation to my perceived usefulness of that language.
I personally believe that people new to programming love rust because they get compiler feedback instead of runtime errors. Given what is currently en vogue, rust does stand out there. OTOH, if looking at some old, boring languages, you already have that. Minus the hype.
The page has been changed since my post. It originally called them “replacements”, which paints a totally different picture. From reading the GitHub Issue thread regarding the name change, it sounds like various people on Reddit were making similar concerns about the original choice of wording too.
I don't like the word "replacement" either. If something works, something works. Very often, it took a lot of time to polish the tool. It won't be easy to recreate all of the functionalities - even if using a new technology making it easier to develop software.
At the same time - I see the list as a testament to enthusiasm about Rust. People use it. People try to reimplement things. Also - it is likely that one day one of these reimplementations will take over (due to performance, functionality, or simply - because of being maintained).
For CLI programs Rust is a really nice alternative to languages that require a runtime. I've significantly sped up the time for my bashrc to execute (my bashrc is heavy) by replacing certain tools with Rust tools.
If I wanted my IDE to execute something like ShellCheck on every save, or even every few character presses, I feel like I'd much rather not have to spin up a big runtime.
I really doubt the GHC RTS is appreciably slowing down ShellCheck. Especially if its built with RTS flags optimized for startup. Feels like optimizing the wrong thing a bit.
ShellCheck starts in a few milliseconds on my machine, easily fast enough to run on every character press. Haskell's runtime isn't like the Java JVM, and GHC compiles to machine code, not interpreted bytecode.
> yet this list parades a Rust counterpart to Shellcheck as if it’s automatically better just by the fact it’s written in Rust
You're perceiving a sentiment from the list that I just don't get. All it is is a list of alternatives written in rust. The fact that they're written in rust, is really only valuable to people who are already interested in rust.
> looks more like some misguided worship to a single language
this pretty much sums up Rust atm. There is a lot of hard-core evangelizing which is off-putting imho. I still like the language but wonder if a rewrite for rewrite sake will convince the majority, considering this is a community effort and not a large org controlling the future of the language. Especially that there are other ecosystems like Zig or Nim that are pretty cool too and fit into a similar niche. Hard-core evanglism just feels like advise from somebody who's only tool is a hammer.
on the other hand I do remember the amount of marketing and advertising budget that Sun and later Oracle pumped into Java to make it fly in the late 90ies. They made deals with University professors to make sure the language is favored. It was a hype-shit machine unmatched by anything we've seen since (and when I use the word shit I mean your browser being hi-jacked and reconfigured with a new toolbar and a search engine simply because you installed a JRE. not too different from how malware behaves). It's not that early Java was terrible (though it certainly over-promised and under-delivered for over a decade) but if Java would have had to grow organically back then like Rust/Nim/Zig do today, then Java wouldn't have stood a chance. There were even ads at airports and on TV. Ads for a programming language during a time most people watching TV didn't know or care what computers are for!
So I'm quite glad that this is just a couple of people who are overly excited about what they do. We've had a lot more asymmetry with larg-corp sponsored garbage. Go Rust!! (and Nim and Zig)
In what concerns Zig, I would rather have the use-after-free scenario be taken care of before the whole RIZ starts.
Regarding Java, it had a major killer feature that triggered adoption even before the whole marketing wave started.
It was actually portable, not like trying to write C or C++ code in the middle of ongoing standards, compilers still stuck in the old ways, POSIX on paper and on actual UNIX clones not being 100% the same thing.
It was like a fresh air of portability even with an interpreted runtime, JIT only came in by 1.2 days.
I might not be familiar enough with the original tools, nor the Rust ecosystem, so excuse me if this is misinformed. But the trend seems to be that the Rust tooling is a lot richer than their original counterparts.
Is this accurate? And if so, is it super easy to write rich CLIs, with tabulated, coloured, interactive TUIs in Rust? Are their some common, widely used libraries driving this trend?
If people are still cursing at your programming language 20 or 30 years later, you've done at least a decent job of things. Do you hear many people complaining about Eiffel, Dylan, or Boo lately?
> Are their some common, widely used libraries driving this trend.
The rust community leadership made a strategic decision to pick 4 areas to suggest focus on. CLIs was one of them. (Scroll to “build it in rust” on https://www.rust-lang.org/)
TUIs are in many ways more ergonomical, in particular they make experienced users more productive and they can help reduce RSI by limiting the need to move the mouse.
I love the idea of having more user land tools but are those tools POSIX compliant? If not those aren't replacements.
Now if that's the case that's also cool. I also love new ways of doing old things and we might sooner or later have something that looks like Redox on Linux/BSD kernels.
This silly list (for the several reasons that have been repeated in most comments here) makes me want to see another list that would actually be useful:
A list of "software replacements that are considerably better[0] than most well-known counterparts"
[0] disclaimer for nitpickers: obviously not for 100% of cases, but what we could say for the "general usage". E.g. I'm sure there is someone out there that claims an obscure feature of grep is not replaceable for their workflow... but for the other 99% cases, ripgrep is fantastic.
I know people here are a bit touchy about Rust, and writing everything in Rust. But here is one advantage I have found with Rust programs: The installation just works. Cargo, as a package manager, is just wonderful.
I have lost counts of how many times my "brew" or "pacman" failed to install something. It's much worse if it's Python; and things that work in macOS are not guaranteed to compile in Linux, and vice-versa. On the other hand, my "cargo install" almost never fails.
For that reason alone, I welcome any initiative to write something in Rust.
Wow I think in 10+ years, I've only had 1 or 2 cases of brew failing to install something. It's a piece of software I've been consistently impressed with over the years. I wonder why our experiences differ so much.
It's been a while since I did any coding on macOS but if I remember accurately, I think it didn't play nice with macports (which I needed for some other packages).
Project specific centralized package repositories have the danger of being political (see npm and others). There already is a CoC here that can be used against wrongthink or people you don't like:
Pacman and cargo accomplish completely different tasks. One is a package manager for a language and a language alone, the other is a package manager for a whole operating system. Pacman can not be responsible for a build failing, since it doesnt build anything in the first place...
Apart from that, most modern languages come with a package manager nowadays. Why do you like rust more than go? Or nim? Or elixir? Or crystal?
It's been ~11 years since I used a distro based on pacman as a daily driver, and I was very harsh on pacman at the time. It seemed to lag behind the high-level package management tools of contemporaneous distros in terms of flexibility (and, once you had a lot of things installed, performance).
But I never, ever had it just fail to install a package, and I don't remember much fuss with dependency resolution taking manual fiddling, either.
I'd love to hear your pacman horror stories; gimme the hot package manager gossip :D
The way pacman installs new kernels by default is just replacing the old package with the new one. This is bad because if for some reason the new kernel package gets corrupted, the system will fail to boot, and you will have to arch-chroot into the system to install the kernel again. Fedora on the other hand keeps old kernels for some time before removing them.
Wow, a lot of hate for this list, but I'll unpick something that is slightly below the surface here:
We are currently seeing a bit of a command line renaissance. The justification for this is a 'rust rewrite', but as many have pointed out, just because something is in such-and-such a language doesn't make it good.
However, what I tend to find with the new rust CLI tools is that they bring with them modern design sensibilities. They can use better conventions, better defaults, assume things like colour terminal emulators (or fallback to non-colour if not outputting to a terminal).
The experience of using them is often much different to using the old GNU versions made in the 80s which reimplement tools made in the 60s and 70s.
I'm not saying all the other criticisms in this thread aren't valid, but I think it's worth highlighting the positives of the tools mentioned in a list like this rather than just throwing negativity on it.
Agreed, plus some of them replace Python CLIs which are awfully slow to start (400 to 1500ms in my experience). Having those start in 3ms is a huge win in terms of ergonomics.
My biggest problem with this list is that when I write shell scripts, I usually intend for them to be used by colleagues or others.
Having non-POSIX applications used in these scripts means that portability is limited.
Yes, I know that with aliases I can override this, but I’m not convinced yet that straying from “this will work on everyone’s machine” to “set an alias for X, and X will probably be an appropriate stand-in for Y” is good enough to guarantee zero-issue compatibility.
Honestly this just highlights a weakness of shell scripts. The idea that you cannot use any new dependencies (or even upgraded versions of the current dependencies!) would be unthinkable in most other settings.
Absolutely, and I think this raises the question of "in what sense are these 'replacements' for existing software?".
If we see them as "immediate drop-in replacements" then the value of them must be that they're identical in every way and the value in them would be in the reduced maintenance overhead for the developers, with maybe a small performance boost if we're lucky. This would be fine, but it's not something that we'd notice if our distro went from using C `grep` to using a Rust rewrite of `grep`. We certainly wouldn't need a list telling us what replacements are available.
However, if we see these as spiritual successors, ones that perform the same function but aren't intended to be aliased to be precisely the same thing, then it makes a lot more sense. We're not looking for backwards compatibility, portability or identical feature sets. We're instead being offered a smorgasbord of options on what ideas we want to embrace and which ones we would want to leave behind.
If ripgrep is better in all use cases than grep, then eventually ripgrep will be the one being used in the scripts organically. We shouldn't be dismissing ripgrep out of hand just because a colleague won't have it on their system. Or, at the very least, we should be noting the good points while also noting that we won't be using the tools ourselves for portability reasons.
It's a fair point, but I find I can get along quite well using the new tools myself interactively and falling back to the standard versions if I need to write a shareable script. For me the degree to which ripgrep is better for day-to-day use more than outweighs the cost of remembering how to use grep once in a while.
I see scripting and interacting as inherently different use cases. I absolutely love Fish shell for interactive use, but I don't use its scripting (other than to write functions I use from the command line) because it's not available everywhere. POSIX sh is much more appropriate for scripting, but I definitely wouldn't want to use it as my command line because it lacks all the greatness that Fish brings to the table.
Same with POSIX tools and their replacements: `find` is perfect when I know I'm going to run a script on a bunch of similar Linux hosts where I can't count on `fd` being available, but if fd is around I'm definitely going to use it at the shell prompt over find. As a side note, if you're doing cross-platform work, the new tools are better in some ways than the things they're replacing. Sure, `fd` isn't going to be installed by default on our out-of-the-box EC2 instances, but once it's installed at least it has the same arguments on Linux as on BSD and macOS. If I'm going to have to install GNU utils on a system to manage it, it's no extra work to install fd at the same time.
Here's what my search workflow looks like, this is the main way I program:
1. `rg something`
2. Too many hits? `rg -l something` to just list matching filenames
3. Hopefully based on that we can narrow enough using a glob: `rg something -g "*.m"`. Otherwise write a regular expression to get more specific matches.
4. After I have exactly the set of matches I'm interested in, I pipe the results to a text editor that can interpret grep-style output (e.g., quickly jump between results, I usually use Vim).
The key here is that the command-line excels at iteratively refining a command based on output, and that's exactly what we're looking for to use search effectively. `rg` provides a more ergonomic UI than `grep` (recursive, automatically ignoring version control files, including ignored), which is really important if this is the main command that you're running all-day, which it is for me.
# `fd` (`find` replacement)
This advantages here are the similar as for `rg`, quickly being able to recursively find a file by name, a problem that similarly benefits from iteratively refining a command.
# `bat` (`cat` replacement)
`bat` is just the best way I've found to quickly view a file from the command-line, it adds syntax highlighting and line numbers, which are both invaluable.*
The expressions against hype and the admonitions regarding unmaintained hobbyist projects on GitHub are okay, but personally I'm happy to learn about the tools on this list, and I'm grateful to be able to study reimplementing something we know (coreutils) in a language we ought to know better (Rust).
This doesn't automatically mean we would deploy this stuff when going to Mars; so I don't think some of the sentiments expressed here are the fault of the enthusiastic reimplementers.
(Looking forward to your reimplementations of coreutils in R5RS Scheme.)
The main thing that interests me in Rust and Go rewrites of common tools (bat, ripgrep, broot, etc.) is that they use their newness as an opportunity to have nicer output and simpler CLIs or nice TUIs. It's not really about the language for me.
These Rust replacements are unique in the way that most of them vastly improved on the original. I still don't quite understand what about Rust made that happen. They could have been easily written in, say, Nim years before Rust existed.
Systems programmers got really excited :) I don't know much about Nim but I don't think it, or Go, etc. are aimed as low-level as Rust is. Rust is pretty unique in being as low-level as C, yet providing safety and higher level abstractions.
Nim is aimed at pretty much the same level as Go, I would say. Only with more modern abstractions.
And currently (with Nim 1.4 and its memory management), it's moving towards Rust's field. Their performance has always been in the same space, especially regarding cmdline tools, where latency is more important than most other tools.
Nim has very easy ways to get full C performance in pure Nim with a trivial FFI to just call C and is both higher level than Go and lower level at the same time because of that. Simple analogies often fail to describe. :-( { This is not intended as a dig/criticism but note on communication limits - you should just learn more about Nim at https://nim-lang.org/ }
Remaking traditional UNIX tools is a common exercise when learning system programming.
And because Rust is quite trendy nowadays, I guess a lot of people wanted to try it out, and in some cases, that exercise turned into a viable product. Nim never got the same popularity.
If were talking about libs and APIs then game on, but otherwise, I actually don't care what language a tool is written in. I care how good the tool is.
No, I don't think you will find a high correlation between language and tool quality.
I compiled a list of modern replacements for unix commands [0]. While many of them are written in Rust, there are C, Go, and Python programs on that list as well, because I feel the point is not what language the tool is written in, as long as it solves a problem or is an improvement over existing tooling in some way. Not a knock against this list, I just think it is better to list all tools that may be useful to people, rather than excluding some simply because of the creator's choice of language.
Did anyone else parse the title as “replacements for existing software that is written in Rust”? I thought, wow, Rust is already passé, what’s the new hotness?
Ah Uncle Bob,the philosopher of programing methodologies. I'd imagine writing any program beyond hello world with 100% test coverage would be beneath him.
I (mostly) love Rust, but the fact is that it makes lots of easy things hard. Most software should be written in a GC language. Even Java or Go are a better choice for most “backend” development, despite the fact that from a PL design standpoint they’re vastly inferior to Rust.
Deterministic memory deallocation makes it harder to reason about not just “easy” code, but also linked structures and lock-free algorithms (actually it is a substantial implementation obstacle for the latter). It must offset these costs by providing predictable latency or low memory footprint. Most applications don’t require either.
Perhaps someone already mentioned it here, but GraalVM lets you interop between Java and anything LLVM, and therefore Rust, without the JVM startup cost, with breakpoints across languages, even (as I understand it). Compile time is long, though.
borkdude has been releasing for some very cool stuff using this stuff. Here's a Clojure/Rust combo:
228 comments
[ 3.1 ms ] story [ 284 ms ] thread[0] https://github.com/ezrosent/frawk
Does anybody miss Sendmail? That had also "stood the test of time".
The idea is that language will not solve poorly designed software, but a developer that knows what he is doing like Wietse did with Postix. And Postfix stood the test of time pretty well, considering how complex it is.
Another idea is that a software can have a great design but still be full of security holes. C allows for that.
If we leave it to the "adequately advanced programmer" to avoid those, we're doing it wrong - with plenty of historical examples to prove this.
The only thing that will actually prevent security holes is attentive software engineers deliberately choosing to learn tools and practice habits to avoid security holes by design. The professional judgement of a lot of those engineers is leading them to choose to learn rust right now.
Absolutely. It's just that statistically the memory safety bugs have been the dominant percent of all the bugs, so rewriting core tools in a memory-safe language does make sense.
Time & attention are limited resources. By eliminating the dominant class of bugs you have more time and attention for more subtle bugs.
Beautifully articulated.
No. But it will make sure there are no issues of the type buffer overflow and such.
The rest, we can judge when the software is delivered.
https://www.cvedetails.com/vulnerability-list/vendor_id-1902...
I'm not so sure....
And those are bugs from from people writing core stuff, like you know, the people who should know Rust better than you and me. People writing foundational code that your program will run on top of.
Of special danger is the last CVE, that was rushed and backported into the last release. But nobody talks about it. I can imagine why.
I can bet that the "but the bug happened within unsafe code" excuse will be the next Rust meme.
Rewriting in a memory-safe language is worth it. But it shouldn't be a religion, I completely agree on that.
Even if Rust was invented in the 80's, and such a project was in it, the same could happen in one of those unsafe{} blocks a Rust project like this might need to use (and we are forgetting all the ordinary bugs even in the safe portions of the code).
Sure, being in C might be a part of the problem, and this is a good space for tools like Rust to occupy more space, but there's a bigger picture or else Linux would be breaking all the time, and its a solid, big and complex piece of software that works pretty well because there's a lot of economical incentives to keep it going.
There probably is a Wikipedia page describing this phenomena but I'll settle for calling it "the commodity effect": when something is very widely used then everybody assumes it has been improved to perfection and that it has no flaws (this is not exclusive to software engineering, it happens in many areas of life).
> the same could happen in one of those unsafe{} blocks a Rust project like this might need to use
Sure but they are much easier to search for and audit (like with e.g. `cargo-geiger`). That by itself is a win. Whereas C/C++ needs a special class of linters and scanners, a good chunk of which are paid (a deal-breaker for many software devs not because they're poor but because they don't want to pay for them out of their pocket, and their employer is blind to see the added value and buy those tools for them).
> and we are forgetting all the ordinary bugs even in the safe portions of the code
Only people with agenda forget about those. I worked with Rust in 3 companies in total so far and I've only seen serious, diligent, hard-working devs with attention to detail and a slight case of efficiency mania. :) And they were very mindful of potential bugs and we wrote a lot of tests of different kinds like acceptance/integration, unit, property, and probably others.
No normal smart developer forgets that memory safety bugs are just one class of all the bugs. Turns out however that those are the most widespread bugs in general so IMO it is sensible to use a language that eliminates them by the mere virtue of compiling your program -- and not using unsafe{} in certain ways.
As another poster said, time and attention are limited resources. By removing memory safety bugs from the picture our brains are now free to pay proper attention to the more subtle bugs.
> or else Linux would be breaking all the time, and its a solid, big and complex piece of software that works pretty well because there's a lot of economical incentives to keep it going.
It is solid... as far as we know. Truth is, nobody is fuzzing the Linux kernel 24/7 except maybe the worst possible actors like national intelligence agencies or malevolent hacking groups -- and they of course would never share and help fix a zero-day because it can help them make money by exfiltrating sensitive information and/or breaking in protected networks.
Don't get me wrong, I admire the Linux kernel devs. They are a standing tribute to all ideas of free and open source and open project management. But Linus himself said security isn't the first priority of the kernel which already means that certain potential problems in the code are being overlooked in favor of speed and stability.
(There was a story some weeks ago on HN showing that there was a pretty nasty exploitable bug in the Linux kernel but I lost the link.)
1. Rust is new and cool and I want to be a Rust developer.
2. There are almost no Rust jobs out there.
3. I don't have original ideas or any actual use cases for this language.
4. So I will rewrite something that already exists in it.
True. But most other classes of bug are not security bugs by default in the way that memory safety bugs are.
Also half the software mentioned in that gist should never be used in security-focused applications anyway (if your depending on ‘cat’ or ‘awk’ to be bug free for your application to be hardened then you’re already doing it wrong)
An overflow bug in awk is only exploitable if someone can craft input into awk. And if you’re allowing people to do that then you’ve already given them access to remotely run code without them needing a bug.
This is how the vast majority of build systems are configured. If I have owned a dev and can push code to a branch I can likely execute code in their build environment. If they then parse that build output with something like awk, that's attack surface.
Is it the first thing I'd try to attack? Nope. But I do wonder what the cumulative impact of every binary on your system being memory safe would be. I'd definitely feel better.
But then you’re also executing that code as a trusted user. Ie the real hack is owning the dev.
> This is how the vast majority of build systems are configured.
No. The vast majority of build systems are either locked to private repositories, and/or a subset of developers, and/or run in an ephemeral sandbox.
> But I do wonder what the cumulative impact of every binary on your system being memory safe would be.
I’d already answered that. It’s negligible because if someone can exploit a buffer overflow in awk then they can already run arbitrary code in awk (remember, awk is a programming language). So if you’re not already taking precautions about who is running what on your build system and how running code is sandboxes then it’s not going to take a buffer overflow bug to RCE it.
That's like saying the real hack is sending a phishing email, ignoring all privesc, lateral movement, etc.
> No. The vast majority of build systems are either locked to private repositories, and/or a subset of developers, and/or run in an ephemeral sandbox.
I didn't say otherwise. I said that the majority allow untrusted execution. If you consider your engineers laptops to be trusted you're thinking about networks the way people did ~10+ years ago.
> It’s negligible because if someone can exploit a buffer overflow in awk then they can already run arbitrary code in awk
This isn't true, but even if it were you can just replace awk with anything else that's not turing complete.
i never said anything about trusting engineers laptops.
Let me put it another way, are you running your build pipelines bare metal, allowing anyone to instantiate the pipeline against a branch of code that anyone in the public has written? Or is that build pipeline running against a private repository, or triggered by trusted users, or does it run inside a temporary VM or container that gets destroyed afterwards? That’s where you harden CI/CD. I’m not suggesting fixing buffer overflows aren’t worthwhile (security should be multilayered after all) but it’s missing the real risk since you’ve already got code remotely executing. Thus your threat model should be about ensuring you minimise the blast radius of that code.
To use your colourful language: if your build pipelines are running bare metal then you’re thinking about infra like people did ~10 years ago.
> This isn't true, but even if it were you can just replace awk with anything else that's not turing complete.
But you’re still not addressing the real issue. If you can’t trust the the code executing and you’re not sandboxing that code then you’ve already lost the fight. You can’t give unrestricted access to execute code remotely and then moan “we wouldn’t have been exploited if everything was written in Rust”. The bone headed design was allowing unrestricted remote code execution and not the buffer overflow bug in ‘cat’.
> That's like saying the real hack is sending a phishing email, ignoring all privesc, lateral movement, etc.
It’s really not. What I’m taking about is the equivalent of saying “phishing attacks are always going to happen so you need MFA et al to harden against the array of of ways one can manipulate the system”
You’re entire security model seems to be focused around fixing every fucking executable against every possible class of bug and that’s insanely impractical. Whereas I’m suggesting one shouldn’t implicitly trust remote code in build pipelines to begin with.
The last study I saw was that 52% of security vulnerabilities were still basic memory safety vulnerabilities. Memory safety isn't the only thing, but it's still the biggest thing.
> (if your depending on ‘cat’ or ‘awk’ to be bug free for your application to be hardened then you’re already doing it wrong)
People run cat or awk on log files (where an attacker could easily craft particular data patterns) all the time. Maybe they shouldn't, but they do.
My point wasn’t that memory safety isn’t a big issue. It’s that it’s not the only cause of vulnerabilities. Assuming your 52% figure is accurate (and I’m happy to take your word on that) that still means that roughly half of all vulnerabilities are not memory safety. Which means your figure actually just reinforces my point.
> People run cat or awk on log files (where an attacker could easily craft particular data patterns) all the time. Maybe they shouldn't, but they do.
I think there’s enough obscurity there hidden from the attacker that such usage should be safe from all but the most determined of attacker. And even then, they’d likely need some other attack to probe for usage about what systems are listening to what logs and using which POSIX utils that you could argue the attacker already has the access they need without going to the pain of crafting an overflow bug in a log file.
Im not saying those bugs don’t exist nor shouldn’t be addressed, I’m just being pragmatic about their exploitability.
The bigger issue is CI/CD pipelines. Often they’re define in the same mono-repo as the source itself and they’ll have execution rights automatically because that’s literally their function as a build and test pipeline. But because said pipelines are already visible and already executable, you need to have greater controls around that pipeline to prevent abuse otherwise it’s already game over even without an overflow bug.
Sure, but that half is every other category of bug put together. I guess if your model is that every other class of bug will be significantly increased by a rewrite then maybe that logic makes sense, but that seems unlikely to me.
Try reading the thread again
And yes, I did say that at the start of the conversation. Sorry I’d forgotten about that part because this conversation had taken a tangent on discussing other specifics in security.
I do stand by my comments, a rewrite does introduce new bugs and not all security bugs are buffer overflows. Rust doesn’t protect you against bugs like Shellshock. They protect against bugs like Heartbleed. But people frequently forget about the former when focusing on the latter. So absolutely we need to be considered when replacing battle tested software with an entirely new code. It’s not an unfair meme when people state this. If it were an easy and risk free upgrade then we’d have already done so in one of the other safer languages that predate Rust. A lot of what’s changed between then and now isn’t that languages have gotten better, it’s simply that Rust is trendier than any of the safer languages before them. And that’s a pretty awful reason to reason to rush a complete rewrite of the entire stack.
Source: I’m an old fart who’s been writing safe code in safe languages for 30+ years.
"Try reading the thread again" is extremely condescending.
> Rust doesn’t protect you against bugs like Shellshock. They protect against bugs like Heartbleed. But people frequently forget about the former when focusing on the latter. So absolutely we need to be considered when replacing battle tested software with an entirely new code. It’s not an unfair meme when people state this.
Like I said, over half of security bugs are the kind of bug that Rust protects you against, whereas the other half (apparently more like a quarter, per side thread) is every other category of bug put together. So unless your model is that every other category of bug will be significantly increased by a rewrite, the rewrite wins out on net.
> If it were an easy and risk free upgrade then we’d have already done so in one of the other safer languages that predate Rust. A lot of what’s changed between then and now isn’t that languages have gotten better, it’s simply that Rust is trendier than any of the safer languages before them.
Per another side thread, I do agree that the reason people are rewriting in Rust now when they weren't rewriting in OCaml 10 years ago is mostly trendiness. But I disagree about which of those two decisions is the mistake.
Rewrite does increase other types of bugs though. But the more specific point I’m making is that the domain matters massively here. A buffer overflow bug in ‘cat’ isn’t equivalent to a buffer overflow bug in ‘openssh’. If you’re worried about RCEs in ‘cat’ then the fix isn’t to rewrite ‘cat’, it’s to fix the calling pipeline. Eg is the containerisation service secure? Using Rust as a blunt hammer to fix all security bugs demonstrates a real lack understanding about how to build hardened systems in the first place.
This is the point I’m making. Not that Rust shouldn’t be used as part of the solution but rather that people need to stop bleating on about Rust being “the” solution. People love to quote vulnerabilities in Chrome and OpenSSL as a reason everything should be rewritten in Rust but fail to understand that coreutils are called in entitled different circumstances and thus have an entirely different security model.
By the way, your blog link needs updating in your HN profile (to github.io). Some really interesting content on there and I really enjoyed your writing style too. Thanks for publishing your thoughts :)
Don’t get me wrong, I’m all for writing stuff in safer languages (I personally haven’t touched C in years for that reason). But simply throwing Rust at the problem without assessing the risks is just as careless. In the case of most POSIX base utils, the risk isn’t buffer overflows, it’s people exposing internal tools to untrusted external users.
Take shellshock for example, that’s a prime example for just how careless people can be. And while there was patches to fix that vulnerability in Bash, people shouldn’t have been writing software that passed untrusted data to Bash in the first bloody place.
My point is: Security is a multi-layered problem. So by all means let’s pivot towards newer, safer, languages of it can remove an entire class of bugs. But let’s not pretend that rewriting the entire of POSIX in Rust is either practical nor the end of the discussion. And if we acknowledge it’s not, then we must also acknowledge there’s already things we can do to harden against our C++ utils today while those C++ utils are still a necessity.
This is why the Rust argument fucks me off. People expect too much from it and use it as an argument whenever other countermeasures should have been in place irrespective of the vulnerability that was inevitably used.
As for other kinds of bugs, yeah, that's inevitable, but I still prefer to have the memory safety bugs eliminated from the get go so I'd still say it's a worthy sacrifice to walk the road to safe tools one more time just for the memory safety.
So yeah, no need to rewrite tools in already memory safe languages but when it comes to C/C++ tools then yes, we need to rewrite them.
Sure that given enough resources, motivation and time, Anything is possible. But new code means bugs and uncertainty.
Some of the tools listed are not small if a one for one replacement is the target. Vim is listed??
How long before these tools are considered as matured and ready for production?
This is changing, happily, but OCaml I always viewed as kind of an elitistic intellectuals club. Happy to be proven more and more wrong lately, though!
I am looking forward, eagerly, to OCaml 5.0 and having an amazingly fast compiler with solid multicore runtime system support! It's very likely I might abandon Rust for OCaml when that time comes (8 - 12 months).
Instead, I think it comes from the fact that when it was first developed in the 90s, it was viewed in the context of C (and this attitude has carried over somewhat to the modern day where it makes much less sense). For example, building non-trivial projects with just the compiler is very painful from most modern perspectives, but it's very similar to what you have to do for C.
What you say is fair. It seems that OCaml's niche with time moved from competing with C to competing with Haskell and Rust -- at least from where I am standing. Maybe some members of the community aren't OK with that goalpost moving. That would be understandable.
But to be fair, I like OCaml more than Rust but I got very spoiled by both Elixir's and Rust's tooling -- both are excellent enablers of productivity.
Once OCaml overcomes this barrier (and introduces multicore) I am definitely going in, neck deep! :)
Another factor is that creating a language (Rust, Go etc.) from scratch lets you have nice unified tooling to an extent that is probably just not possible with languages with baggage. So I doubt OCaml will ever manage to be quite as seamless as those, but IMO it's already gone from significantly worse tooling than e.g. Python/Java to significantly better, and is still improving all the time!
* joy of coding
* (human) efficiency of coding
* performance of idiomatic code
* interoperability with C
* deployability
I'm totally sold on Rust. Yet: Why would I want to replace tried and tested tools with new versions just because they're written in Rust?
I burned myself on this when I was young and naive: Having just learned Haskell, I tried to use Haskell replacements for everything. It was incredibly stupid and counterproductive (the biggest real-world pain was clinging to darcs over git for a long time at the beginning of the DVCS wars – not because of darcs' arguable good features, but because it is written in Haskell; so stupid).
* Old tools are well established, not looking for major feature additions or changes
* Hard to add new features to old C-based system tools due to mature codebase and being written in C
* Using Rust to reach a low-bug/maintainable state more quickly than a re-write in C or C++
* Adding a bunch of quality of life features that make it worth switching
The big downside with these tools, of course, is that they are re-writes of tools that have been standard Linux/Unix system tools for a long time and can be generally expected to be present on any system for the purpose of shell scripts, services etc. Using any of these tools in a shell script requires the user to install it, along with the whole dependency graph that rust typically comes with (at least when building from source which is typically the case with rust).
Nowadays the distroless movement and the general zeal towards minimizing the overhead of a Docker image have the positive side-effect of making us less reliant on the older tools. So we can opt for newer ones (although it's very hard to beat 120KB of a classic old tool to 4MB of a new Golang/Rust tool still).
As for rewriting in Rust, the main point for me is always the memory safety. When it comes to writing programs, there's a ton of very nice languages out there already.
I don't believe performance will be significantly different if you're comparing a C vs a Rust tool though.
sd is an order of magnitude faster than sed, but it's missing some features like deleting matching lines.
You could rewrite some of those older core utils written in C to make them run concurrently...
Rust and Go thus occupy a sweet spot for CLI tools, because they have the advantage of high-level maintainable code AND being fast, easily installable binary executables.
I could not care less about the hype squad screaming endlessly about every single side project that is written in Rust because of the sake of the language. If it is useful and it makes me money then fine, other than that I ignore it.
If the tool that is being developed doesn't help me make money, then I do not care or I won't use it.
As for choosing the best programming language? I do not care unless it is the one that makes me the most money. in some use cases, it is definitely not Rust.
This is Hacker news. It's news for people who want to fuck around with tech.
Yet this site seems to be hosted by a company whose goal is purely about making money by investing in companies they find which are likely to give huge returns. i.e A VC fund.
So rewriting it in Rust for the sake of Rust is irrelevant to them and doesn't give any advantage whatsoever and won't get their attention.
I absolutely care if the BT stack of my phone is written in a safe language, because many of the security flaws that make many, many Android devices permanently unsafe (due to end of support) are caused by memory unsafety.
On the other hand, I have the suspicion that Rust is voiced a bit too much by a loud minority - I like Rust as much as I don't like Golang, nonetheless I think that for the majority of non-performance critical projects, Golang may be a better fit.
And a lot of those projects are just someone’s pet project, often written as a task for learning Rust, and certainly likely to have numerous new bugs that haven’t yet been found just by virtue of being a ground up rewrite.
As I said, I’m fully in favour of using newer and safer languages but we need to be careful not to get carried away with thinking anything new is better simply because it’s new.
Especially since one huge strength of Rust is its C interoperability, one would imagine that incrementally replacing or extending parts of the legacy tools would be a much more sound approach. But of course, the point you make does answer this:
> And a lot of those projects are just someone’s pet project, often written as a task for learning Rust, and certainly likely to have numerous new bugs that haven’t yet been found just by virtue of being a ground up rewrite.
Many of the replacements are indeed not meant to actually replace the tools. They're learning projects. That's great, but it's a bit scary that people wanna use them for anything else.
When it is unlikely that the maintainers of existing tool written in C would accept pull requests that add Rust as a dependency (and I imagine this would usually be the case), why wouldn't you start anew? It also allows easier experimentation on the architecture of the original tool, and breaking compatibility. For example, ripgrep isn't a drop-in replacement for grep and the author never intends it to be, so forking grep probably wouldn't have made it easier to develop.
I think the difference between a learning project and a "real" project is mostly how many people end up using it.
I think 40% of the complaints people have about new JavaScript frameworks are from people failing to have the analogous insight to this.
Rust solves another problem. Most of the security vulnerabilities out there come from buffer underflows or overflows.
Rewriting tools people rely on every day in a language that prevents those is not failing to learn from JS.
If you're rewriting something anyway, or creating something new then Rust should be considered. But a lot of the value of old software is not the set of features listed in the Readme. Not every area comes with its own "don't roll your own crypto" warning, but almost everything has its share of edge cases and quirks that were already handled in existing tools.
Point however is, if you have a SaaS which uses Rust, the chances for triggering a buffer underflow / overflow exploit are zero.
As for other aspects of security, Rust makes no special effort there. The main win for using Rust is to eliminate the most widespread bugs (the memory safety ones) from the get go. Everything else is still fair game and has to be paid proper attention to.
Doesn't the mere existence of unsafe make this untrue? From a quick google I can see at least one[1] so the chances are definitely a lot higher than zero
[1] https://github.com/servo/rust-smallvec/issues/252
But that is also absolutist. Instead, you might say, "In safe Rust, the chances of <insert memory safety problem here> are zero, modulo bugs in the compiler."
But that's not quite the fully story either. So instead, you might say, "In safe Rust, the chances of <insert memory safety problem here> are zero, modulo bugs in the compiler or any dependencies or any other uses of 'unsafe' in your application."
And maybe that's still not quite it. So let's try again: "In safe Rust, the chances of <insert memory safety problem here> are zero, modulo bugs in the compiler, any dependencies or any other uses of 'unsafe' in your application and any other platform specific tricks for arbitrarily rewriting memory used by your process."
That probably covers it. But it's a mouthful. It's probably best to avoid saying the chances of anything are zero. Instead, the key value add of Rust is that it flags areas of code that permit introduction of UB for easier auditing, and also enables building safe---for all inputs---abstractions for others to use. But humans are fallible, so there is and always will be an opportunity to fuck something up somewhere. What Rust gives you over something like C or C++ is a way to make assumptions that are more fine grained than, "I trust that every line of code written in the transitive dependency chain is free of UB."
Nobody is saying Rust is completely bullet-proof: if anything, our current hardware doesn't allow for it.
But it gets very close.
That is of course a very different thing!
But yes, I meant the majority of the security bugs indeed.
The language needs a way to express the lack of safety of an external library in order to clearly isolate it from the safe code sections. In Rust that's the unsafe block.
Something important to be noted that RIIR does not represent the attitude of the Rust community - actually, the instances I've read until now were from people who had little or no experience.
I think this distinction is important because there are some negative attitudes attributed to the community that I personally haven't found (e.g. "screaming" how good Rust is; committed/professional Rust developers are already busy screaming at the BCK ;)).
Thousands of new developers, eager to learn Rust and get a job, rewriting anything they can get their hands on and pushing it out for the community to use.
Endless churn ensues. See the javascript ecosystem.
But there current "github-centered" development culture makes it very difficult to tell if authors are invested in their FOSS projects are going to maintain them or not.
As the churn keeps increasing we can end up with ecosystems collapsing.
In comparison, the library maintained since 2003 by the Nebraska person is less worrying. https://xkcd.com/2347/
I exaggerate, of course, but not by much
And of course Javascript... shudder
To tick the checkbox of some HR people that insist that I provide something.
Total amount of times anyone asked me anything about them on a job interview, zero.
> And a lot of those projects are just someone’s pet project, often written as a task for learning Rust, and certainly likely to have numerous new bugs that haven’t yet been found just by virtue of being a ground up rewrite.
I'd say 6 of them are not as serious as the others, but even them have 5 contributors or more. Is this what you mean by "a lot", 6 out of 34 (at the time of writing this) ?
If the repo advertised itself as a curated list of interesting Rust alternatives to standard tools then I probably wouldn’t have commented. But it said “replacements” and that suggests a higher level of maturity and community support.
I think there's an obvious case for being aware of rewrites from high-level memory-safe languages (not just Haskell, of course) to Rust, although the case is obviously different from rewriting C/C++ code. Besides the obvious improvement in performance (Rust being on par with C/C++ itself) it gives us a better idea of whether Rust is actually missing features from these languages in a way that meaningfully impacts maintainability.
And the ONLY way C/C++, that cost BILLONS on bugs alone, is replaced is that people rewrite things on something better (and Rust IS better).
And this start for fun.
And it could get new bugs? yeah... but it also NOT continue the old bugs that C/C++ cause, some that will be erased of existence (like nulls) for the rest of the life of the codebase.
And this is the point. Logic bugs eventually are squashed, but language design mistakes are nearly eternal for the life of the codebase.
Newer tools CAN make use of new information in a better design, but it doesn't always happen.
I have been doing it for 20 years, and all the time Pascal had null-safe strings and C-like performance. And the strings even had mutability xor aliasing with copy on write
The more interesting question is - why is that? Sure, Rust is memory safe, but I suspect the real reason people built high quality software in Rust is because cargo is good and because minimizing copies is part of the culture around the language. This means that libraries are cheap to use, and as a result programming feels productive, because you don't have a reason to reinvent the wheel (see arrays in C, or various Boost libraries that have been moved into stdlib C++). For example, compare the cost of Python's click to Rust's clap. Or look at serde.
I find it more interesting to ask the question "why do people spend so much hobby time writing interesting programs in Rust?".
This is a somewhat comical statement in a thread about "Rewrite it in Rust."
sed vs. sd, for example https://github.com/chmln/sd The semi-lovable but reliable old jalopy overhauled to match modern practices.
My theory is that people that like Rust want to work with Rust, and for that the best way is to prove that it's a good <language> replacement in <space> multiple times. By making modern and good versions of classic CLI tools, they prove that it can replace C in that space. By making good new CLI tools, they prove that it can replace Go (that replaced Python/Ruby) in that space. Some people are trying to replace JS on the frontend (Yew), some people are trying to use it in the backend (Rocket, Actix). Some people working on chromium were (I don't know if they still work on it) trying to prove that it can replace C++/a GC'd language while having the benefits of both. There are lots of examples like that.
Rust often ended up as the most loved language on the stackoverflow survey. I don't know if it's intentional or not, but people in the community seems to understand than marketing is important, and proving your claims is too. And the best way to prove your claims is to deliver working software that's at least as good or better than what was previously made in <language> in <space>, to prove that you could use Rust here.
Of course as you said, cargo really helps here. I'd say other big players are the culture and the type system. The type systems acts as a first layer of documentation and helps ship better code. The culture is also heavily focused on documentation and welcoming newcomers. Compiler errors are a joy to read, rustfmt is a nice addition, clippy is also very good, and people tend to be very nice and welcoming in the community.
What I like with this explanation is that is also in part explains the rewrite it in Rust (RIIR) obsessed people: they see how the community act, fail to understand that delivering working code is the most important part because it's less visible than the "pure marketing" talk, and so blindly repeat the "pure marketing" talk.
Probably for the same reason people spend so much hobby time writing interesting programs in other languages: because they either already have experience in that language and like it or they’re looking for an excuse to learn a new language.
Personally I don’t find it an interesting question to second guess why people have hobbies. Sometimes people enjoy challenges because they’re easy, but sometimes people enjoy challenges because they’re hard. My wife relaxes by watching TV. I relax by writing OSS. Everyone is different.
Do we actually have factual evidence that people do write high quality software in Rust.
Something showing the average rust program is "higher quality" than the average C++ program?
I'm thinking rust would come out on top of those, but would be interesting to measure!
Maybe just as a learning experience or to be able to say "I know $hypedLanguage." Nothing against rust on my side, but at least for me, time spent on writing something in a language new to me is in no relation to my perceived usefulness of that language.
I personally believe that people new to programming love rust because they get compiler feedback instead of runtime errors. Given what is currently en vogue, rust does stand out there. OTOH, if looking at some old, boring languages, you already have that. Minus the hype.
At the same time - I see the list as a testament to enthusiasm about Rust. People use it. People try to reimplement things. Also - it is likely that one day one of these reimplementations will take over (due to performance, functionality, or simply - because of being maintained).
by potentially beginners.
If I wanted my IDE to execute something like ShellCheck on every save, or even every few character presses, I feel like I'd much rather not have to spin up a big runtime.
You're perceiving a sentiment from the list that I just don't get. All it is is a list of alternatives written in rust. The fact that they're written in rust, is really only valuable to people who are already interested in rust.
this pretty much sums up Rust atm. There is a lot of hard-core evangelizing which is off-putting imho. I still like the language but wonder if a rewrite for rewrite sake will convince the majority, considering this is a community effort and not a large org controlling the future of the language. Especially that there are other ecosystems like Zig or Nim that are pretty cool too and fit into a similar niche. Hard-core evanglism just feels like advise from somebody who's only tool is a hammer.
on the other hand I do remember the amount of marketing and advertising budget that Sun and later Oracle pumped into Java to make it fly in the late 90ies. They made deals with University professors to make sure the language is favored. It was a hype-shit machine unmatched by anything we've seen since (and when I use the word shit I mean your browser being hi-jacked and reconfigured with a new toolbar and a search engine simply because you installed a JRE. not too different from how malware behaves). It's not that early Java was terrible (though it certainly over-promised and under-delivered for over a decade) but if Java would have had to grow organically back then like Rust/Nim/Zig do today, then Java wouldn't have stood a chance. There were even ads at airports and on TV. Ads for a programming language during a time most people watching TV didn't know or care what computers are for!
So I'm quite glad that this is just a couple of people who are overly excited about what they do. We've had a lot more asymmetry with larg-corp sponsored garbage. Go Rust!! (and Nim and Zig)
Regarding Java, it had a major killer feature that triggered adoption even before the whole marketing wave started.
It was actually portable, not like trying to write C or C++ code in the middle of ongoing standards, compilers still stuck in the old ways, POSIX on paper and on actual UNIX clones not being 100% the same thing.
It was like a fresh air of portability even with an interpreted runtime, JIT only came in by 1.2 days.
Is this accurate? And if so, is it super easy to write rich CLIs, with tabulated, coloured, interactive TUIs in Rust? Are their some common, widely used libraries driving this trend?
The rust community leadership made a strategic decision to pick 4 areas to suggest focus on. CLIs was one of them. (Scroll to “build it in rust” on https://www.rust-lang.org/)
I think this choice and many others by the rust leadership merit a case study in community technical leadership. Or at least a blog post by the author of https://www.kalzumeus.com/2014/04/09/what-heartbleed-can-tea...
Regarding TUIs, have a look at thes screenshots and example codebases for these libraries:
https://crates.io/crates/tui
https://crates.io/crates/spotify-tui
https://crates.io/crates/cursive
TUIs are in many ways more ergonomical, in particular they make experienced users more productive and they can help reduce RSI by limiting the need to move the mouse.
Now if that's the case that's also cool. I also love new ways of doing old things and we might sooner or later have something that looks like Redox on Linux/BSD kernels.
A list of "software replacements that are considerably better[0] than most well-known counterparts"
[0] disclaimer for nitpickers: obviously not for 100% of cases, but what we could say for the "general usage". E.g. I'm sure there is someone out there that claims an obscure feature of grep is not replaceable for their workflow... but for the other 99% cases, ripgrep is fantastic.
I have lost counts of how many times my "brew" or "pacman" failed to install something. It's much worse if it's Python; and things that work in macOS are not guaranteed to compile in Linux, and vice-versa. On the other hand, my "cargo install" almost never fails.
For that reason alone, I welcome any initiative to write something in Rust.
The last straw for me was being unable to do something simple like install vey common Perl packages natively via CPAN.
https://crates.io/policies
These days, I prefer Linux distributions or BSD ports, though Homebrew seems to be pretty casual as well.
Apart from that, most modern languages come with a package manager nowadays. Why do you like rust more than go? Or nim? Or elixir? Or crystal?
But I never, ever had it just fail to install a package, and I don't remember much fuss with dependency resolution taking manual fiddling, either.
I'd love to hear your pacman horror stories; gimme the hot package manager gossip :D
https://matrix.org/docs/projects/server/conduit
We are currently seeing a bit of a command line renaissance. The justification for this is a 'rust rewrite', but as many have pointed out, just because something is in such-and-such a language doesn't make it good.
However, what I tend to find with the new rust CLI tools is that they bring with them modern design sensibilities. They can use better conventions, better defaults, assume things like colour terminal emulators (or fallback to non-colour if not outputting to a terminal).
The experience of using them is often much different to using the old GNU versions made in the 80s which reimplement tools made in the 60s and 70s.
I'm not saying all the other criticisms in this thread aren't valid, but I think it's worth highlighting the positives of the tools mentioned in a list like this rather than just throwing negativity on it.
Having non-POSIX applications used in these scripts means that portability is limited.
Yes, I know that with aliases I can override this, but I’m not convinced yet that straying from “this will work on everyone’s machine” to “set an alias for X, and X will probably be an appropriate stand-in for Y” is good enough to guarantee zero-issue compatibility.
If we see them as "immediate drop-in replacements" then the value of them must be that they're identical in every way and the value in them would be in the reduced maintenance overhead for the developers, with maybe a small performance boost if we're lucky. This would be fine, but it's not something that we'd notice if our distro went from using C `grep` to using a Rust rewrite of `grep`. We certainly wouldn't need a list telling us what replacements are available.
However, if we see these as spiritual successors, ones that perform the same function but aren't intended to be aliased to be precisely the same thing, then it makes a lot more sense. We're not looking for backwards compatibility, portability or identical feature sets. We're instead being offered a smorgasbord of options on what ideas we want to embrace and which ones we would want to leave behind.
If ripgrep is better in all use cases than grep, then eventually ripgrep will be the one being used in the scripts organically. We shouldn't be dismissing ripgrep out of hand just because a colleague won't have it on their system. Or, at the very least, we should be noting the good points while also noting that we won't be using the tools ourselves for portability reasons.
Same with POSIX tools and their replacements: `find` is perfect when I know I'm going to run a script on a bunch of similar Linux hosts where I can't count on `fd` being available, but if fd is around I'm definitely going to use it at the shell prompt over find. As a side note, if you're doing cross-platform work, the new tools are better in some ways than the things they're replacing. Sure, `fd` isn't going to be installed by default on our out-of-the-box EC2 instances, but once it's installed at least it has the same arguments on Linux as on BSD and macOS. If I'm going to have to install GNU utils on a system to manage it, it's no extra work to install fd at the same time.
# `rg` / Ripgrep (`grep` replacement)
`rg` uses `ack`-semantics (i.e., automatically searches files recursively).
Here's what my search workflow looks like, this is the main way I program:
1. `rg something`
2. Too many hits? `rg -l something` to just list matching filenames
3. Hopefully based on that we can narrow enough using a glob: `rg something -g "*.m"`. Otherwise write a regular expression to get more specific matches.
4. After I have exactly the set of matches I'm interested in, I pipe the results to a text editor that can interpret grep-style output (e.g., quickly jump between results, I usually use Vim).
The key here is that the command-line excels at iteratively refining a command based on output, and that's exactly what we're looking for to use search effectively. `rg` provides a more ergonomic UI than `grep` (recursive, automatically ignoring version control files, including ignored), which is really important if this is the main command that you're running all-day, which it is for me.
# `fd` (`find` replacement)
This advantages here are the similar as for `rg`, quickly being able to recursively find a file by name, a problem that similarly benefits from iteratively refining a command.
# `bat` (`cat` replacement)
`bat` is just the best way I've found to quickly view a file from the command-line, it adds syntax highlighting and line numbers, which are both invaluable.*
This doesn't automatically mean we would deploy this stuff when going to Mars; so I don't think some of the sentiments expressed here are the fault of the enthusiastic reimplementers.
(Looking forward to your reimplementations of coreutils in R5RS Scheme.)
And currently (with Nim 1.4 and its memory management), it's moving towards Rust's field. Their performance has always been in the same space, especially regarding cmdline tools, where latency is more important than most other tools.
And because Rust is quite trendy nowadays, I guess a lot of people wanted to try it out, and in some cases, that exercise turned into a viable product. Nim never got the same popularity.
No, I don't think you will find a high correlation between language and tool quality.
[0]: https://github.com/ibraheemdev/modern-unix
borkdude has been releasing for some very cool stuff using this stuff. Here's a Clojure/Rust combo:
https://github.com/borkdude/clojure-rust-graalvm