It's gonna be interesting to see how it will affect the kernel ABIs. High level languages have features that produce nightmarish binary interfaces and it's unrealistic to believe existing code will be able to interface with them. So will the code be restricted to a minimalist subset of Rust?
Yes, but the benefits of Rust won't extend beyond those symbols. Also, certain features of Rust that have no C equivalents cannot be exported. Will the gains in safety be contained entirely inside the modules that are written in Rust?
> certain features of Rust that have no C equivalents cannot be exported
Right, only "repr(C)" types can show up in "extern" APIs. So no data-bearing enums, no Vec<T>, etc.
> Will the gains in safety be contained entirely inside the modules that are written in Rust?
I think for some things the answer is yes, and for other things the answer is no.
If the issue is that callers might retain some pointer or descriptor longer than they should, leading to a use-after-free or something like that, then there's probably no way to fix that with an API-compatible Rust reimplementation. What safe Rust would prefer to do is either 1) force explicit lifetime constraints on the caller or 2) replace the pointer/reference with some sort of smart pointer, which can either keep its referent alive or at least safely detect the fact that the referent is dead. But of course a C-compatible API has no way to represent (1), and (2) would presumably be an incompatible API change, so the Rust reimplementation can't really solve this issue. (Though if both Rust and C APIs get exposed for the new code, then new Rust callers could benefit from (1).)
On the other hand, if the issue is that the caller might pass in some invalid indexes or other metadata, leading to an out-of-bounds array access within the module, Rust can help quite a bit. If the array in question is actually owned by the module (and not just a raw pointer to unknown memory) then bounds checking is required one way or another (unless it's either optimized away or explicitly skipped with unsafe code), and a Rust reimplementation will probably not suffer from out-of-bounds issues. Raw pointers into the caller's memory are harder to check, but even there if they can be converted to a safe slice immediately inside the API boundary, some subset of out-of-bounds accesses can still be caught.
Looking at these two different kinds of bugs, I think it's interesting to think about what "inside the module" means here. In both cases, the UB we want to prevent (either use-after-free, or an out-of-bounds access) is "caused" by the caller's mistakes outside the module, but "occurs" on lines of code inside the module. In the first case, Rust can't protect itself against a bad (C) caller, but in the second case it might be able to.
The primary goal is to write in-tree drivers in Rust, which can still be loadable modules, but they'll be built to whatever ABI that particular version of the kernel uses. And even for out-of-tree modules, you have to recompile them against the specific kernel and ideally with the same version of the compiler etc.; ABI incompatibilities across C compilers are rare, but not unheard of.
Linux will definitely need a very curated setup to reap the full benefits of idiomatic Rust.
Right now both alloc/std and even basic language constructs like indexing into a slice can panic. With little help from the tooling to guard against it. This makes sense when considering that Rust evolved as a language for writing somewhat higher level applications like browsers.
While Rust and especially the library ecosystem is generally great at forcing you to handle application domain error conditions, that mostly doesn't extend to these lower-level concerns.
But I'm hopeful that this can improve the entire ecosystem over the medium term. With things like lints that forbid panics, more focus on custom allocators and fallible allocations in std and the library ecosystem, etc.
edit: just to clarify on the slice index panicking: my point isn't that the C situation is better, but that a big reason for moving to Rust would be to benefit of a safer, more convenient language that doesn't require such extreme carefulness as C.
So the language should help in preventing panics, or making it obvious where they can occur. This is perfectly possible in Rust by using accessors that return `Option<T>` or iterators. But the tooling for such use cases can be improved, for example with lints, and the API surface can be customized to mostly prevent such conditions with the type system.
The kernel uses a non standard `kmalloc` to allocate memory anyway, I don't know how they plan to integrate Rust into the kernel but I would be very surprised if they imported any of Rust's std as-is.
Actually most of the time you probably wouldn't want a Vec that reallocates automatically in the kernel, you'd probably want a dynamically allocated array of fixed length the majority of the time.
Yes, indexing into arrays in C (well, really, de-referencing pointers) in ways that are out of bounds will in the best case result in a fault which results in a BUG() invocation in Linux.
Panicing on invalid slice indexing is a non-issue.
> Panicing on invalid slice indexing is a non-issue.
Well, it's less of a issue that what we already have. It would still be nice to be able to tell the compiler: "Issue a diagnostic for any [some qualifiers] panic that isn't optimized out by unreachable code analysis.".
> There is no index checks in the current kernel C code at all. It is better to panic than violate memory safety.
This isn't the question here. malloc (or other memory operation) failures should never panic. According to Torvalds[1]:
> Allocation failures in a driver or non-core code - and that is by definition all of any new Rust code - can never EVER validly cause panics. Same goes for "oh, some case I didn't test used 128-bit integers or floating point".
Kernel code is a bit quirky in this sense that it will always favor system stability (and expect proper errors to trickle down) instead of crashing. Also, Linux runs on plenty of low-memory embedded systems so failing on a malloc may not even be a particularly rare occurrence.
The parent post mentioned both allocation and indexing. Both have the potential to panic in Rust programs based on the full standard library. Allocation needs to be dealt with, as you mention, but out-of-bounds indexing could likely continue to be a Rust panic, which could be translated to a Linux BUG() call.
I suppose this can’t be stopped. I hope someone has the resources to curate a working fork of the Linux kernel that doesn’t have all the “improvements” it’s about to get.
The Rust compiler is very slow and has very high memory requirements. Compiling the Kernel on e.g. older Raspberry Pis is already barely possible and will potentially become completely impossible once rust enters the kernel.
There are many MCUs that run Linux but definitely are incapable of compiling it.
IIRC most of the build time of a rust program is spent not in the rust compilation proper but in LLVM doing code generation and optimization. I hope both LLVM will become more efficient with time, and rust will learn to pass it such IR that it can process faster. There are obvious incentives for both of these things.
But of course it would be sad if the minimum requirements to compile the kernel grew again, and now excluded older RPis.
gcc and clang are pretty slow on the scale of all programming languages. Rust is basically the only thing that modern C/C++ compilers are faster than. Should we abandon C and C++ for something else then?
Rust is for writing blog posts about how it’s the most liberating, empowering and inspiring experience of your life. Boring technology like C and Java are for writing software.
There’s already an OS completely written in Rust called Redox. Nobody uses it. I’m not entirely sure you could do useful work with it, which is contrary to the Rust marketing team’s talking points that Rust code is guaranteed to be fast and guaranteed to be bug free because Rust is just so darn great and the compiler authors so wonderful and smart. That there are basically no success stories other than a couple side projects after a decade of marketing is of no consequence.
Don’t read the blog posts. Go to the repositories. Check out the crates.io landfill. Grab the Rust book and try to write anything more significant than a CLI toy. I promise that you will know exactly why nobody uses the language in anger.
The Fuschia team, who are partly responsible for the Rust myth don’t even use it in their kernel. But they want it in Linux. I wouldn’t be shocked if this was all some conspiracy by Google and Microsoft to make a personal computing unusable so that everyone has to use their cloud offerings.
> I wouldn’t be shocked if this was all some conspiracy by Google and Microsoft to make a personal computing unusable so that everyone has to use their cloud offerings.
So you would rather believe in a ridiculous conspiracy than admit that large companies might just be choosing Rust for some valid reasons. Got it.
Every sufficiently large company is using or has used literally any XYZ you can think of, whether Rust, Forth or even Haskell. Therefore, knowing that there are projects at Google, Facebook, IBM, and Apple using XYZ tells you less than nothing. People pointing at them are either deluded, or seek to delude.
A small company that has become a success based on a whole XYZ stack may count for something, depending on how ambitious the company was, how hard it was to do, and how much competition they face. Lots of companies could succeed equally well with Rust, JS, Java, C#, Kotlin, Python, Ruby, Erlang, or Perl; those successes don't count. Lots more potter along, neither succeeding nor failing, exactly. Those don't count either, unless you want to give Rust credit for their not exactly failing.
And, many fail. We are never told about those. They might well count, but there are a lot of ways to fail.
It is ridiculous to believe in a Google conspiracy to outcompete Linux by getting it to incorporate Rust, because a conspiracy requires a crime, and that would not be one. But anyway Google doesn't have that much influence.
Firstly, I disagree with your argument about language adoption being irrelevant. When a language is picked up by a serious player, it helps signal to other users and companies that there might be practical benefits to using the language. Two good examples that come to mind are Erlang/WhatsApp and OCaml/Jane Street.
Secondly, there is a huge difference between a language being used in general and a language being used for serious, visible projects within a company. Think: writing a small utility in Zig at company X vs. rewriting the backend of a core product in Zig and pointing to measurable gains in performance and reliability.
> It is ridiculous to believe in a Google conspiracy to outcompete Linux by getting it to incorporate Rust [..]
Great, but you should be telling that to the user I was replying to.
I did not say language adoption is irrelevant. The examples you cite are exactly the kind I spelled out as being possibly relevant. So, you are agreeing me while saying you don't.
Similarly: you then agree that "BigCo uses XYZ" is, in fact, not compelling. A big company betting its future on XYZ would count, but no big company does that. Betting a substantial new business unit on XYZ could count, but that is not cited as "BigCo uses XYZ", that is "BigCo relies on XYZ for UnitX to succeed", a possibly stronger statement, according as how important UnitX is to BigCo.
Calling a claim ridiculous needs reasons why it is ridiculous, which I supplied, and you did not.
No, I said that there is a difference between using a language and using it in a core product or area within the company. In the case of Rust, it is already seeing use in critical areas within several of the large tech companies. This is an excellent signal - in fact, it’s a much better signal than a small company relying on Rust exclusively.
I find that a better signal is simply the quality of the tool or technology.
It's like surfing. Some people want to see hundreds of surfers on the wave before they start paddling out. That's fine. There's nothing wrong with that. But I personally believe that a better way to surf, especially when it comes to technology which tends to have short cycles, is to learn how to spot the wave as it builds, while it's still a swell, before it breaks. I've seen time and again that there's incredible value to be had if you can learn how to do that.
One of the best examples of this value being that you're able to hire really talented people, because they form the critical mass at the beginning of a successful wave. Once the larger corporates come in, this advantage disappears.
I think the challenge would be in defining "quality" in a way that allows you to compare between multiple different technologies. If the definition ends up aligning with what large companies look for when adopting a new technology, then the signal ends up being the same.
I definitely agree with the notion that the early adopter group is more often than not more talented than average. But, I'm sure companies also take this into account when deciding whether or not to pick a new technology.
Is it really a challenge if "quality" is a measure of being fit for purpose? You don't necessarily have to compare across every technology if you are able to recognize a great tool that does what you need.
"If the definition ends up aligning with what large companies look for when adopting a new technology, then the signal ends up being the same."
And even then, the signal is only the same, but it's not worse. Whereas if large companies don't always optimize for quality, but have other incentives at play, or if they are less nimble, or less efficient, then looking for what large companies look for might actually end up setting you back with worse signal. So there's no upside I think.
The value of the tool or technology is what is in question.
A notable project choosing it and succeeding is signal. But there is a lot of noise. An individual judgment about the value is noise unless that individual has demonstrated a pattern of picking winners early and correctly.
Early adopters are often smarter than the average bear, but are also much more likely to jump ship the moment the next fad breaches. That might be OK if you only need them for a short time.
My full time job is writing Rust for large-scale production systems. The language is great - and as an organization we're doubling down on our investment in Rust this year.
I mean, you could click through to the profile and find out that they work for Discord. Or you could ask politely.
As for purpose, there’s this blog post [0]:
> At this point, Discord is using Rust in many places across its software stack. We use it for the game SDK, video capturing and encoding for Go Live, Elixir NIFs, several backend services, and more.
>There’s already an OS completely written in Rust called Redox. Nobody uses it.
I find it would be very difficult to convince anyone to use any other OS than the currently mainstream ones. You should instead see this as an accomplishment of what Rust can achieve.
I also think you are undermining Redox OS developers efforts, which is kinda sad.
>Rust code is guaranteed to be fast and guaranteed to be bug free
Nobody claimed this, what people claim is that it can be as fast as C, and it has some safety guarantees. You can still have bugs.
>Rust is for writing blog posts...
What's so wrong about people being excited about something? Personally I find it awesome that a programming language can bring so much enthusiasm to people. In my opinion, what you are seeing here is people being excited about progress being made.
>the compiler authors so wonderful and smart
I sense some envy here, people achieved a great thing through effort and collaboration, we should celebrate this.
It's the foundation for Fargate and Lambda, so a <<ton>> of AWS customer code runs on top of it.
Considering Microsoft, Google, basically every big company backs it now, I'm sure there are other infra projects using it. They're just not all very visible because these companies only open source a small percentage of their infra.
Better yet, read the epic rant that Zed Shaw made about Ruby. It is certainly juvenile, but I don’t think it’s baseless. Now, consider that the core members of the rust community are exactly the same people he was talking about when rails was the new shiny.
I don't think that's true. Yehuda and I would be the two people you're talking about (And Yehuda doesn't work on Rust anymore) and we both, as far as I know, appeared a few years after that rant.
I wonder if Zig [1] would be a better fit than Rust for the Linux Kernel. I haven't touched systems programming for a decade, but looking from the outside, it seems like the author put even more effort towards C ABI compatibility than the Rust team.
Zig does seem like a closer match to C and so could have been better suited than Rust. It doesn't provide the same safety as Rust, so it's not as big of a payoff, though. Also, it's still fairly new relative to rust.
The kernel project distributes several userspace utilities and new ones of those might be worth writing in Zig.
"Dead man walking" is quite harsh. Zig is a one man passion project. Those are awesome. Rust is becoming and industrial scale tool. Those are awesome as well.
However, each has their place. One man passion projects should not be used in anything mission critical. They totally should be used for fun, enjoyment and inspiration, and they may potentially grow to industry scale.
A bit too harsh, I admit. Look at something in-between (project length / status-wise) like D. Similar story. Critical traction or a passion uphill road.
Zig is still quite early. Not sure what eventual traction will be. As is there seems to be lots of love from lots of hacker / demo / oldschool types which may snowball into an industrial scale ecosystem in this or some parallel universe (i.e I would not bet either way just yet).
After the initial commit in the "group" repo, development of the language has switched to a team instead of a single person, with the introduction of additional contributors. This holds even if you only look at the pre-1.0 days which established the core concepts of the Rust language as it stands today (the original language when the team started to work on it was a bit different to 1.0).
The work since 1.0 can be classified as a massive refinement project, which also does not have a single leader. Zig on the other hand does have a single main contributor/maintainer/creator:
Yes, many projects have started like that. Projects can grow and mature. But there really is no crystall ball for that and you should only choose existing industry scale tools for production projects.
There is lots of valid criticism of Zig. The fact that it's run by a single person is not.
hostapd is maintained by a single human. They are responsible for about 69% of all commits in the git repo, 55% if you look at the last year. It's still the most widely deployed WiFi implementation of the world. https://www.openhub.net/p/hostapd/contributors/summary
Changing WiFi stack is less of a problem than changing the implementation language. The perceived fragility of a an element of architecture should be inversely proportional to the effort of replacing the said module chosen. Large effort to replace an element (like the implementation languag) means the element should be of low fragility.
This is not an argument against Zig or that it would not become an industrial scale tool. It just does not feel "lindyproof" yet for real work.
But is _now_ an industrial scale tool. Any project can mature to industry scale, but a language needs to be currently industry scale for a current industry deployed project. You can't choose "possibly in the future" industry scale language for current project.
Individual insight is often much better than a design by committee, but a language needs a quite robust ecosystem of support from industry bodies at some point to become a viable tool for generic industry scale development.
Zig is newer than Rust, but the Zig Software Foundation was in fact founded before the Rust Foundation, and already employs several core members. For example, Jakub Konka who resigned from Microsoft to work for the ZSF (https://ziglang.org/news/jakub-konka-hired-full-time/), and whose linker was the first to enable cross-compilation to Apple Silicon, and is already making Rust's cross-compilation just work (https://news.ycombinator.com/item?id=27245369).
Inspiring how such a young language can already be so far ahead when it comes to something as vital as the compile chain.
> Funny how the "toy" is ahead of the "industrial scale tool" when it comes to something as basic as the compile chain.
It's not funny if you understand the relative priorities of the two teams. Rust tends to prioritize working with the system, so all of our default configurations use the system linker. Zig doesn't have the same constraints.
This feature is awesome, and I would love to see it in Rust as well, but like, it's not surprising.
No, Zig absolutely has an awesome compiler. It's not "a toy" or a "one man passion project" and it's condescending to reduce it to those terms, as if it is not also somehow "industrial scale".
I absolutely agree that it is awesome. That doesn't mean that returning condescension with more of it is helping the conversation, or getting said awesome compiler the respect it deserves.
I think it’s worthwhile for researchers/hobbyists to develop toy kernels in Zig. Using it for Linux is just plain irresponsible. The bottom line is that, unlike Rust, Zig is unstable and unsuitable for production software. If real money or human lives depend on the software (as is the case with Linux) the choice of programming language should be somewhat conservative.
This is not a criticism of Zig, nor is it an insinuation that Zig is unable to write correct software. It is a reflection of that fact that compilers are very complex.
That is a compelling argument for no more C in Linux. But a compelling argument and $5 gets you a cuppa.
Any argument that Rust, by contrast, would be OK in the kernel applies at least as strongly to C++, because C++ can read kernel C headers, where Rust will need hand-maintained equivalents. (Some automation is possible, but not enough.) The FUD Linus flings about C++ applies equally to Rust. (Anybody who says Rust is less complicated does not know Rust well at all.)
But argument validity is not what carries the day: bad arguments tend to fail, but good arguments get pot luck. Ultimately, it comes down to biases of those with power to decide: C seems to them like a familiar quantity (full of risks, but favored risks), while Rust happens to be hot just now.
In three years' time, Rust could very well not be so hot, and a new new thing might take over its social role. Zig could conceivably be that thing. (Zig isn't "safe", by Rust's definition, but by then something else may be more important; anyway the lack of safety does not disqualify C.) The new hotness will necessarily already exist today, to be mature enough to consider then, but not necessarily one that immediately comes to mind. Being different from Rust could be, by then, among the new hotness's attractive qualities, much as percieved differences from C++ boost Rust in some milieus.
> In three years' time, Rust could very well not be so hot, and a new new thing might take over its social role. Zig could conceivably be that thing.
Zig is totally different from Rust. For starters, it does not have a safe subset of the language that prevents an entire class of bugs. It's slightly nicer C. There are plenty of languages like that.
Zig's approach to safety is quite different to Rust's. As someone who has done a fair amount of security work, I actually prefer Zig's design decisions. I believe they make for safer, simpler, more readable programs. Attackers love complex systems, and the majority of low hanging fruit out there is not for lack of memory safety.
Also, the more familiar one is with a subject, the less one tends to conflate it with "plenty" of others. Zig is a completely different language to C (while also targeting the same systems space where you would need "unsafe" Rust to get anything done). In fact, it's hard to find too many languages that come close to Zig's comptime, plus the entire array of safety features offered by Zig. Your argument is a weak strawman at best.
More importantly, you can cause exploitable crashes in Zig programs while in Rust you need unsafe for it (at some point in the program).
And yes, each Rust codebase has unsafe somewhere, but there are quite many codebases that don't have a single line and instead use dependencies that encapsulate the unsafe in a safe to use API.
How about checked arithmetic? I'm familiar with the post you linked. Again, as someone who has done security work, I was fascinated by footnote 3 under the table:
"not by default, but available via compiler setting or by linting against unchecked arithmetic".
Would you say also that Rust's macros offer a stronger type system than Zig's comptime?
There is this misconception that Rust will make systems secure simply "because memory safety". But this overlooks the most foundational point of systems safety: simplicity. Most failures or exploits (two sides of the same coin really) originate through excessive dimensionality, a point where Rust's complexity unfortunately dwarfs Zig's.
Overflow panics in debug, two's compliment wrapping in release. You can choose a more specific behavior by using specific APIs if you want something specific for overflow in release. In the future, it may be panic-ing in release too. We'll see if the checks get cheap enough.
I don’t want to get into a Zig-vs-Rust debate (nor should anyone!) but I did take some exception to this:
> [Zig is a] slightly nicer C. There are plenty of languages like that.
Are there “plenty of [actively supported] languages like that?” The design space of relatively simple, typesafe, imperative native programs strikes me as rather unexplored since the early 90s. At least I can’t think of many languages I would consider a nicer alternative to C - indeed actual C alternatives like Pascal and Fortran are arguably less nice. Maybe this is my ignorance.
There are various languages “between” C and C++ in terms of abstraction and complexity, and the space of “somewhat C-like” languages is heavily explored (D, Java, C#, Objective-C, and in a different sense Ada or Swift). There are plenty of alternatives to C++. But Zig is one of the only ones that’s not dramatically more complex than ISO C.
Happy to be corrected if I am missing something obvious (I mostly do functional programming).
Zig definitely looks interesting, but it hasn't been nearly as much battle-tested yet, so it's a bit early to know whether it would be useful (or even usable) in the context. It is possible that opening the kernel to Rust would also open it to other system languages.
I'll agree with other commenters that Zig is no where near mature enough for Linux yet. However, I definitely think that it will be a better fit than Rust in a few years. It's a fantastic C replacement.
(I'm thinking of making HN's software render links to previous threads in this format automatically. If anyone can think of a downside to that, please let me know - maybe at hn@ycombinator.com so as not to take this thread off topic.)
If I’m not mistake, the BPF interpreter is gone. All those programs are JITed. I don’t know BPF well enough to know, but I suspect you can call kernel C functions from there.
Overall though, your overarching point still stands that the Rust development is big in that this marks the first time actual Linux kernel code will be written in a language other than C (or platform-specific assembly).
The BPF JIT is still optional. When it is enabled, there is a separate option to disable the interpreter and always use the JIT (CONFIG_BPF_JIT_ALWAYS_ON).
Running eBPF does not involve a JIT at all. The code is compiled to native, if it is, exactly when it is inserted, not when it is first used. Most uses insert code once or at specific junctures, and execution mostly happens at points widely separated from those.
People talk about the insertion process as if it were JIT just because it reuses infrastructure originally invented for JIT.
Yes the kernel is entirely written in C and assembly, there’s a Linus rant about why that is and why he doesn’t want C++ in it. Curious as to what changed his mind about rust
No. eBPF is C++, or Rust, or C, or D, or pretty much any language that can be compiled to LLVM intermediate code and doesn't necessarily rely on a runtime.
What can be done in eBPF code is somewhat limited, but nowhere nearso as you might imagine from the restrictions imposed on eBPF programs. Things eBPF code is not allowed to do directly are routinely done in functions eBPF is allowed to call, or in the code that calls the eBPF program in a loop.
Odds are it's not going to happen. You just need to read between the lines. There's a lot of handwaving from the Rust side and very little of actual importance.
You gotta remember that most of the heavy hitters in Linux Kernel development haven't even commented yet. When the Rust side presents something mergeable, the massacre will begin.
It's just my intuition built by reading LKML and following Linux kernel development for 20 years. There are a lot of folks waiting for something mergeable before a torrent of criticism (and I'd imagine outright NAKs) comes down.
I think a lot of Rust people read Linus's "I don't hate it" as explicit approval. But anyone who's followed LKML knows that this is not the case.
That’s a surprising read given Linus’ general overall positive sentiment to the preliminary interactions. Yes there are concerns but it seems like the team involved is working on solving them & believes it is a tractable problem, not to mention strong support from the Rust standard library maintainers. There are always risks but I wager this is far more serious than past attempts like the C++ one which never had any significant commitments from the standards body to support kernel needs. That Rust is building a language that can fit wildly different use-cases (embedded, kernel, web servers, systems programming, etc) without serious sacrifices for other use-cases is truly impressive to me.
Even if Linus wants it, and this is a big stretch since "I don't hate it" is best characterized as neutral and not positive, there are others who have a say.
Remember what happened with KDBUS? Gregkh fought long and hard to get it merged but in the end it didn't happen because too many people didn't want it.
I agree, Linus is likely pretty neutral on the topic. In the world of kernel development though, feeling "neutral" about something is about the highest praise it can receive. In other words, there's not much else that Torvalds could do to welcome Rust developers besides writing a kernel patch in the language himself.
Given the current target seems to be more or less Rust for loadable device drivers, even if it never gets merged, it could develop to the point where out of tree modules could be done consistently in Rust.
> You gotta remember that most of the heavy hitters in Linux Kernel development haven't even commented yet.
There is nothing to comment yet. I've read those driver, it seems unconvincing for me despite me being a rust fanboy. Linus said about panics, I agree completely with him, and with the possibility to get rid of those panics. There is unsafe code where it shouldn't be, for example driver uses unsafe to create and to initialize a semaphore. I believe this unsafety could be removed by using Pin or something like, but when it removed, then it would be the time to comment. And I mean it. Kernel code heavily uses list_head, for example, which is PITA with rust. How to deal with it? I'm not sure that the rust will be better than C, just because it would be rust used in a C style, but it doesn't work. When I met Rust for the first time, I tried it with a C style of designing programs. Many people do, and it just doesn't work.
For now it is impossible to judge how rust will help, we might believe that it will help, because rust is a safety-oriented language, blah-blah-blah. But it is a belief, not something that we could touch, play with, see how it helps and how it obstructs.
They haven't officially, but the last layoffs gutted the teams working on rust to such an extent that it seems it's no longer a prioritization, at least if they remain for the same amount of funding.
What do you mean "they haven't officially"? The link I posted states "the first major Rust components were shipped in Firefox 56", is this not official?. Do you believe that is wrong? (I'm just trying to reconcile what I'm reading in these comments with what is stated in the link I posted.)
> After Mozilla laid off all Servo developers in 2020, governance of the project was transferred to the Linux Foundation. Development work officially continues at the same GitHub repository, but only volunteers remain, so there has merely been maintenance activity.
It's so sad that Mozilla blamed covid for this, as never before people have been spending so much time at home and in front of their computers, causing record earnings for digital companies. Including Google, which finances Firefox.
Daily usage time of Firefox has actually went up around April 2020, by around 24 minutes and has been at that level since.
Also the employees found really good employment at other companies so there seems to be high demand for Rust developers, at least in the USA, and at least for this segment of the market.
When rust can significantly beat C for security, speed of development, and speed of execution then it will start to infect the kernel. Why use rust if it only improves security but has other downsides?
This is an amazing peoject because it will encourage rust to improve to be better than C for multiple dimensions.
I’m not a rust fanboy, but why do you think it doesn’t beat C for speed of development or speed of execution? While the former is hard to prove without empirical studies, I would be very surprised to find a language with such a weak abstraction power as C more productive than Rust. And for the latter, there are benchmarks showing Rust can surpass C in execution speed — and even just theoretical it has more metadata available so it can make in theory make better optimizations.
A pet peeve of mine is the almost complete lack of empiricism in Computer Science. "Nevertheless, we believe that, even today, the advantages of using Rust [in the Linux kernel] outweighs the cost." Where is the evidence?
"Rust is obviously a better choice for Linux than C." "Ok, if it is obvious, it should be easy to demonstrate?" This is not only about Rust - 99% or so of all new tech can't demonstrate any benefits.
It's not just being able to demonstrate the benefits empirically, but the combination of that with constantly loudly touting them.
When someone thinks their hashmap is faster in theory they get asked to benchmark it... it often isn't in practice.
But when someone decides that some norm in whitespace, use of parenthesis, or use favored control-flow concepts will result in a lower defect rate or reduced maintenance costs we somehow are expected to accept these conclusions without clear evidence.
Even though everywhere is in computer science where we can easily measure results there are countless examples of theoretically better stuff that works less well in practice. Why should we expect our intuition on human interactions to be any better than our intuitions over which data structures are faster?
If you want empiricism try academic papers instead of mailing lists ;) Some numbers: most critical bugs in the Linux kernel are due to memory safety: 40 out of 65 bugs allowing for code execution found in Linux in 2017 could have been prevented by using a memory-safe language [0]. The paper is about Go, but the same logic applies for Rust. Fun fact: 39 out of these 40 bugs were in drivers [1], so I think starting with drivers in Rust is a good idea.
This is a popular argument, but I'd argue isn't really true. Looking at electrical engineering, there is no standard license and no standard of performance. Like programming, it's just too broad across different areas with vastly different requirements. The equivalent of very sloppy web development might be a blinky (non-kids) toy, that fails after a few uses with no particular requirements (other than "make it as cheap as possible"). Or look at how often smartphones and laptops fail electrically (or compound failures) for really obvious reasons, that just weren't important enough to fix.
To address the particular point being made here, lots of things in EE are experience driven, for example which decoupling capacitors to use or to not use 90 degree angels on pcbs. I think many aspects of typical engineering are way more driven by experience and vague rules of thumb than we expect looking in from the outside.
The reason why this is more obvious in software development may be because it's pretty hard to have good measurements and understanding failures is pretty hard and working around them is pretty easy.
Very few countries in the world have "licensure" for engineers. It's not a protected term of any sort in the US nor in most of Europe. Canada is one of the few places that's strict about it. Occupational licensing is a form of job market restriction that's generally frowned upon.
In the three European countries I've lived in (2 in EU + UK), licensed engineers must sign off on new building construction plans, and take responsibility for the suitability of the design.
Ridiculous. Have you met engineers from other disciplines? They employ empiricism constantly. You can't build the golden gate bridge without proving that the steel you want to use can take the strain. "Obviously steel C is better than steel Y" won't cut it.
If anything, software development is sandcastle-building. We all have ideas on how to build it, but at the end of the day we're just amateurs fucking around with plastic shovels.
Arguably, other disciplines doesn’t have to handle undecidable problems everywhere. Building a bridge can basically be “statically analyzed” completely, in that every requirement is known beforehand and there is a very constrained mutable state there.
On the other hand, even the most basic program one can write will have properties not decidable without actually running the program. I would like to see much more rigorous testing/quality control in user-facing apps as well like websites, but lower level tools usually are written with much higher quality. You would be hard pressed to routinely find a bug in eg. the JVM — and writing formally verified programs is infeasible for any significant complexity.
Good point. And also, it is possible to add safety margins to other areas of engineering. One may be able to eg. store an array with padded nulls (actually we sort of do it at an OS level, just not with null but some “magic number”), but if it’s a loop/recursion, non-correct software will blow up either way - here it is more close to mathematics than science or engineering in my opinion.
We expect software to work, in places that are engineered. A SpaceX Falcon launch control system that encounters anything undecidable, whether in launch preparation or in flight, has utterly failed.
It was Microsoft that got people used to software failing on a routine basis, and "have you rebooted?". Before, anything that ever crashed, you sent back for a refund, or expected a tech to come out and fix it post-haste. Clearly MS could not make money that way, but getting people with no experience to believe failure was normal was quite cheap to arrange.
You are comparing apples and oranges here. Bridge building should be compared to aerospace software engineering, not to the average CRUD app.
Apart from that, I think you are giving too much credit to construction engineering outside of the highly regulated sectors. They mess up just as often, and expensively so. But stakeholders in construction are slighly more aware that their requirements are going to be set in stone, pun fully intended.
What really makes a difference is the malleability of software. This is an invitation for feature creep and last-minute changes. Also, the impact of overengineered, unmaintainable software is not quite in-ya-face as for physical assets. Therefore, in software engineerin project managers need to be religious about these matters. And software engineers must get better at communicating about these concerns.
Do you think it would be preferable for the Rust-in-the-kernel advocates to not engage with the kernel community at all until they have a prepared a large array of demonstrations?
It can be a pet peeve of mine as well, but at the same time, there are statements that require no empirical analysis to “prove”. Like, would you disagree that writing a program in brainfuck is more error-prone than Java?
But even in medicine, probably the area with the most extensive empiricism (we have insane amounts of data on mouses and humans), there are claims that won’t ever get to the point of doing a study on. There are stupid things that get tested, eg homeopathy (it has plenty of studies showing it is quack), but I think noone would say it is irresponsible for a doctor to claim that hand healing over a gun shot is not effective, even without direct evidence.
Of course there are surprising findings everywhere and as other commenters wrote, especially in low level optimizations the “obviously slower” solution can easily win. But I would wager that C is so terrible as a language with so little protection from even trivial mistypings, no real abstraction power requiring text-based macros that simply should have never existed to begin with, and the worst of all, not even goddamn namespaces making everything’s name indecipherable. On the other hand, I’m not necessarily bought that Rust’s borrow checker is the best way for handling memory-related errors, for example a syntactically better-than-C language like C++, or Zig might reduce memory bugs just as much, we don’t know.
> the almost complete lack of empiricism in Computer Science
Claiming that an entire science field "lacks empiricism" is quite brave, and definitely false in the case of CS.
Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
> "Nevertheless, we believe that, even today, the advantages of using Rust [in the Linux kernel] outweighs the cost." Where is the evidence?
Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
Furthermore, many of the claims I wrote in the RFC are simple facts (e.g. features that Rust has) that you can easily corroborate. For other claims, you can look up plenty of articles, scholarly and otherwise, about Rust benefits.
But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
> "Rust is obviously a better choice for Linux than C."
The RFC does not claim it is "obviously" a better choice.
Quite the opposite, in fact: see the "Why not?" section.
> Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
I know, but it doesn't contain (empirical) evidence.
> But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust. Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
> I'm not the first one lamenting the lack of experimentation in CS. See ...
Take your first IEEE paper's abstract:
"Empirical software engineering research needs research guidelines to improve the research and reporting processes."
They are pointing out problems with the empirical research that is being done, not claiming "almost complete lack of empiricism in Computer Science" as you did.
There are issues in all science fields, their research, their statistics and their reporting, in particular outside their major journals (and not even those are infallible). Calling out an entire science field in particular is quite insulting to all their researchers.
> I know, but it doesn't contain (empirical) evidence.
Empirical evidence of what? Please do not remove/ignore parts of my reply to suit your argument.
You may claim that you would prefer the RFC to include references to papers and articles; but you cannot claim the evidence does not exist ("99% or so of all new tech can't demonstrate any benefits") or that an entire field has no clue ("the almost complete lack of empiricism in Computer Science").
> That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
No, it is not equivalent, because we are not unconditionally adding Rust into the kernel (much less "rewriting it" as you claim). In fact, one is free to have a C and a Rust version of the same driver and compare whatever metrics you need. Even the RFC contains such an example and I am aware of at least one company doing precisely that.
Moreover, Rust is not an "unknown language" in the sense of an "unknown drug". On the contrary: it was explictly designed to address some issues such as memory-safety.
> A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust.
Not really, because 1. you would need to show the same results would apply to the Linux kernel and 2. we are not rewriting the kernel in Rust.
> Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
As mentioned, I am aware of at least one company running an even better experiment than the one you propose.
If you want faster results, then you will need to put money into it yourself.
I find it interesting how virulently opposed Linus was to introducing C++ to the kernel in contrast to the embrace of Rust. It made me think that the idea of another language in the kernel wasn't the issue in itself, but that C++ was a bad language and Rust wasn't, and also that there were tangible benefits to adopting Rust like memory safety that couldn't be had with C.
I do hope that Rust doesn't suffer from an explosion in complexity and compile times, though. Memory safety and overengineering shouldn't have to go hand in hand.
C++ is like nuclear power. Very powerful but very dangerous if only used slightly wrong. It would have been a constant fight over which parts of C++ are off limits and which parts are good to use. In recent years the language has improved but it carries a lot of baggage. Also it's too close to C in some regards and invites C programmers who just don't know enough dark corners of C++ yet.
To me C++ is a language of last resort. It can do everything you need but you should not touch it if there is any other option.
Python et al.: normal household batteries; everyone can use/operate them, not much power but can be used with little to no training and a screwup happening means you went looking for trouble
Java et al.: electric batteries; substantially more power but has to stop relatively often (GC pauses), everyone can use the things running on them at least reasonably well [0] but one little thing can potentially cause a catastrophic chain reaction (one cell out of thousands => thermal runaway (eg. the infamous NullPointerException))
Rust, Go et al.: industrial backup generators or hydroelectric plants; substantially more power yet again but have a startup/setup time (learning a more complicated language/ecosystem), as reliable as can be realistically done but have to be spec'd right for the building/installation/etc. at hand (time vs. resources tradeoffs); screwups happening means you probably went looking for trouble (eg. in Rust: excessive cloning of data, building of circular references without weak references; not so sure about specific examples in Go but didn't want to lump it together with Java et al.)
C, C++ et al.: "nuclear power [...] very powerful but dangerous if only used slightly wrong" like in your metaphor; should only really be used by experts but even they are not safe from introducing catastrophic failures with something seemingly harmless (seen as such at the time of writing the code); it gets safer over time but there is the huge inertia of existing installations running with huge time and resource costs associated with dismantling or even modernisations; footguns are everywhere and not always immediately recognisable especially since there recommendations about usage and safety concerns floating around which are sometimes years or decades out of date
[0]: AFAICT a substantial part of the business software world runs on Java code cobbled together by not-necessarily-full-on-experts
PS:
My personal experiences/biases:
- learned C++11 for a while, didn't like it, SEGFAULTs and associated debugging/fine-tooth-combing over the some-tens-or-hundreds-of-lines long code got really annoying really quickly
- learned Java, Haskell and Prolog in Uni; Haskell was a nice intro to functional programming but nothing I would want to use even occasionally, Prolog was meh, Java was/is a convoluted, resource-hungry mess with huge amounts of boilerplate for every little thing and will IMHO always be associated with bad assignments in class(es)
- Python3 I use every other week or so but not for "real programming (TM)", but instead for small (automation/download/crawler) scripts; I like it but for serious programming in a business environment - ie. something I personally would want to rely on for money - the duck-typing is just too unstable for my liking, even though I still like it for rapid prototyping of concepts coming into my head which I then later may or may not transfer into:
- Rust... currently and personally my favorite language for "real programming (TM)" even though I struggled with the borrow checker for a long, long time (and still run into it on occasion); sometimes really long compile times but nice runtime performance in most cases even without special care and tuning while I still have the confidence that if it compiles it runs and any errors encountered at testing-/runtime are probably more due to logic errors than running into something like "oops, in this deep branch of cascading ifs I accidently didn't set some object's field or dict's value meaning at some later point the program blew up due to NoneError or something similar"
- learned a bit of Go (before even finding out about Rust) but I just didn't like it, it just didn't "gel with me", don't even really remember the specific reason(s)
Not a very good analog there — most problems can be solved with any programming language, but there is a very very significant difference between batteries and power plants.
Also,
> but has to stop relatively often (GC pauses)
is severely outdated and I don’t see how NPEs are something specific to Java.
Linus explained his reasons for not wanting C++ specifically in the kernel. I observe that very few people correctly understood his reasons, and every time it comes up I am reminded of that. The reasons, as far as I understand Rust, do not apply to Rust, hence the different reaction.
Since I'm on mobile, here's a link to a previous explanation of mine: https://news.ycombinator.com/item?id=1422160 It is meant to be read in conjunction with the original post, addressing what I saw as the point most people were missing, not a complete restatement of his points.
Not to nitpick, but wouldn’t some of the same arguments against virtual functions apply to function pointers in C? The kernel uses a lot of “obj->obj_ops->foo(obj, arg)” calls which are similarly dependent on the precise type of obj. In the case of these calls, the general assumption is that one can infer roughly what they _should_ be doing based on the type of obj. For example, if it was fd->fops->read(fd, buf, len) you would expect it to put some data in buf even if you had no idea what fd’s runtime type was.
Your post also mentions operator overloading, which is a bit nastier since the calls aren’t explicit. But Rust also has operator overloading, so similar complaints would probably apply? In practice I’d expect the kernel to be fairly strict about limiting operator overloading (which would apply for both C++ hypothetically as well as Rust).
If you look at it from the assembler point of view, even function pointer structs in C have a bound on them.
I can't directly speak to Rust's overloading support. If it's still one static type lookup he may be ok with it, plus you have the clarity around ownership of any intervening constructions. I'm sure the Rust Evangelism Strike Force (awesome 80s cartoon music intensifies) will be along presently to answer the question of how much assembler one can generate with an overloaded operator.
C++ really does have a particular ability to stack a huge number of abstractions into one little line. Ultimately it's a continuum rather than a sharp line, but C++ is really far on one side.
I did get your point but would like to point out that using these slurs isn't helping any discussion.
There are fanatics in every area everywhere. Their existence doesn't devalue any of the bigger groups they belong to. Their existence is a fact of life.
I've worked with plenty of Rust devs (and I am an aspiring one myself although not all the way there yet) and I've only ever seen hard-working people with an excellent eye for detail and desire to maximally utilize the hardware and have better safety than other languages they worked with in the past. Zero fanatics.
C++ and Rust both have a particular ability to integrate many abstractions in one place, all to be squeezed out by the compiler leaving exactly optimal object code for the specific uses in that specific place. Their ability to combine and particularize abstractions is a strength. Rust got it from C++ because it is so damned powerful.
Just at the moment, C++ is notably better at this, in important cases, but I gather there are plans for Rust to adopt more of C++'s capabilities. The Rust designers fully understand the cases I mention, but are very busy.
Rust lets you choose between static and dynamic dispatch, and so operator overloading, 99.99% of the time, is statically dispatched, as that's the default.
As is the case, in fact, in C++ code: so, a distinction without a difference.
Linus's ignorant diatribe about C++ tells us plenty about Linus, but nothing meaningful about C++ or its suitability for kernel coding. That every detail he rails about in C++ applies equally to Rust is unintentionally revealing. That each such detail makes C++ and Rust better for large systems, moreso.
Yes, the famous post is old, if i recall correctly is from 2004, so its addressed to the C++ 98 and style that was in vogue back then.
Take some vintage C++ code from late nineties, like games, and there's no concept of ownership, before smart pointers and move, and full of global's everywhere.
It was also when code were transitioning for a multi-threading era, so using the same code style of the previous sequential era was really bad.
In 2004, template support in compilers in the wild (most particularly, MSVC) was still very shaky, so 1990s style C++ was common. Virtual functions are almost a curiosity today because we got better things for most uses. But everything he complained of is still there, and still sometimes the right thing. That his opinion didn't change as the facts changed under him reveals much, unless his public acceptance of identically the same features in Rust is meant to count as a sort of mea culpa. (To actually say so would be gracious.)
Of course, all the new features C++ gained since '98 only make the language more useful for coding large, long-lived systems, as is amply demonstrated in the tens or hundreds of billions of lines of C++ code implementing exactly those systems. Rust's adoption of so many of those same features increases its usefulness for a growing number of those same purposes.
That tells us nothing about C++ vs Rust because the decision to use C++ for Zircon would have been made back in 2016 or 2015. At that time Rust was too immature to warrant serious consideration for such a large project. More recently use of Rust in Fuchsia has been expanding, although the C++ kernel is obviously far enough along that they're not going to re-write it in Rust.
Its not just that. The experience of the teams also counts.
For this kind of project its more important to have people with pratical experience in kernels, than experience in a particular language.
These people will have a experience in languages they used before, so even if the decision was taken today, you wouldn't find enough people with experience in kernels AND Rust, but will have much more people with experience in kernels C and/or C++, so C++ would still be a good default in comparison to C.
It would be pretty amateurish to get the kernel expert folks to learn Rust while implementing it, as it would probably mean that the end result in Rust would be worse than the end result in C++ as they were more comfortable and experienced in that language.
It will take more time to form specialists in some key areas so that decision can be made with less side-effects.
First, it _would_ make sense to use a narrow c-like subset of c++, but lots of people wanted to bring 'good practices' from c++ to the kernel.
And many of these were _bad_, C++ was a different language back then and the culture of how you write C++ was different. There were times when it was fashionable to write OO spaghetti almost as bad as the stereotypes about java, or templates that worked except for some unforeseen footgun (then you could template libraries that worked correctly but figuring out how was a trip). And before all of that there was a decade when C++ compilers were just _full_ of bugs, and before that there was no standard and every compiler was different. The negative stereotypes about C++ come from all that.
I think he had every reason to fear that e.g. he would have to fight someone who implemented control flow using an inheritance hierarchy because it's a pattern. And he could find himself in the minority.
(Also I'd say there are still counterproductive fads, and it's still common to write C++ that's beautiful on the outside and completely inscrutable inside, or generally pay a huge price in code size and complexity for marginal gains elsewhere, or to depend on optimizations that don't trigger. Same in many other higher-level languages but somehow not in C.)
Second, more important, i don't remember any talk of adapting C++ to bare metal programming, while there is an ongoing cooperation with rust folks already.
> Second, more important, i don't remember any talk of adapting C++ to bare metal programming, while there is an ongoing cooperation with rust folks already.
C++ already is used in bare metal environments and in at least one production kernel (XNU) so there just isn't any adapting to do. The standard explicitly distinguishes between hosted and freestanding environments and defines what you should expect to have available.
Furthermore, it would absolutely not "make sense to use a narrow C-like subset". That would exercise none of the power of the language to generate constructively-correct, high-level, yet optimal code. You would just have the same buggy code with slightly different syntax. That would be very dumb indeed; although, once you start building the native C code with the C++ compiler, it is easy to incrementally improve matters, keeping the system working continually throughout the process. As one does.
That does not mean you would try to use virtual functions (much), or std::vector or std::shared_ptr (ever). Instead, you would use abstractions tailored precisely to the kernel environment, and arrange that wrong code would tend to not compile, so that when code compiles, it works. In C, you have to enforce all conventions by obeying instructions in comment blocks, and updating all uses every time the comment block changes. In C++, you can encode the rules directly into the type system and put that to work, not just checking correctness, but actually generating correctness.
This is the same as one would do in Rust modules. Failing to make full use of Rust's type system would be a grave mistake that we need not fear will happen.
As popular and ubiquitous as Linux is, there was a time before it. It wouldn't shock me if, in another decade or two, a new operating system kernel written entirely in Rust came along and filled most of the niches that Linux fills while addressing all the security, stability, portability (mobile/low power device support), and licensing concerns that have emerged in the three decades since Linus announced version 0.02 to the world.
Yeah, forking Linux/making at least the drivers backward compatible is a must for any chance at a new OS emerging. But Linux has so many features that I don’t see a new contender getting anywhere close in the next few decades at least.
> My eyes have been reading C for almost 30 years by now, they have a lexer built in the optical nerve; reading something that looks vaguely like C but is definitely not C is an utterly painful experience.
> You're asking to join us, not the other way around. I'm fine in a world without Rust.
Such a luddite view. "My car gets 40 rods to the hogs head..." I'm always amazed to see such stuck-in-the-mud views from people working in technology which is such a modern fast moving field.
Cars benefit from newer scientific advancements like better materials and computerized control. What scientific advancements make C++ or Rust fundamentally different from C?
Outside of `unsafe` blocks Rust is memory safe. It also has a much stronger type system which means code is more likely to work.
That's something you have to experience first hand to understand, but you know that feeling you get writing C or C++ where new code appears to work the first time you run it? "That's suspicious. It must not have rebuilt, or maybe I forgot to call the new code."
With Rust the compiler can catch so many mistakes that by the time you actually run the code there's a pretty good chance that it is right. My instinct is no longer "suspicious! Where have I gone wrong?"; it's "ha it worked! Though I'd still better double check".
It's sad that a bunch of legit criticism around Rust in the kernel (like around tooling and maintenance burden if nothing else) probably won't lead to actual improvements being made because a tiny handful of people can't help themselves but dial up their condescension up to eleven and derail any attempt at productive discussion.
On the other hand, I continue to be amused by arguments along the lines of "I'd already made up my mind that Rust is bad, but then I actually looked at some Rust code and realized it doesn't use C syntax, and now I'm even more upset"...
There's a little bit of snark that results from any criticism of the suitability of Rust and its ecosystem for integration into the kernel.
Rust's tooling isn't quite as mature as it could be, and it achieves its advanced capabilities and performance by eschewing a runtime (which would make it unacceptable) in favour of labourious compilation.
The snark to me seems unnecessary; a more even headed discussion would acknowledge the limitations and focus instead on what can and will be improved or mitigated and what compromises are acceptable.
The tooling will mature with time, panics will be mitigated, and the compile time performance is often a sacrifice worth making for the sake of the safety provided.
I don't want to call anyone out by making specific examples. I'm talking about comments that in their wording presuppose agreement that the person you're talking to doesn't know what they're doing, or the thing you're talking about is misguided and deserves to be sabotaged.
Implications like that may very well be correct, but as always, not sticking to the technical aspects of the subject makes it harder to get anything done.
I am sure I am not the first to think of this but here's an idea:
Can we have a compiler flag or a Cargo switch inside the project's file that disables all std API that can panic? Basically complain that the function is not found.
I for one always strive to write my Rust code with `Result<T>` and never rely on panics so I'd love it if that can actually be enforced and checked by the compiler.
Obviously those of us not in the core team can only fill wish lists. :) And I don't want to sound entitled!
But I hope that if it's both possible and not very hard, then it will be considered. I'd personally love to double down on Rust's safety and dial it to eleven in all my hobby and commercial projects.
> [T]he interest in allowing Rust modules into the kernel are a problem for us, due to Rust trademark restrictions which prevent us from applying patches in our distribution without express permission. We patch to remove non-free software, unlicensed files, and enhancements to user-privacy anywhere it is applicable. We also expect our users to be able to re-use our code without any additional restrictions or permission required.
> This is also in part why we use UXP, a fully free browser engine and application toolkit without Rust, for our mail and browser applications.
182 comments
[ 3.0 ms ] story [ 226 ms ] threadRight, only "repr(C)" types can show up in "extern" APIs. So no data-bearing enums, no Vec<T>, etc.
> Will the gains in safety be contained entirely inside the modules that are written in Rust?
I think for some things the answer is yes, and for other things the answer is no.
If the issue is that callers might retain some pointer or descriptor longer than they should, leading to a use-after-free or something like that, then there's probably no way to fix that with an API-compatible Rust reimplementation. What safe Rust would prefer to do is either 1) force explicit lifetime constraints on the caller or 2) replace the pointer/reference with some sort of smart pointer, which can either keep its referent alive or at least safely detect the fact that the referent is dead. But of course a C-compatible API has no way to represent (1), and (2) would presumably be an incompatible API change, so the Rust reimplementation can't really solve this issue. (Though if both Rust and C APIs get exposed for the new code, then new Rust callers could benefit from (1).)
On the other hand, if the issue is that the caller might pass in some invalid indexes or other metadata, leading to an out-of-bounds array access within the module, Rust can help quite a bit. If the array in question is actually owned by the module (and not just a raw pointer to unknown memory) then bounds checking is required one way or another (unless it's either optimized away or explicitly skipped with unsafe code), and a Rust reimplementation will probably not suffer from out-of-bounds issues. Raw pointers into the caller's memory are harder to check, but even there if they can be converted to a safe slice immediately inside the API boundary, some subset of out-of-bounds accesses can still be caught.
Looking at these two different kinds of bugs, I think it's interesting to think about what "inside the module" means here. In both cases, the UB we want to prevent (either use-after-free, or an out-of-bounds access) is "caused" by the caller's mistakes outside the module, but "occurs" on lines of code inside the module. In the first case, Rust can't protect itself against a bad (C) caller, but in the second case it might be able to.
The primary goal is to write in-tree drivers in Rust, which can still be loadable modules, but they'll be built to whatever ABI that particular version of the kernel uses. And even for out-of-tree modules, you have to recompile them against the specific kernel and ideally with the same version of the compiler etc.; ABI incompatibilities across C compilers are rare, but not unheard of.
Right now both alloc/std and even basic language constructs like indexing into a slice can panic. With little help from the tooling to guard against it. This makes sense when considering that Rust evolved as a language for writing somewhat higher level applications like browsers.
While Rust and especially the library ecosystem is generally great at forcing you to handle application domain error conditions, that mostly doesn't extend to these lower-level concerns.
But I'm hopeful that this can improve the entire ecosystem over the medium term. With things like lints that forbid panics, more focus on custom allocators and fallible allocations in std and the library ecosystem, etc.
edit: just to clarify on the slice index panicking: my point isn't that the C situation is better, but that a big reason for moving to Rust would be to benefit of a safer, more convenient language that doesn't require such extreme carefulness as C.
So the language should help in preventing panics, or making it obvious where they can occur. This is perfectly possible in Rust by using accessors that return `Option<T>` or iterators. But the tooling for such use cases can be improved, for example with lints, and the API surface can be customized to mostly prevent such conditions with the type system.
AFAIK they're not using std. And isn't this kind of panic the kind of thing that causes BUG()s today? Probably can just map one to the other.
Actually most of the time you probably wouldn't want a Vec that reallocates automatically in the kernel, you'd probably want a dynamically allocated array of fixed length the majority of the time.
Panicing on invalid slice indexing is a non-issue.
Well, it's less of a issue that what we already have. It would still be nice to be able to tell the compiler: "Issue a diagnostic for any [some qualifiers] panic that isn't optimized out by unreachable code analysis.".
This isn't the question here. malloc (or other memory operation) failures should never panic. According to Torvalds[1]:
> Allocation failures in a driver or non-core code - and that is by definition all of any new Rust code - can never EVER validly cause panics. Same goes for "oh, some case I didn't test used 128-bit integers or floating point".
Kernel code is a bit quirky in this sense that it will always favor system stability (and expect proper errors to trickle down) instead of crashing. Also, Linux runs on plenty of low-memory embedded systems so failing on a malloc may not even be a particularly rare occurrence.
[1] https://lkml.org/lkml/2021/4/14/1099
There are many MCUs that run Linux but definitely are incapable of compiling it.
IIRC most of the build time of a rust program is spent not in the rust compilation proper but in LLVM doing code generation and optimization. I hope both LLVM will become more efficient with time, and rust will learn to pass it such IR that it can process faster. There are obvious incentives for both of these things.
But of course it would be sad if the minimum requirements to compile the kernel grew again, and now excluded older RPis.
There’s already an OS completely written in Rust called Redox. Nobody uses it. I’m not entirely sure you could do useful work with it, which is contrary to the Rust marketing team’s talking points that Rust code is guaranteed to be fast and guaranteed to be bug free because Rust is just so darn great and the compiler authors so wonderful and smart. That there are basically no success stories other than a couple side projects after a decade of marketing is of no consequence.
Don’t read the blog posts. Go to the repositories. Check out the crates.io landfill. Grab the Rust book and try to write anything more significant than a CLI toy. I promise that you will know exactly why nobody uses the language in anger.
The Fuschia team, who are partly responsible for the Rust myth don’t even use it in their kernel. But they want it in Linux. I wouldn’t be shocked if this was all some conspiracy by Google and Microsoft to make a personal computing unusable so that everyone has to use their cloud offerings.
So you would rather believe in a ridiculous conspiracy than admit that large companies might just be choosing Rust for some valid reasons. Got it.
A small company that has become a success based on a whole XYZ stack may count for something, depending on how ambitious the company was, how hard it was to do, and how much competition they face. Lots of companies could succeed equally well with Rust, JS, Java, C#, Kotlin, Python, Ruby, Erlang, or Perl; those successes don't count. Lots more potter along, neither succeeding nor failing, exactly. Those don't count either, unless you want to give Rust credit for their not exactly failing.
And, many fail. We are never told about those. They might well count, but there are a lot of ways to fail.
It is ridiculous to believe in a Google conspiracy to outcompete Linux by getting it to incorporate Rust, because a conspiracy requires a crime, and that would not be one. But anyway Google doesn't have that much influence.
Secondly, there is a huge difference between a language being used in general and a language being used for serious, visible projects within a company. Think: writing a small utility in Zig at company X vs. rewriting the backend of a core product in Zig and pointing to measurable gains in performance and reliability.
> It is ridiculous to believe in a Google conspiracy to outcompete Linux by getting it to incorporate Rust [..]
Great, but you should be telling that to the user I was replying to.
Similarly: you then agree that "BigCo uses XYZ" is, in fact, not compelling. A big company betting its future on XYZ would count, but no big company does that. Betting a substantial new business unit on XYZ could count, but that is not cited as "BigCo uses XYZ", that is "BigCo relies on XYZ for UnitX to succeed", a possibly stronger statement, according as how important UnitX is to BigCo.
Calling a claim ridiculous needs reasons why it is ridiculous, which I supplied, and you did not.
It's like surfing. Some people want to see hundreds of surfers on the wave before they start paddling out. That's fine. There's nothing wrong with that. But I personally believe that a better way to surf, especially when it comes to technology which tends to have short cycles, is to learn how to spot the wave as it builds, while it's still a swell, before it breaks. I've seen time and again that there's incredible value to be had if you can learn how to do that.
One of the best examples of this value being that you're able to hire really talented people, because they form the critical mass at the beginning of a successful wave. Once the larger corporates come in, this advantage disappears.
I definitely agree with the notion that the early adopter group is more often than not more talented than average. But, I'm sure companies also take this into account when deciding whether or not to pick a new technology.
"If the definition ends up aligning with what large companies look for when adopting a new technology, then the signal ends up being the same."
And even then, the signal is only the same, but it's not worse. Whereas if large companies don't always optimize for quality, but have other incentives at play, or if they are less nimble, or less efficient, then looking for what large companies look for might actually end up setting you back with worse signal. So there's no upside I think.
A notable project choosing it and succeeding is signal. But there is a lot of noise. An individual judgment about the value is noise unless that individual has demonstrated a pattern of picking winners early and correctly.
Early adopters are often smarter than the average bear, but are also much more likely to jump ship the moment the next fad breaches. That might be OK if you only need them for a short time.
My full time job is writing Rust for large-scale production systems. The language is great - and as an organization we're doubling down on our investment in Rust this year.
As for purpose, there’s this blog post [0]:
> At this point, Discord is using Rust in many places across its software stack. We use it for the game SDK, video capturing and encoding for Go Live, Elixir NIFs, several backend services, and more.
[0]: https://blog.discord.com/why-discord-is-switching-from-go-to...
I find it would be very difficult to convince anyone to use any other OS than the currently mainstream ones. You should instead see this as an accomplishment of what Rust can achieve.
I also think you are undermining Redox OS developers efforts, which is kinda sad.
>Rust code is guaranteed to be fast and guaranteed to be bug free
Nobody claimed this, what people claim is that it can be as fast as C, and it has some safety guarantees. You can still have bugs.
>Rust is for writing blog posts...
What's so wrong about people being excited about something? Personally I find it awesome that a programming language can bring so much enthusiasm to people. In my opinion, what you are seeing here is people being excited about progress being made.
>the compiler authors so wonderful and smart
I sense some envy here, people achieved a great thing through effort and collaboration, we should celebrate this.
It's the foundation for Fargate and Lambda, so a <<ton>> of AWS customer code runs on top of it.
Considering Microsoft, Google, basically every big company backs it now, I'm sure there are other infra projects using it. They're just not all very visible because these companies only open source a small percentage of their infra.
[1] https://ziglang.org/
The kernel project distributes several userspace utilities and new ones of those might be worth writing in Zig.
However, each has their place. One man passion projects should not be used in anything mission critical. They totally should be used for fun, enjoyment and inspiration, and they may potentially grow to industry scale.
https://github.com/graydon/rust-prehistory/graphs/contributo...
After the initial commit in the "group" repo, development of the language has switched to a team instead of a single person, with the introduction of additional contributors. This holds even if you only look at the pre-1.0 days which established the core concepts of the Rust language as it stands today (the original language when the team started to work on it was a bit different to 1.0).
https://github.com/rust-lang/rust/graphs/contributors?from=2...
The work since 1.0 can be classified as a massive refinement project, which also does not have a single leader. Zig on the other hand does have a single main contributor/maintainer/creator:
https://github.com/ziglang/zig/graphs/contributors?from=2015...
hostapd is maintained by a single human. They are responsible for about 69% of all commits in the git repo, 55% if you look at the last year. It's still the most widely deployed WiFi implementation of the world. https://www.openhub.net/p/hostapd/contributors/summary
This is not an argument against Zig or that it would not become an industrial scale tool. It just does not feel "lindyproof" yet for real work.
Individual insight is often much better than a design by committee, but a language needs a quite robust ecosystem of support from industry bodies at some point to become a viable tool for generic industry scale development.
Zig is newer than Rust, but the Zig Software Foundation was in fact founded before the Rust Foundation, and already employs several core members. For example, Jakub Konka who resigned from Microsoft to work for the ZSF (https://ziglang.org/news/jakub-konka-hired-full-time/), and whose linker was the first to enable cross-compilation to Apple Silicon, and is already making Rust's cross-compilation just work (https://news.ycombinator.com/item?id=27245369).
Inspiring how such a young language can already be so far ahead when it comes to something as vital as the compile chain.
It's not funny if you understand the relative priorities of the two teams. Rust tends to prioritize working with the system, so all of our default configurations use the system linker. Zig doesn't have the same constraints.
This feature is awesome, and I would love to see it in Rust as well, but like, it's not surprising.
No, Zig absolutely has an awesome compiler. It's not "a toy" or a "one man passion project" and it's condescending to reduce it to those terms, as if it is not also somehow "industrial scale".
To be fair, your comment would also be better served without the insinuation of "just trying to score internet points".
I've made the edit. Thanks for the tip. I've enjoyed and learned from this exchange with you.
It is worth looking at all the serious bugs in deficiencies in the upcoming 0.8 milestone: https://github.com/ziglang/zig/milestone/10
This is not a criticism of Zig, nor is it an insinuation that Zig is unable to write correct software. It is a reflection of that fact that compilers are very complex.
Any argument that Rust, by contrast, would be OK in the kernel applies at least as strongly to C++, because C++ can read kernel C headers, where Rust will need hand-maintained equivalents. (Some automation is possible, but not enough.) The FUD Linus flings about C++ applies equally to Rust. (Anybody who says Rust is less complicated does not know Rust well at all.)
But argument validity is not what carries the day: bad arguments tend to fail, but good arguments get pot luck. Ultimately, it comes down to biases of those with power to decide: C seems to them like a familiar quantity (full of risks, but favored risks), while Rust happens to be hot just now.
In three years' time, Rust could very well not be so hot, and a new new thing might take over its social role. Zig could conceivably be that thing. (Zig isn't "safe", by Rust's definition, but by then something else may be more important; anyway the lack of safety does not disqualify C.) The new hotness will necessarily already exist today, to be mature enough to consider then, but not necessarily one that immediately comes to mind. Being different from Rust could be, by then, among the new hotness's attractive qualities, much as percieved differences from C++ boost Rust in some milieus.
Zig is totally different from Rust. For starters, it does not have a safe subset of the language that prevents an entire class of bugs. It's slightly nicer C. There are plenty of languages like that.
Also, the more familiar one is with a subject, the less one tends to conflate it with "plenty" of others. Zig is a completely different language to C (while also targeting the same systems space where you would need "unsafe" Rust to get anything done). In fact, it's hard to find too many languages that come close to Zig's comptime, plus the entire array of safety features offered by Zig. Your argument is a weak strawman at best.
Zig might have some safety features, but it's dwarfed by what Rust has to offer: https://scattered-thoughts.net/writing/how-safe-is-zig/
More importantly, you can cause exploitable crashes in Zig programs while in Rust you need unsafe for it (at some point in the program).
And yes, each Rust codebase has unsafe somewhere, but there are quite many codebases that don't have a single line and instead use dependencies that encapsulate the unsafe in a safe to use API.
Would you say also that Rust's macros offer a stronger type system than Zig's comptime?
There is this misconception that Rust will make systems secure simply "because memory safety". But this overlooks the most foundational point of systems safety: simplicity. Most failures or exploits (two sides of the same coin really) originate through excessive dimensionality, a point where Rust's complexity unfortunately dwarfs Zig's.
> [Zig is a] slightly nicer C. There are plenty of languages like that.
Are there “plenty of [actively supported] languages like that?” The design space of relatively simple, typesafe, imperative native programs strikes me as rather unexplored since the early 90s. At least I can’t think of many languages I would consider a nicer alternative to C - indeed actual C alternatives like Pascal and Fortran are arguably less nice. Maybe this is my ignorance.
There are various languages “between” C and C++ in terms of abstraction and complexity, and the space of “somewhat C-like” languages is heavily explored (D, Java, C#, Objective-C, and in a different sense Ada or Swift). There are plenty of alternatives to C++. But Zig is one of the only ones that’s not dramatically more complex than ISO C.
Happy to be corrected if I am missing something obvious (I mostly do functional programming).
(This specific post:)
https://news.ycombinator.com/item?id=26812047
https://news.ycombinator.com/item?id=26831841
(Previous Rust-in-kernel work:)
https://news.ycombinator.com/item?id=24334731
https://news.ycombinator.com/item?id=23800201
https://news.ycombinator.com/item?id=20833639
Linus Torvalds on Rust support in kernel - https://news.ycombinator.com/item?id=26831841 - April 2021 (290 comments)
An RFC that adds support for Rust to the Linux kernel - https://news.ycombinator.com/item?id=26812047 - April 2021 (261 comments)
Supporting Linux kernel development in Rust - https://news.ycombinator.com/item?id=24334731 - Aug 2020 (354 comments)
Linux kernel in-tree Rust support - https://news.ycombinator.com/item?id=23800201 - July 2020 (491 comments)
Linux kernel drivers in Rust might become an option in the future - https://news.ycombinator.com/item?id=20833639 - Aug 2019 (254 comments)
(I'm thinking of making HN's software render links to previous threads in this format automatically. If anyone can think of a downside to that, please let me know - maybe at hn@ycombinator.com so as not to take this thread off topic.)
For example NetBSD has Lua in the kernel and FreeBSD has or had a Forth interpreter in the bootloader.
Overall though, your overarching point still stands that the Rust development is big in that this marks the first time actual Linux kernel code will be written in a language other than C (or platform-specific assembly).
People talk about the insertion process as if it were JIT just because it reuses infrastructure originally invented for JIT.
What can be done in eBPF code is somewhat limited, but nowhere nearso as you might imagine from the restrictions imposed on eBPF programs. Things eBPF code is not allowed to do directly are routinely done in functions eBPF is allowed to call, or in the code that calls the eBPF program in a loop.
https://fuchsia.dev/fuchsia-src/development/languages/c-cpp/...
You gotta remember that most of the heavy hitters in Linux Kernel development haven't even commented yet. When the Rust side presents something mergeable, the massacre will begin.
I think a lot of Rust people read Linus's "I don't hate it" as explicit approval. But anyone who's followed LKML knows that this is not the case.
Remember what happened with KDBUS? Gregkh fought long and hard to get it merged but in the end it didn't happen because too many people didn't want it.
There is nothing to comment yet. I've read those driver, it seems unconvincing for me despite me being a rust fanboy. Linus said about panics, I agree completely with him, and with the possibility to get rid of those panics. There is unsafe code where it shouldn't be, for example driver uses unsafe to create and to initialize a semaphore. I believe this unsafety could be removed by using Pin or something like, but when it removed, then it would be the time to comment. And I mean it. Kernel code heavily uses list_head, for example, which is PITA with rust. How to deal with it? I'm not sure that the rust will be better than C, just because it would be rust used in a C style, but it doesn't work. When I met Rust for the first time, I tried it with a C style of designing programs. Many people do, and it just doesn't work.
For now it is impossible to judge how rust will help, we might believe that it will help, because rust is a safety-oriented language, blah-blah-blah. But it is a belief, not something that we could touch, play with, see how it helps and how it obstructs.
https://wiki.mozilla.org/Oxidation
Parts of Google Fuchsia are apparently developed in Rust, too.
https://en.wikipedia.org/wiki/Servo_(software)
> After Mozilla laid off all Servo developers in 2020, governance of the project was transferred to the Linux Foundation. Development work officially continues at the same GitHub repository, but only volunteers remain, so there has merely been maintenance activity.
Daily usage time of Firefox has actually went up around April 2020, by around 24 minutes and has been at that level since.
https://data.firefox.com/dashboard/user-activity
Also the employees found really good employment at other companies so there seems to be high demand for Rust developers, at least in the USA, and at least for this segment of the market.
This is an amazing peoject because it will encourage rust to improve to be better than C for multiple dimensions.
"Rust is obviously a better choice for Linux than C." "Ok, if it is obvious, it should be easy to demonstrate?" This is not only about Rust - 99% or so of all new tech can't demonstrate any benefits.
When someone thinks their hashmap is faster in theory they get asked to benchmark it... it often isn't in practice.
But when someone decides that some norm in whitespace, use of parenthesis, or use favored control-flow concepts will result in a lower defect rate or reduced maintenance costs we somehow are expected to accept these conclusions without clear evidence.
Even though everywhere is in computer science where we can easily measure results there are countless examples of theoretically better stuff that works less well in practice. Why should we expect our intuition on human interactions to be any better than our intuitions over which data structures are faster?
If you want empiricism try academic papers instead of mailing lists ;) Some numbers: most critical bugs in the Linux kernel are due to memory safety: 40 out of 65 bugs allowing for code execution found in Linux in 2017 could have been prevented by using a memory-safe language [0]. The paper is about Go, but the same logic applies for Rust. Fun fact: 39 out of these 40 bugs were in drivers [1], so I think starting with drivers in Rust is a good idea.
[0] https://www.usenix.org/conference/osdi18/presentation/cutler (really good read!)
[1] https://arxiv.org/pdf/1909.06344.pdf (disclaimer: I'm a co-author of that paper)
To address the particular point being made here, lots of things in EE are experience driven, for example which decoupling capacitors to use or to not use 90 degree angels on pcbs. I think many aspects of typical engineering are way more driven by experience and vague rules of thumb than we expect looking in from the outside.
The reason why this is more obvious in software development may be because it's pretty hard to have good measurements and understanding failures is pretty hard and working around them is pretty easy.
If anything, software development is sandcastle-building. We all have ideas on how to build it, but at the end of the day we're just amateurs fucking around with plastic shovels.
This. And even then, after the calculations are done the engineer add a “safety margin” (and it's not 10%, more like 500 or 1000%).
It was Microsoft that got people used to software failing on a routine basis, and "have you rebooted?". Before, anything that ever crashed, you sent back for a refund, or expected a tech to come out and fix it post-haste. Clearly MS could not make money that way, but getting people with no experience to believe failure was normal was quite cheap to arrange.
Apart from that, I think you are giving too much credit to construction engineering outside of the highly regulated sectors. They mess up just as often, and expensively so. But stakeholders in construction are slighly more aware that their requirements are going to be set in stone, pun fully intended.
What really makes a difference is the malleability of software. This is an invitation for feature creep and last-minute changes. Also, the impact of overengineered, unmaintainable software is not quite in-ya-face as for physical assets. Therefore, in software engineerin project managers need to be religious about these matters. And software engineers must get better at communicating about these concerns.
But even in medicine, probably the area with the most extensive empiricism (we have insane amounts of data on mouses and humans), there are claims that won’t ever get to the point of doing a study on. There are stupid things that get tested, eg homeopathy (it has plenty of studies showing it is quack), but I think noone would say it is irresponsible for a doctor to claim that hand healing over a gun shot is not effective, even without direct evidence.
Of course there are surprising findings everywhere and as other commenters wrote, especially in low level optimizations the “obviously slower” solution can easily win. But I would wager that C is so terrible as a language with so little protection from even trivial mistypings, no real abstraction power requiring text-based macros that simply should have never existed to begin with, and the worst of all, not even goddamn namespaces making everything’s name indecipherable. On the other hand, I’m not necessarily bought that Rust’s borrow checker is the best way for handling memory-related errors, for example a syntactically better-than-C language like C++, or Zig might reduce memory bugs just as much, we don’t know.
Claiming that an entire science field "lacks empiricism" is quite brave, and definitely false in the case of CS.
Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
> "Nevertheless, we believe that, even today, the advantages of using Rust [in the Linux kernel] outweighs the cost." Where is the evidence?
Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
Furthermore, many of the claims I wrote in the RFC are simple facts (e.g. features that Rust has) that you can easily corroborate. For other claims, you can look up plenty of articles, scholarly and otherwise, about Rust benefits.
But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
> "Rust is obviously a better choice for Linux than C."
The RFC does not claim it is "obviously" a better choice.
Quite the opposite, in fact: see the "Why not?" section.
> Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
I'm not the first one lamenting the lack of experimentation in CS. See https://www.cs.princeton.edu/~jrex/teaching/spring2005/fft/m..., http://www.inf.fu-berlin.de/inst/ag-se/teaching/S-Komponente..., https://ieeexplore.ieee.org/abstract/document/1027796, https://ieeexplore.ieee.org/abstract/document/4221632
> Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
I know, but it doesn't contain (empirical) evidence.
> But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust. Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
Take your first IEEE paper's abstract:
"Empirical software engineering research needs research guidelines to improve the research and reporting processes."
They are pointing out problems with the empirical research that is being done, not claiming "almost complete lack of empiricism in Computer Science" as you did.
There are issues in all science fields, their research, their statistics and their reporting, in particular outside their major journals (and not even those are infallible). Calling out an entire science field in particular is quite insulting to all their researchers.
> I know, but it doesn't contain (empirical) evidence.
Empirical evidence of what? Please do not remove/ignore parts of my reply to suit your argument.
You may claim that you would prefer the RFC to include references to papers and articles; but you cannot claim the evidence does not exist ("99% or so of all new tech can't demonstrate any benefits") or that an entire field has no clue ("the almost complete lack of empiricism in Computer Science").
> That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
No, it is not equivalent, because we are not unconditionally adding Rust into the kernel (much less "rewriting it" as you claim). In fact, one is free to have a C and a Rust version of the same driver and compare whatever metrics you need. Even the RFC contains such an example and I am aware of at least one company doing precisely that.
Moreover, Rust is not an "unknown language" in the sense of an "unknown drug". On the contrary: it was explictly designed to address some issues such as memory-safety.
> A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust.
Not really, because 1. you would need to show the same results would apply to the Linux kernel and 2. we are not rewriting the kernel in Rust.
> Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
As mentioned, I am aware of at least one company running an even better experiment than the one you propose.
If you want faster results, then you will need to put money into it yourself.
I do hope that Rust doesn't suffer from an explosion in complexity and compile times, though. Memory safety and overengineering shouldn't have to go hand in hand.
To me C++ is a language of last resort. It can do everything you need but you should not touch it if there is any other option.
Building from your metaphor, I would add:
Python et al.: normal household batteries; everyone can use/operate them, not much power but can be used with little to no training and a screwup happening means you went looking for trouble
Java et al.: electric batteries; substantially more power but has to stop relatively often (GC pauses), everyone can use the things running on them at least reasonably well [0] but one little thing can potentially cause a catastrophic chain reaction (one cell out of thousands => thermal runaway (eg. the infamous NullPointerException))
Rust, Go et al.: industrial backup generators or hydroelectric plants; substantially more power yet again but have a startup/setup time (learning a more complicated language/ecosystem), as reliable as can be realistically done but have to be spec'd right for the building/installation/etc. at hand (time vs. resources tradeoffs); screwups happening means you probably went looking for trouble (eg. in Rust: excessive cloning of data, building of circular references without weak references; not so sure about specific examples in Go but didn't want to lump it together with Java et al.)
C, C++ et al.: "nuclear power [...] very powerful but dangerous if only used slightly wrong" like in your metaphor; should only really be used by experts but even they are not safe from introducing catastrophic failures with something seemingly harmless (seen as such at the time of writing the code); it gets safer over time but there is the huge inertia of existing installations running with huge time and resource costs associated with dismantling or even modernisations; footguns are everywhere and not always immediately recognisable especially since there recommendations about usage and safety concerns floating around which are sometimes years or decades out of date
[0]: AFAICT a substantial part of the business software world runs on Java code cobbled together by not-necessarily-full-on-experts
PS: My personal experiences/biases:
- learned C++11 for a while, didn't like it, SEGFAULTs and associated debugging/fine-tooth-combing over the some-tens-or-hundreds-of-lines long code got really annoying really quickly
- learned Java, Haskell and Prolog in Uni; Haskell was a nice intro to functional programming but nothing I would want to use even occasionally, Prolog was meh, Java was/is a convoluted, resource-hungry mess with huge amounts of boilerplate for every little thing and will IMHO always be associated with bad assignments in class(es)
- Python3 I use every other week or so but not for "real programming (TM)", but instead for small (automation/download/crawler) scripts; I like it but for serious programming in a business environment - ie. something I personally would want to rely on for money - the duck-typing is just too unstable for my liking, even though I still like it for rapid prototyping of concepts coming into my head which I then later may or may not transfer into:
- Rust... currently and personally my favorite language for "real programming (TM)" even though I struggled with the borrow checker for a long, long time (and still run into it on occasion); sometimes really long compile times but nice runtime performance in most cases even without special care and tuning while I still have the confidence that if it compiles it runs and any errors encountered at testing-/runtime are probably more due to logic errors than running into something like "oops, in this deep branch of cascading ifs I accidently didn't set some object's field or dict's value meaning at some later point the program blew up due to NoneError or something similar"
- learned a bit of Go (before even finding out about Rust) but I just didn't like it, it just didn't "gel with me", don't even really remember the specific reason(s)
Also,
> but has to stop relatively often (GC pauses)
is severely outdated and I don’t see how NPEs are something specific to Java.
Shorter pause times than Go.
These are bad comparisons.
Since I'm on mobile, here's a link to a previous explanation of mine: https://news.ycombinator.com/item?id=1422160 It is meant to be read in conjunction with the original post, addressing what I saw as the point most people were missing, not a complete restatement of his points.
Your post also mentions operator overloading, which is a bit nastier since the calls aren’t explicit. But Rust also has operator overloading, so similar complaints would probably apply? In practice I’d expect the kernel to be fairly strict about limiting operator overloading (which would apply for both C++ hypothetically as well as Rust).
I can't directly speak to Rust's overloading support. If it's still one static type lookup he may be ok with it, plus you have the clarity around ownership of any intervening constructions. I'm sure the Rust Evangelism Strike Force (awesome 80s cartoon music intensifies) will be along presently to answer the question of how much assembler one can generate with an overloaded operator.
C++ really does have a particular ability to stack a huge number of abstractions into one little line. Ultimately it's a continuum rather than a sharp line, but C++ is really far on one side.
I did get your point but would like to point out that using these slurs isn't helping any discussion.
There are fanatics in every area everywhere. Their existence doesn't devalue any of the bigger groups they belong to. Their existence is a fact of life.
I've worked with plenty of Rust devs (and I am an aspiring one myself although not all the way there yet) and I've only ever seen hard-working people with an excellent eye for detail and desire to maximally utilize the hardware and have better safety than other languages they worked with in the past. Zero fanatics.
Just at the moment, C++ is notably better at this, in important cases, but I gather there are plans for Rust to adopt more of C++'s capabilities. The Rust designers fully understand the cases I mention, but are very busy.
Linus's ignorant diatribe about C++ tells us plenty about Linus, but nothing meaningful about C++ or its suitability for kernel coding. That every detail he rails about in C++ applies equally to Rust is unintentionally revealing. That each such detail makes C++ and Rust better for large systems, moreso.
Take some vintage C++ code from late nineties, like games, and there's no concept of ownership, before smart pointers and move, and full of global's everywhere.
It was also when code were transitioning for a multi-threading era, so using the same code style of the previous sequential era was really bad.
Of course, all the new features C++ gained since '98 only make the language more useful for coding large, long-lived systems, as is amply demonstrated in the tens or hundreds of billions of lines of C++ code implementing exactly those systems. Rust's adoption of so many of those same features increases its usefulness for a growing number of those same purposes.
Google chose C++ 17 as the implementation language of the Zircon kernel for the fuchsia OS.
https://fuchsia.dev/fuchsia-src/development/languages/c-cpp/...
PS: C++ compile times are actually shorter than rust compile times.
For this kind of project its more important to have people with pratical experience in kernels, than experience in a particular language.
These people will have a experience in languages they used before, so even if the decision was taken today, you wouldn't find enough people with experience in kernels AND Rust, but will have much more people with experience in kernels C and/or C++, so C++ would still be a good default in comparison to C.
It would be pretty amateurish to get the kernel expert folks to learn Rust while implementing it, as it would probably mean that the end result in Rust would be worse than the end result in C++ as they were more comfortable and experienced in that language.
It will take more time to form specialists in some key areas so that decision can be made with less side-effects.
And many of these were _bad_, C++ was a different language back then and the culture of how you write C++ was different. There were times when it was fashionable to write OO spaghetti almost as bad as the stereotypes about java, or templates that worked except for some unforeseen footgun (then you could template libraries that worked correctly but figuring out how was a trip). And before all of that there was a decade when C++ compilers were just _full_ of bugs, and before that there was no standard and every compiler was different. The negative stereotypes about C++ come from all that.
I think he had every reason to fear that e.g. he would have to fight someone who implemented control flow using an inheritance hierarchy because it's a pattern. And he could find himself in the minority.
(Also I'd say there are still counterproductive fads, and it's still common to write C++ that's beautiful on the outside and completely inscrutable inside, or generally pay a huge price in code size and complexity for marginal gains elsewhere, or to depend on optimizations that don't trigger. Same in many other higher-level languages but somehow not in C.)
Second, more important, i don't remember any talk of adapting C++ to bare metal programming, while there is an ongoing cooperation with rust folks already.
C++ already is used in bare metal environments and in at least one production kernel (XNU) so there just isn't any adapting to do. The standard explicitly distinguishes between hosted and freestanding environments and defines what you should expect to have available.
That does not mean you would try to use virtual functions (much), or std::vector or std::shared_ptr (ever). Instead, you would use abstractions tailored precisely to the kernel environment, and arrange that wrong code would tend to not compile, so that when code compiles, it works. In C, you have to enforce all conventions by obeying instructions in comment blocks, and updating all uses every time the comment block changes. In C++, you can encode the rules directly into the type system and put that to work, not just checking correctness, but actually generating correctness.
This is the same as one would do in Rust modules. Failing to make full use of Rust's type system would be a grave mistake that we need not fear will happen.
https://news.ycombinator.com/item?id=26831841 (292 comments)
https://news.ycombinator.com/item?id=26812047 (262 comments)
There is a lot more hardware and software now than when Linux got started.
For something different to overcome the initial bootstrapping problem it will need to be significantly better and be pushed by a very large player.
Googles Fuchsia is the only possible current candidate I know of.
> You're asking to join us, not the other way around. I'm fine in a world without Rust.
Such a luddite view. "My car gets 40 rods to the hogs head..." I'm always amazed to see such stuck-in-the-mud views from people working in technology which is such a modern fast moving field.
That's something you have to experience first hand to understand, but you know that feeling you get writing C or C++ where new code appears to work the first time you run it? "That's suspicious. It must not have rebuilt, or maybe I forgot to call the new code."
With Rust the compiler can catch so many mistakes that by the time you actually run the code there's a pretty good chance that it is right. My instinct is no longer "suspicious! Where have I gone wrong?"; it's "ha it worked! Though I'd still better double check".
On the other hand, I continue to be amused by arguments along the lines of "I'd already made up my mind that Rust is bad, but then I actually looked at some Rust code and realized it doesn't use C syntax, and now I'm even more upset"...
Sorry, what does this mean exactly? Could you come up with a hypothetical sentence that is like this? Thanks!
Rust's tooling isn't quite as mature as it could be, and it achieves its advanced capabilities and performance by eschewing a runtime (which would make it unacceptable) in favour of labourious compilation.
The snark to me seems unnecessary; a more even headed discussion would acknowledge the limitations and focus instead on what can and will be improved or mitigated and what compromises are acceptable.
The tooling will mature with time, panics will be mitigated, and the compile time performance is often a sacrifice worth making for the sake of the safety provided.
Implications like that may very well be correct, but as always, not sticking to the technical aspects of the subject makes it harder to get anything done.
Can we have a compiler flag or a Cargo switch inside the project's file that disables all std API that can panic? Basically complain that the function is not found.
I for one always strive to write my Rust code with `Result<T>` and never rely on panics so I'd love it if that can actually be enforced and checked by the compiler.
But I hope that if it's both possible and not very hard, then it will be considered. I'd personally love to double down on Rust's safety and dial it to eleven in all my hobby and commercial projects.
> [T]he interest in allowing Rust modules into the kernel are a problem for us, due to Rust trademark restrictions which prevent us from applying patches in our distribution without express permission. We patch to remove non-free software, unlicensed files, and enhancements to user-privacy anywhere it is applicable. We also expect our users to be able to re-use our code without any additional restrictions or permission required.
> This is also in part why we use UXP, a fully free browser engine and application toolkit without Rust, for our mail and browser applications.