"In this paper, we show that it is possible to incrementally transition the kernel code from C to a memory safe programming language, D, by porting and integrating a device driver. In addition, we propose a series of code transformations that allow the D compiler to reason about the safety of certain memory operations. Our implementation increases the security guarantees of the kernel without incurring any performance penalties."
> Our choice of D was based on three criteria: syntax similarity to the C programming language, interoperability with C programs and high performance generated code.
I'm a bit perplexed tying to figure out how any of these criteria is a reason to choose D over rust
We believe that the D programming language can be a great candidate for writing kernel device drivers in a safe programming language.
D is fully compatible with C and mechanically checks, during compilation, the user code for unsafe behavior. The language also boasts powerful compile-time features that make it possible to write highly specific, high performance code.
It also has a smooth learning curve and does not force language features onto the users, allowing them to gradually improve their code base, as they become more comfortable with the language.
We were able to integrate D in the kernel with minimal effort. It took us (a team of 4) 3 months to port the driver to D. We were able to use the existing kernel build system and weren't required to add extra support in the kernel for D.
Both D and Rust are a great step forward to writing safe software.
Imhmo, having multiple options to choose from helps developers.
Doesn’t D require a GC to achieve the safety guarantees that Rust provides without one, and many third-party libraries require this GC? Just looking over a comparison between them on Reddit.
Memory safety is broken up into a bunch of different categories. It isn't just one feature.
Yes D uses the GC for lifetime issues currently, but it does not need it for doing bounds checking, escape analysis or preventing common issues surrounding pointers. All of which are very useful things to have with or without the GC.
Just those features alone would prevent some pretty big name issues that have cropped up in C code over the years.
> Doesn’t D require a GC to achieve the safety guarantees that Rust provides without one
No. The paper covers this. The authors used D's fat pointers, scope, slices, @safe, and ownership/borrowing features to improve code safety.
It is trivial to turn off D's GC, and the authors did not use it in the project described in their paper.
> and many third-party libraries require this GC?
Yes, a lot of D libraries depend on GC, which means they wouldn't be available for use for kernel development.
The D compiler will error out if D code marked with the @nogc attribute depends on GC (or depends on code dependent on the GC), which makes going GCless easier.
D has attributes @safe and @nogc. You can use @nogc to ensure that your code isn't calling the GC, and (though there are apparently a small number of holes in it) you can use @safe to ensure that your code is memory safe. Of course, if you limit yourself to @safe @nogc code, then you might have some difficulty with the standard library (as well as third-party libraries). There are things that the compiler can statically confirm are safe in Rust, that a D compiler is not yet able to match (there is some work on @live to enable somewhat similar features).
D is realistically somewhere in a middle ground between C and Rust for kernel work. It leans more toward familiarity for C programmers with some added safety checks (indeed, you do lose something by not using a garbage collector, but you can still go a pretty long way without it thanks to auto bounds checks and such) but does not enable as much static checking - nor require as much new learning - as Rust.
D is pretty easy to pick up and get productive in, even compared to C. I imagine it's probably harder to use with @nogc and @safe, but it's still a great improvement over C. On the plus side it integrates nicely with C code and libraries.
> I'm a bit perplexed tying to figure out how any of these criteria is a reason to choose D over rust
The first one (syntax similarity to the C programming language) is obviously a valid reason to prefer D over Rust for Linux kernel programming (which mostly uses a dialect of C). The other two (interoperability with C programs and high performance generated code) are a necessity for a programming language to be considered for Linux kernel programming (and Rust scores highly on both). So the first criteria would favor D, while the other two would be neutral between D and Rust.
(There probably are other criteria, outside these three, which favor Rust over D, given that Rust support has already been tentatively merged into the Linux kernel.)
It has taken huge amounts of work to get the Rust ecosystem to where it is. Even still, it has huge regions where it is immature. Is anyone in the D community able to talk about how mature the ecosystem is?
I had always assumed (perhaps naively) that not many people used D and that it was a bit esoteric…? Not to offend anyone. Kind of like F.
Should developers be investing time into learning D?
D has been around for 21 years. It has had a compiler in the gcc project (gdc) for four years. It debugs easily enough under gdb. Several editors support D syntax (including geany and vscode). It has quite a few libraries (though its interoperability with C libraries has muted native D library development somewhat), centralized at https://code.dlang.org/
Are there specific parts of the ecosystem you worry about not being mature?
As for it being a bit esoteric, yes, it's not a very widely known language, though I'm not sure why. I fell in love with it in 2018, and it has thoroughly spoiled me for writing C.
One big issue with D compared to Rust for performance-oriented code is the lack of move semantics. Rust and C++ idiomatically almost never rely on copy-on-write, whereas D so far has a lot of it. I'm not sure how much this matters for kernel modules, but I imagine that it is something users would consider. I had seen some proposals for move semantics in D, so for all I know this is under works right now.
Maturity in a systems programming sense is different. Rust had a lot of effort pored (and still going) into the nostd variant to be used in bare metal applications like microcontrollers and kernels. Does D have that same level of formalism?
On a more serious note, I’m skeptical D would take. It seems like Rust has captured mind share and already has ongoing substantial work integrating it into the kernel (+ it’s the first and only non C language in kernel mainline). Given that context, I don’t see the motivation that works spur a similar level of support from kernel maintainers.
I'm sorry, but shouldn't we critique Dlang itself instead of the constant comparisons with Rust?
Dlang has had significant work poured into it, and the project has delivered a quality language with safety in mind. I don't mean to be defensive, but these threads always seem to ask the same question, it's boring at this point.
I think it's fair game to discuss Rust in this context since there's a major effort underway to support writing in-kernel drivers in Rust. Unless you think the kernel will be a bazaar of languages, then it's useful to think about which language is the most likely to "win".
There are other OSes out there where D could eventually make a mark.
Naturally the whole issue is that there is no big movement regarding writing OSes in D for that to matter anyway.
Better luck using TinyGo, Java, .NET or Oberon in bare metal workloads (where runtime plays the OS role), as there are a couple of products for them, than D.
> Rust had a lot of effort pored (and still going) into the nostd variant to be used in bare metal applications like microcontrollers and kernels.
D has had "dcompute" since 2017, for native execution on GPUs and other accelerators. People have been writing D for microcontrollers since at least 2011, possibly earlier. D is fairly mature in the systems space.
> I’m skeptical D would take.
Maybe, maybe not. Some languages "click" better than others for individual programmers. It's good to have options.
> it’s the first and only non C language in kernel mainline
Do you foresee it being the only non-C language used in the kernel forever?
> Do you foresee it being the only non-C language used in the kernel forever?
It's possible that maybe the maintainers are just trying out Rust to see if it can be wrangled into the kernel to start the migration process. So now may be the right time to explore other languages.
Once a next language is "chosen" then yes, I do think that there will be only 1 language for the kernel itself (modulo the very long transition time to convert all components so you'll have C + 1 other language). I wouldn't be surprised if at some point the kernel decided they're not going to accept any more C drivers because driver code is a lot more variable in quality and lower in oversight and testing. Core kernel code will take a longer time as even now that's out-of-scope for most such efforts.
30 comments
[ 2.8 ms ] story [ 79.9 ms ] threadI'm a bit perplexed tying to figure out how any of these criteria is a reason to choose D over rust
It also has a smooth learning curve and does not force language features onto the users, allowing them to gradually improve their code base, as they become more comfortable with the language.
We were able to integrate D in the kernel with minimal effort. It took us (a team of 4) 3 months to port the driver to D. We were able to use the existing kernel build system and weren't required to add extra support in the kernel for D.
Both D and Rust are a great step forward to writing safe software. Imhmo, having multiple options to choose from helps developers.
Yes D uses the GC for lifetime issues currently, but it does not need it for doing bounds checking, escape analysis or preventing common issues surrounding pointers. All of which are very useful things to have with or without the GC.
Just those features alone would prevent some pretty big name issues that have cropped up in C code over the years.
No. The paper covers this. The authors used D's fat pointers, scope, slices, @safe, and ownership/borrowing features to improve code safety.
It is trivial to turn off D's GC, and the authors did not use it in the project described in their paper.
> and many third-party libraries require this GC?
Yes, a lot of D libraries depend on GC, which means they wouldn't be available for use for kernel development.
The D compiler will error out if D code marked with the @nogc attribute depends on GC (or depends on code dependent on the GC), which makes going GCless easier.
The first one (syntax similarity to the C programming language) is obviously a valid reason to prefer D over Rust for Linux kernel programming (which mostly uses a dialect of C). The other two (interoperability with C programs and high performance generated code) are a necessity for a programming language to be considered for Linux kernel programming (and Rust scores highly on both). So the first criteria would favor D, while the other two would be neutral between D and Rust.
(There probably are other criteria, outside these three, which favor Rust over D, given that Rust support has already been tentatively merged into the Linux kernel.)
I had always assumed (perhaps naively) that not many people used D and that it was a bit esoteric…? Not to offend anyone. Kind of like F.
Should developers be investing time into learning D?
Are there specific parts of the ecosystem you worry about not being mature?
As for it being a bit esoteric, yes, it's not a very widely known language, though I'm not sure why. I fell in love with it in 2018, and it has thoroughly spoiled me for writing C.
It will just plain copy certain types on assignment though so perhaps that's what you meant.
On a more serious note, I’m skeptical D would take. It seems like Rust has captured mind share and already has ongoing substantial work integrating it into the kernel (+ it’s the first and only non C language in kernel mainline). Given that context, I don’t see the motivation that works spur a similar level of support from kernel maintainers.
Dlang has had significant work poured into it, and the project has delivered a quality language with safety in mind. I don't mean to be defensive, but these threads always seem to ask the same question, it's boring at this point.
Naturally the whole issue is that there is no big movement regarding writing OSes in D for that to matter anyway.
Better luck using TinyGo, Java, .NET or Oberon in bare metal workloads (where runtime plays the OS role), as there are a couple of products for them, than D.
This implies comparing them, so I understand your point now.
D has had "dcompute" since 2017, for native execution on GPUs and other accelerators. People have been writing D for microcontrollers since at least 2011, possibly earlier. D is fairly mature in the systems space.
> I’m skeptical D would take.
Maybe, maybe not. Some languages "click" better than others for individual programmers. It's good to have options.
> it’s the first and only non C language in kernel mainline
Do you foresee it being the only non-C language used in the kernel forever?
It's possible that maybe the maintainers are just trying out Rust to see if it can be wrangled into the kernel to start the migration process. So now may be the right time to explore other languages.
Once a next language is "chosen" then yes, I do think that there will be only 1 language for the kernel itself (modulo the very long transition time to convert all components so you'll have C + 1 other language). I wouldn't be surprised if at some point the kernel decided they're not going to accept any more C drivers because driver code is a lot more variable in quality and lower in oversight and testing. Core kernel code will take a longer time as even now that's out-of-scope for most such efforts.
Do you see it differently?
https://github.com/alexandrumc/d-virtio
https://dconf.org/2019/talks/militaru.html