As a longtime embedded C developer (20 years) I always look forward to truly learn Rust someday. But unfortunately I don't have much time now to learn it. And I'm in a kind-of hate-love relationship with Rust at the moment. The same applies to modern C++.
> many people report a sense of high "cognitive overhead" in using it, and "learning curve" remains the most common reason not to use Rust
This is me.
And I was super excited when I read "we will identify and eliminate many of those patterns and idiosyncracies".
Then... the post follows with "extend the language", "Extend our async-await support", "Broaden the set of traits", and I know what this means. More things to learn, more time to reach an operative state.
RISR – Rust 1.0 was that language. It was already a significant reduction over some other languages: having no GC makes the language simpler implementation-wise than 95% of modern languages. Memory safety is then brought back in by ownership and lifetimes, but those _restrict_ what you can do, not the other way. In a way, Rust 1.0 felt like a minimal design that achieved the goals: great performance and low-level control with memory safety without GC.
(Okay, there were some stuff around traits and type inference that were not _strictly_ necessary, I guess.)
However, it was _not_ nice to use. Even with current Rust, which is a whole lot more ergonomic, there are many places that feel unnecessarily restrictive. Patterns that feel like they should work, yet they don't.
Do you have any better idea how to make this stuff better without extending the language? Or maybe "worse is better", in which case Rust is not for you?
New features is not incompatible with easier to learn.
User will have to learn at some point how to do X with the language. Without the new feature, that might be some complex pattern which is difficult for user to remember, let alone come up with by themselves. While using an additional feature it may be trivial to learn that feature to do X.
Don't get discouraged by this. The features being added may sound academic and obscure, but they make the language simpler to use.
This pattern has already been repeated a few times:
Before: "Uuugh, I want to do X, but the compiler says I can't! Why can't I do this simple thing!??"
After: It works.
For example, Rust used to have an embarrassing problem with cloning arrays larger than 32 elements, for silly obscure reasons. Since then it has added a "const generics" feature, and cloning and iteration over arrays works just as you'd expect it to always work.
The current issue is that traits (which are a bit like interfaces in Java) don't quite support async functions, for silly obscure reasons. Currently if you want to have an interface that is both abstract and async, you need to manually implement workarounds that add complexity and overhead. Rust is removing this problem by adding language-level plumbing that was preventing it from having traits just work for any type of function.
Rust's evolution isn't adding bells and whistles, but filling in holes.
This is the perfect example. For instance, I don't need Traits or Async or generics for embedded. I can pass a struct into functions as I've been doing my whole life. Async is not supported (but I can generate my own interrupt-driven state machines).
In C I don't have an "abstract and async traits interface". I wouldn't even imagine why I need one "in the language". Do I need one? Does a bluetooth stack or kernel driver need one? I can make a struct with pointers to functions, and there you have an "interface", done with the basic elements the language provides.
I undertand that these features are a "nice-to-have plus", but what all these updates actually generate in a system/embedded developer like me is fatigue, and not only at cognitive level but also "I've changed compiler version and I have to revalidate all over again". Why? some purple-haired 25 years old across the ocean needed an "abstract and async traits interface". Now the bread on my family table depends on what they've changed.
I imagine that perhaps Rust tries to make everyone happy, but here are my two cents from my PoV.
Rust doesn't cater to small (short) programs, which some embedded programs may be. It's been originally envisioned for making a modern browser engine, which is almost an entire OS these days, so many of its features start making sense only when you need to work with programs big enough that you can't fit them all in your head at once, and have to maintain for a long time, cooperate with other developers, deal with 3rd party code, etc.
Features that are layers of abstraction are hard to explain on their own, so I don't think I can justify them in just a HN comment. I remember when I have been reading about Encapsulation in OOP for the first time, and I completely didn't get it — why would I hide struct fields from myself!? I know they're there!
Rust's traits are such an abstract feature that doesn't directly translate to things that CPU does, but it's a building block for a few higher-level language features:
• Traits can be used like a Base Class or Interface in OOP, so for example you can make structs "Serializable" and have a data format writer know how to dump them to disk (assuming you need something smarter than fwrite(&struct) due to pointers or portability). Instead of having a one-off function for serializing your particular state, you can make each struct describe itself and "just work" when combined with a JSON/Msgpack/binary serializer. Rust makes good use of this for https://serde.rs that is pretty nice to use and surprisingly efficient.
• Traits can be used with static dispatch and combined with other types at compile time ("Generics" in Rust jargon). It makes them replace many things you'd do with horrible C macro hacks, and you get usually nicer syntax and type checking. For example, you can use containers (hash maps, btree maps) without void* cost and casts, and without include macro hacks. Linear search and linked lists can get you far, but when you need a better container, it's really nice to have best-in-class one available, and as fast as if you handcrafted it yourself.
• If you prefer the struct-of-function-pointers approach, Traits can do dynamic dispatch that too ("dyn Trait" in Rust jargon).
• Traits are nice to coordinate APIs across libraries. For example in C if you want to stream a file from a network, decompress it on the fly, and decode JSON, you need to write a bunch of glue code, because every library handles buffers slightly differently. In Rust there's io::Read trait, and people agreed to use it, so `json_decode(ungzip(request(addr)))` works, and it's streaming (functions here don't return malloced char*, they return sockets/handles that implement a trait that describes how to get more bytes from them). Sort of like a file descriptor, but using general-purpose feature, and it can also be integrated at compile time, so everything is as inlined and optimized as bespoke code.
Embedded doesn't mean "small programs that fit in a head". Some projects I maintain are tens of thousands LoC. Modern microcontrollers are being built with 1 or 2 MB of flash (where a full exFAT FS plus an SDXC driver fits in around 30KB, to give you an idea).
Look, I understand and really appreciate your answer, but I keep thinking these are niceties, and if I were to implement (I don't know) SSL from scratch on desktop/server or a browser as you said, I'll probably consider Rust, because of these niceties.
For the rest, low-level system/kernel/embedded/BSP, I'd prefer what I called RISR in a parent comment. That is, Rust stripped from niceties (traits, async, generics), keeping the language to a minimum with safety as primary objective; and something that hardly updates.
Perhaps it could born as an agreement between embedded/kernel Rust developers, to not got to certain places through certain paths.
Because the more you add, the more complex it becomes, and it will always be difficult to keep everybody happy, test, validate, learn, support and interop with/within different architectures, avoid corner cases, etc. Just for the sake of doing "one-liners" like `json_decode(ungzip(request(addr)))`, which I will never do, because I don't have network access, nor persistent media, nor memory for gzipping, or memory to keep and navigate a JSON structure.
It's like Rust tried to first "convince" javascript/go/haskell programmers, and system developers later, when it should have been the other way around. Because it's super cool to have automatic panics on unwrap() and failed allocations, but the moment Rust has been put to do one of the things it's supposed to be doing (kernel drivers?), the panic thing didn't worked anymore, in April 2021. I'm sure you know what I mean.
I hate that Rust has fallen for the "Linux can't OOM" fallacy too. Fortunately the mistake is only in its stdlib. Rust-the-language doesn't allocate, and kernel/embedded projects turn stdlib off. Panic doesn't have to unwind. It can be a trap instruction or `while(true);`, but in case of an unrecoverable bug it has to stop execution somehow.
Rust's initial goal was specifically Servo, a browser engine. It was based on a bunch 80s/90s programming languages, and Ocaml which is a bit Haskell-y, but JS/Go specifically and explicitly were a non-goal[1].
Direct C replacement for embedded programming was Rust's goal early on too. Async is for the JS/Go crowd, but it's a late addition to the language, and it's also designed to work without any runtime at all. It can be used in embedded and kernel code, and for non-network things too. At its core it's representing function's stack as a struct. Fancier things are just libraries built from it.
If you want compile-time safety and minimal runtime cost, it's unfortunately impossible to do without a more-complex-than-C type system. There really isn't much you can remove from Rust without either creating loopholes in the safety, or losing features that make its safety practical to use. For example, you can't have a safe set that stores void*. You need HashSet<T>, so that the compiler can track of what type it stores (because T could be a struct with a pointer to the stack, etc.). But then you need to hash and compare the arbitrary T type, and that needs some abstraction over types, which traits provide. "Small Safer C" has been tried at least in Checked-C, Zig, and Cyclone, and from these it's clear the problem has inherent "choose two" trade-off of {runtime type checks, complex type system, safety loopholes}.
Programming on a type level is definitely is a very different mindset from what C does, but dismissing it as merely unnecessarily complex is a shallow and inaccurate view IMHO. There is an inherent complexity in writing high-performance safe software. The complexity has to go somewhere. If the language doesn't have tools to with it, then you have to deal with it in the software yourself (like the level of vigilance and tooling that's required to have safe C). Rust is a bigger language, but its toolbox is for reduction of complexity in the software you write. The things you dismiss are not optional nice-to-haves, they are part of the solution, and the pay-off for not having to deal with the complexity in the software, because it's already dealt with in the language.
Yeah, I know I was being dismissive. Sorry. But I feel and understand the pain, too. I know if I have to make a safe function that can operate over any kind of data then I would have to accept a void* (or uint8_t*) and a size, and trust the caller, which at the end of the day is what this is all about. A way to fix it would be to have some sort of generic values and a way to interact with them. I get it.
On the other hand, "trusting the caller" is also how the world operates today (like files and sockets) so any improvement is a "nice thing to have" (depending on the degree of complexity it carries).
Like: constructors/destructors sound like a good idea! Fast-forward 20 years and then you have the rule of 3, of 5, etc.
The line of thought is that if Generics increase my firmware size then I wouldn't use them if possible. If I don't use Generics (if possible), then I probably won't need other things. I definitely don't need async. The language is growing to a height and depth I don't really care.
But if you say that there's no escape and there is little to remove and this is the way it is, then I take it for granted. I can only hope there isn't much to add either so I can catch up someday! Perhaps I have to look elsewhere.
BTW, C++ shat the bed for other better-than-C languages, but C++’s complexity and design failures are their own. In Rust you don’t have rule-of-whatever, it won’t let you write a half-assed destructor. There’s one way to initialize a variable.
Rust is becoming a primary language for modern blockchains and hence the roadmap should account for their demands as well, particularly three of them:
1. Readable and debuggable macros. Blockchain devs use them heavily, and it's virtually impossible to understand what's going on for a person, who wasn't involved in the development of that code with macros.
2. Floating point operations reproducibility across environments. When it comes to finding and hashing the consensus among many machines about floating point calculations, Rust gets ugly: in fact, as a developer, you'll have to use either integers or a made up fixed point numbers instead of floats to ensure that calculations are identical on different machines. You need that because hash function will be completely different for two numbers, which are even slightly different.
3. Zero-exception safety mode checking at compile-time. Blockchain code should run forever and never halt. Because if it does, the whole blockchain is at risk of stopping, and some chains may never recover from that. So, just as good as Rust prevents poor memory management, there should be a special optional compile-time check, that prevents runtime exceptions and halts, and forces the developer to wrap all possible points of exceptions in nice workarounds.
Blockchain devs are the best Rust adopters out there, we're excited to see how the language is becoming defacto a standard for web3 development.
I agree this is a concern. However, proc-macro errors can be improved, and the people that should (and can) do so are their developers, not the compiler developers (although if they can it will be good, but I doubt it).
> Floating point operations reproducibility across environments.
I'm not proficient with blockchain, but Rust's floating point is and will always be what the hardware provides. Libraries can provide custom float types.
> Zero-exception safety mode checking at compile-time.
This is what `Result` is for. If you want to also catch panics, you can use `std::panic::catch_unwind()`. There is no good way to prevent them at compile time, and cannot be without major changes to the language (though see https://github.com/dtolnay/no-panic).
13 comments
[ 3.5 ms ] story [ 55.5 ms ] thread> many people report a sense of high "cognitive overhead" in using it, and "learning curve" remains the most common reason not to use Rust
This is me.
And I was super excited when I read "we will identify and eliminate many of those patterns and idiosyncracies".
Then... the post follows with "extend the language", "Extend our async-await support", "Broaden the set of traits", and I know what this means. More things to learn, more time to reach an operative state.
And this kicks in: https://en.wikipedia.org/wiki/Overchoice
Someone should invent RISR (Reduced Instructions Set Rust).
(Okay, there were some stuff around traits and type inference that were not _strictly_ necessary, I guess.)
However, it was _not_ nice to use. Even with current Rust, which is a whole lot more ergonomic, there are many places that feel unnecessarily restrictive. Patterns that feel like they should work, yet they don't.
Do you have any better idea how to make this stuff better without extending the language? Or maybe "worse is better", in which case Rust is not for you?
Maybe you are looking after Zig? https://ziglang.org/
Of course not! If I have to choose, regarding system/embedded programming languages, then I'd go with "less is better".
> Maybe you are looking after Zig?
Definitely Zig is under my radar too.
Fortunately, after all, I'm not alone. Many others I hear from the industry feel the same way.
User will have to learn at some point how to do X with the language. Without the new feature, that might be some complex pattern which is difficult for user to remember, let alone come up with by themselves. While using an additional feature it may be trivial to learn that feature to do X.
This pattern has already been repeated a few times:
Before: "Uuugh, I want to do X, but the compiler says I can't! Why can't I do this simple thing!??"
After: It works.
For example, Rust used to have an embarrassing problem with cloning arrays larger than 32 elements, for silly obscure reasons. Since then it has added a "const generics" feature, and cloning and iteration over arrays works just as you'd expect it to always work.
The current issue is that traits (which are a bit like interfaces in Java) don't quite support async functions, for silly obscure reasons. Currently if you want to have an interface that is both abstract and async, you need to manually implement workarounds that add complexity and overhead. Rust is removing this problem by adding language-level plumbing that was preventing it from having traits just work for any type of function.
Rust's evolution isn't adding bells and whistles, but filling in holes.
In C I don't have an "abstract and async traits interface". I wouldn't even imagine why I need one "in the language". Do I need one? Does a bluetooth stack or kernel driver need one? I can make a struct with pointers to functions, and there you have an "interface", done with the basic elements the language provides.
I undertand that these features are a "nice-to-have plus", but what all these updates actually generate in a system/embedded developer like me is fatigue, and not only at cognitive level but also "I've changed compiler version and I have to revalidate all over again". Why? some purple-haired 25 years old across the ocean needed an "abstract and async traits interface". Now the bread on my family table depends on what they've changed.
I imagine that perhaps Rust tries to make everyone happy, but here are my two cents from my PoV.
Features that are layers of abstraction are hard to explain on their own, so I don't think I can justify them in just a HN comment. I remember when I have been reading about Encapsulation in OOP for the first time, and I completely didn't get it — why would I hide struct fields from myself!? I know they're there!
Rust's traits are such an abstract feature that doesn't directly translate to things that CPU does, but it's a building block for a few higher-level language features:
• Traits can be used like a Base Class or Interface in OOP, so for example you can make structs "Serializable" and have a data format writer know how to dump them to disk (assuming you need something smarter than fwrite(&struct) due to pointers or portability). Instead of having a one-off function for serializing your particular state, you can make each struct describe itself and "just work" when combined with a JSON/Msgpack/binary serializer. Rust makes good use of this for https://serde.rs that is pretty nice to use and surprisingly efficient.
• Traits can be used with static dispatch and combined with other types at compile time ("Generics" in Rust jargon). It makes them replace many things you'd do with horrible C macro hacks, and you get usually nicer syntax and type checking. For example, you can use containers (hash maps, btree maps) without void* cost and casts, and without include macro hacks. Linear search and linked lists can get you far, but when you need a better container, it's really nice to have best-in-class one available, and as fast as if you handcrafted it yourself.
• If you prefer the struct-of-function-pointers approach, Traits can do dynamic dispatch that too ("dyn Trait" in Rust jargon).
• Traits are nice to coordinate APIs across libraries. For example in C if you want to stream a file from a network, decompress it on the fly, and decode JSON, you need to write a bunch of glue code, because every library handles buffers slightly differently. In Rust there's io::Read trait, and people agreed to use it, so `json_decode(ungzip(request(addr)))` works, and it's streaming (functions here don't return malloced char*, they return sockets/handles that implement a trait that describes how to get more bytes from them). Sort of like a file descriptor, but using general-purpose feature, and it can also be integrated at compile time, so everything is as inlined and optimized as bespoke code.
Look, I understand and really appreciate your answer, but I keep thinking these are niceties, and if I were to implement (I don't know) SSL from scratch on desktop/server or a browser as you said, I'll probably consider Rust, because of these niceties.
For the rest, low-level system/kernel/embedded/BSP, I'd prefer what I called RISR in a parent comment. That is, Rust stripped from niceties (traits, async, generics), keeping the language to a minimum with safety as primary objective; and something that hardly updates.
Perhaps it could born as an agreement between embedded/kernel Rust developers, to not got to certain places through certain paths.
Because the more you add, the more complex it becomes, and it will always be difficult to keep everybody happy, test, validate, learn, support and interop with/within different architectures, avoid corner cases, etc. Just for the sake of doing "one-liners" like `json_decode(ungzip(request(addr)))`, which I will never do, because I don't have network access, nor persistent media, nor memory for gzipping, or memory to keep and navigate a JSON structure.
It's like Rust tried to first "convince" javascript/go/haskell programmers, and system developers later, when it should have been the other way around. Because it's super cool to have automatic panics on unwrap() and failed allocations, but the moment Rust has been put to do one of the things it's supposed to be doing (kernel drivers?), the panic thing didn't worked anymore, in April 2021. I'm sure you know what I mean.
Rust's initial goal was specifically Servo, a browser engine. It was based on a bunch 80s/90s programming languages, and Ocaml which is a bit Haskell-y, but JS/Go specifically and explicitly were a non-goal[1].
[1]: http://venge.net/graydon/talks/intro-talk-2.pdf
Direct C replacement for embedded programming was Rust's goal early on too. Async is for the JS/Go crowd, but it's a late addition to the language, and it's also designed to work without any runtime at all. It can be used in embedded and kernel code, and for non-network things too. At its core it's representing function's stack as a struct. Fancier things are just libraries built from it.
If you want compile-time safety and minimal runtime cost, it's unfortunately impossible to do without a more-complex-than-C type system. There really isn't much you can remove from Rust without either creating loopholes in the safety, or losing features that make its safety practical to use. For example, you can't have a safe set that stores void*. You need HashSet<T>, so that the compiler can track of what type it stores (because T could be a struct with a pointer to the stack, etc.). But then you need to hash and compare the arbitrary T type, and that needs some abstraction over types, which traits provide. "Small Safer C" has been tried at least in Checked-C, Zig, and Cyclone, and from these it's clear the problem has inherent "choose two" trade-off of {runtime type checks, complex type system, safety loopholes}.
Programming on a type level is definitely is a very different mindset from what C does, but dismissing it as merely unnecessarily complex is a shallow and inaccurate view IMHO. There is an inherent complexity in writing high-performance safe software. The complexity has to go somewhere. If the language doesn't have tools to with it, then you have to deal with it in the software yourself (like the level of vigilance and tooling that's required to have safe C). Rust is a bigger language, but its toolbox is for reduction of complexity in the software you write. The things you dismiss are not optional nice-to-haves, they are part of the solution, and the pay-off for not having to deal with the complexity in the software, because it's already dealt with in the language.
On the other hand, "trusting the caller" is also how the world operates today (like files and sockets) so any improvement is a "nice thing to have" (depending on the degree of complexity it carries).
Like: constructors/destructors sound like a good idea! Fast-forward 20 years and then you have the rule of 3, of 5, etc.
The line of thought is that if Generics increase my firmware size then I wouldn't use them if possible. If I don't use Generics (if possible), then I probably won't need other things. I definitely don't need async. The language is growing to a height and depth I don't really care.
But if you say that there's no escape and there is little to remove and this is the way it is, then I take it for granted. I can only hope there isn't much to add either so I can catch up someday! Perhaps I have to look elsewhere.
1. Readable and debuggable macros. Blockchain devs use them heavily, and it's virtually impossible to understand what's going on for a person, who wasn't involved in the development of that code with macros.
2. Floating point operations reproducibility across environments. When it comes to finding and hashing the consensus among many machines about floating point calculations, Rust gets ugly: in fact, as a developer, you'll have to use either integers or a made up fixed point numbers instead of floats to ensure that calculations are identical on different machines. You need that because hash function will be completely different for two numbers, which are even slightly different.
3. Zero-exception safety mode checking at compile-time. Blockchain code should run forever and never halt. Because if it does, the whole blockchain is at risk of stopping, and some chains may never recover from that. So, just as good as Rust prevents poor memory management, there should be a special optional compile-time check, that prevents runtime exceptions and halts, and forces the developer to wrap all possible points of exceptions in nice workarounds.
Blockchain devs are the best Rust adopters out there, we're excited to see how the language is becoming defacto a standard for web3 development.
I agree this is a concern. However, proc-macro errors can be improved, and the people that should (and can) do so are their developers, not the compiler developers (although if they can it will be good, but I doubt it).
> Floating point operations reproducibility across environments.
I'm not proficient with blockchain, but Rust's floating point is and will always be what the hardware provides. Libraries can provide custom float types.
> Zero-exception safety mode checking at compile-time.
This is what `Result` is for. If you want to also catch panics, you can use `std::panic::catch_unwind()`. There is no good way to prevent them at compile time, and cannot be without major changes to the language (though see https://github.com/dtolnay/no-panic).