Technically no (only for half of libraries) but effectively yes, if you're comparing to C and C++. The fact that there might be an `unsafe` somewhere in your dependency tree just means that instead of being there being a 0% chance of a memory bug, there's 0.0001%. Whereas a C/C++ codebase of moderate size is virtually guaranteed to have many (orders of magnitude might be off but point stands).
I think the problem here is not the absolute safety but the illusion of safety. You approach problems differently when you expect memory bugs to appear compared to thinking the chance is 0%.
A study of the frequency of memory corruption in rust programs would be useful here. If the real-world frequency is let's say <0.1% of bugs uncovered, then I say this mindset is a blessing. Working with C++ the feeling of paranoia you have with 1/3[1] of bugs reported, that maybe again it's memory corruption, is sucking the energy out of developers.
[1]: That's the frequency of this paranoia creeping in that I observed, not of detected memory corruption, but that too I think would be counted in an integer number of percents IME.
That's a great point. Their study is a good starting point but it's hard to get anything concrete out of it. I hope they can continue with something like what you proposed.
The real-world data shows that most Rust libraries fail due to (memory-safe) panics or logic errors leading to resource exhaustion, but plain old memory corruption is quite rare:
I don't think this is as strong a claim here as you think.
The point of unsafe is to point you toward the areas you need to audit. Unsafe code can't be proven memory safe by the compiler [...but the rest can be]. It's far easier to audit 10 or 100 LoC of unsafe rust than 10 or 100 thousand of C++.
And even in C++ there are subsets of the language that are supposedly memory safe. But even when writing in those subsets I'm not convinced that the abstractions I'm building on are as well validated as rust.
There is no illusion of safety. It's 100% safe, within the code you right, if you don't use unsafe. If there is unsafe in code you use (write or shared), it's a handful of lines which you can nearly grep for it's so easy to locate.
Rust devs (like myself) aren't illusioned into believing that my code and _everything below it_ is 100% safe.
However i can positively assure you my code is 100% safe. Well, assuming no bug in Rust-lang/compiler. My _programs_ are not guaranteed to be 100% safe unless no unsafe is used, but Rust lets you know for certain what is safe, and what is unsafe.
The point isn't absolute safety in your program. The point is absolute safety where you think there is absolute safety. To strictly know what is unsafe. And to reduce the total lines that are unsafe.
edit: I'm confused by the downvotes, what is incorrect or controversial about this?
> You approach problems differently when you expect memory bugs to appear compared to thinking the chance is 0%.
The chance is never 0%, and nobody should be thinking that way. Memory safety bugs routinely arise in JITs (e.g. JavaScript engine CVEs). In fact, memory safety issues routinely arise in CPUs (e.g. rowhammer).
In reality, terms like "memory safety" are rarely ever exact, but they're still useful descriptions. Practically, people should be thinking about frequencies, not in black and white.
I've created a memory leak in java. Not a "I forgot to free memory in a map" memory leak but a honest to goodness "This JVM is allocating heap space and losing track of that allocation". Something that required me to install a special malloc in the VM to track down the leak. This was using standard libraries (You can do it too! Go play with the Deflater and Inflater API and you'll find it makes calls off to Zlib. If not properly handled you end up leaking heap allocations).
That was the first and only time I've ran into that issue, but I ran into that issue.
The guardrails rust gives you are essentially the same that the JVM gives you. That is, use only safe code and (barring a compiler bug) you are safe. However, bad things COULD (not will) happen if you use unsafe blocks. Just like bad things can happen in JNI blocks or places where Java makes use of C libraries for functionality.
And that is what makes rust fantastic. It's got memory safety up there with JVM safety without a garbage collector. It even has nifty keywords (unsafe) to keep developers on their toes when they need it. Just like Java's JNI or Pythons C extensions.
In fact, I'd argue that rust unsafe ends up being more safe than either JNI or C extensions primarily because it is something that can be audited very quickly across all rust code bases. Have a memory issue? It's in the unsafe block.
Let's not forget that there are other languages that are memory safe and might be good enough for many tasks, so limiting the comparison only with C/C++ feels unfair to me.
Every serious C/C++ shop I've seen is seriously considering incorporating or switching to Rust. I don't think that's true for many other languages unless they would otherwise be switching to an unsafe language like C/C++ for performance reasons, which is probably a minority.
Are they switching mid project and doing the "Rewrite It in Rust" meme? Or only for new project?
If for new projects what type of project that is not some kernel, browser or some low level library is Rust the best tool for the job? For most type of projects I am thinking Python,Java, C#, C++ still is better .
Any codecs or anything like that seem like a screamingly good target.
Right now they're one of our most egregious sources of security vulnerabilities (mostly due to a lot of hands-on bit-twiddling that results in a lot of possibilities for overflow). It's gotten bad enough that a number of system developers (browsers, OSes) have been shifting to a model where codecs are strictly sandboxed, and just get a single binary input blob, and a single output blob, and don't get any other kind of access to the system they're on - it's a scorched earth problem, but it really is that bad.
The thing about trying to write one with Rust is, if you actually 'went with the flow of the language' and didn't just immediately pop into `unsafe`, you'd be able to write a really safe codec.
-but-
The real payoff is that the performance would be surprisingly good - usually a suggestion like the one I just made would absolutely wreck performance, but one of the big things about Rust is that because it lets you describe what your intent is as a programmer in such higher-level terms (despite being a machine-level language), it gives the compiler a lot more information about the scope of your intent, which opens the floodgates to potential optimizations.
For a really easy example - Rust does all variables as "immutable by default". When you're programming in C, and something is compiling your code, there are tons of opportunities where you'd go "oh, well we already calculated this, why don't we just cache it?" But the compiler can't, because it doesn't actually know what you, the human, know - that that thing won't change. Because that knowledge is available to the compiler through Rust, the compiler actually DOES know, so tons of things can get inlined, cached, and otherwise optimized for you which you'd otherwise have to do by hand.
Usually when people do these sorts of things by hand, they'll ace a couple key optimizations (usually things that show up in profiling hotspots) and completely overlook hundreds, even thousands of other potential optimizations.
There are tons of other "high level intent" things that give the compiler more info it can use to optimize, but the "immutability by default" one is a really easy one to explain.
In Rust, if I have two u8 variables and I add them together, Rust will try to figure out if it's sure at compile time that this is an overflow and if so reject it, but otherwise I get a binary which might have an overflow in it.
But in WUFFS when you add those variables together, WUFFS wants you to justify why that can't overflow, or, tell it that you know it does overflow and you want wrapping or whatever. As a result, WUFFS gets to be very confident the resulting (awful, machine generated) C is safe and correct.
As you observed, this sort of confidence also permits a lot of performance tricks. WUFFS has some very impressive (ie better than existing Rust code) benchmark scores for the (much simpler) codecs like JPEG or PNG but I don't think there's any reason it couldn't shoot for MPEG or beyond.
WUFFS is no rival to Rust. It doesn't have IO for example. Which is fine, your codec implementation shouldn't do IO. WUFFS deals in blobs of bytes. Bytes you got over the network or from a file go in, and bytes represented a decoded image, decompressed file, or whatever come out.
Also - Rust code can directly link to C++ code and vice-versa. If you're an existing C++ shop with a big codebase (say, a large game engine), you don't have to "rewrite it in rust".
You can just start building new parts in rust. Like, if you're building a new audio subsystem, you can do "just that subsystem" in rust, and expose an interface for the rest of the legacy engine code to use.
TLDR: You've tried to exclude something like 90% of all the systems programming problem domains & then said Rust (a systems programming language) is the wrong tool. Yes, but you're kind of missing the forest for the trees.
You've got MSFT, Facebook, & Google all investing heavily into Rust as a replacement for C/C++. It's still early days. Rust's sweet spot for being the best tool is anything that's currently written in C/C++. If you're writing Python, Java, or C# code, then Rust is probably not the language for you (albeit Rust is getting deployed in all these places where perf is needed because it offers the performance of C/C++ with the safety of more managed languages).
At no point will it be "replace everything in Rust" (at least not in the immediate term). It'll be more like "write this component with a thin API in Rust" or "with codebases that have a C++ runtime, work out a way for the peripheries to be written in Rust".
I'll give you an example. We used the venerable addr2line (or even llvm-symbolizer) to symbolicate addresses for an executable. We used it by spinning up a process for each address to symbolicate (which is dumb, but whatever). By writing a wrapper over Gimli-rs/addr2line-rs, we were able to improve our symbolication time 1000x & reduce our memory footprint 3x. The memory footprint improvement is because Gimli is 0-copy. 100x of that speedup is because Gimli's 0-copy is also just faster at parsing the DWARF data. The next 10x is because we only parsed the DWARF once & then used that precomputed information for all addresses. The fact that that Rust tool was correctly engineered as a library + frontend meant this was relatively easy, cheap & painless to integrate into our existing C++ codebase (using the cxx crate).
Like all things, Rust is a tool. Knowing how to find the right tool for the job & deploying it well is 99% of the job. Personally, I've found the Rust libraries I've come across for systems programming to be higher quality than the equivalent C/C++ libraries because the field is green, the userbase is extremely passionate, enthusiastic & early adopteree (i.e. more technically adept), & Rust makes certain kinds of optimizations easier than they were before (so naturally people want to play around with them). Additionally, because these are all rewrites, 90% of the time the improvement is just applying more modern state of the art knowledge to existing APIs/algorithms (e.g. doing 0-copy for symbolication is a "no-brainer" but it's hard to do safely in C++).
"Rewrite" is not an option for large companies with a ton of legacy code, but any company whose business depends on C++ better be thinking about what their codebase should look like in 10 years and in 20 years.
None of the C++ places I've worked are considering switching. Rust lacks the libraries and ecosystem, and it's hard enough to hire C++ developers as it is.
Every 'serious' C/C++ is seriously considering incorporating or switching to Rust.
So all serious shops are going through this self discovery process but none have decided yet they are weighing the pros and cons daily. Unable to reach that conclusion but moving closer to Rust everyday.
That sounds like someone is hopeful they bet on the right horse.
The title is a little misleading, and the results are at a glance a little alarming. If you base your definition of safe on whether you or your dependencies use unsafe anywhere, then obviously you're going to find something of concern. However, if you measure the safety of your code line by line, then the results are much better. Similarly, there's a balance to be found when assuming unsafe code is unsound.
The question is ultimately a reminder to check your dependencies to make sure they're active and take reasonable steps for validating unsafe code.
> Our results indicate that software engineers use the keyword unsafe in less than 30% of Rust libraries, but more than half cannot be entirely statically checked by the Rust compiler because of Unsafe Rust hidden somewhere in a library's call chain. We conclude that although the use of the keyword unsafe is limited, the propagation of unsafeness offers a challenge to the claim of Rust as a memory-safe language.
I don’t think it’s accurate to say that any use of unsafe necessarily propagates to all code in the call chain. Often safe abstractions are built around the unsafe blocks to effectively limit the scope of unsafeness. It’s not clear from the abstract whether this analysis accounts for such scenarios.
Maybe it would be a fun blog project. Take popular crates.io crates which have safety wrappers (unsafe code, wrapped in a safe function, we aren't interested in crates that just expose an unsafe function, or are purely safe) and:
Each week pick a safety wrapper at random, and take it to pieces. Is the safety rationale convincing? Missing altogether? Bogus for some reason?
This seems like it's potentially an interesting exercise, it's educational, and if you find any bugs this way you improve the ecosystem for everybody. Mmm. Am I missing a reason this would be a terrible idea?
> Am I missing a reason this would be a terrible idea?
Nope, people in the Rust community love seeing these sorts of things. Here's one from just last week where someone went through and fuzzed all the popular HTTP client libraries, which went on to become the third-most upvoted post on /r/rust of all time: https://www.reddit.com/r/rust/comments/oir1g1/ive_tested_rus...
Interesting paper. Here's one of their key conclusions:
> Our analysis shows that while publicly available Rust libraries rarely use the unsafe keyword (even very popular libraries), most of them are still not Safe Rust, because of unsafe use in dependencies.
I think this might be a misapprehension of safety in Rust by the authors: safety doesn't mean that absence of the `unsafe` keyword in your dependency tree, it means safety by construction through safe wrappers of fundamentally unsafe code.
The Rust standard library is the perfect example of this: it exposes safe interfaces that are built on top of fundamentally unsafe OS primitives and system calls. That doesn't make any code that uses them "unsafe"; it parametrizes the notion of safety on safe abstractions. That's all Rust has ever promised, and it's still significantly better than the status quo.
Other than that, the paper's other observations are excellent: we should be more aware of the presence of `unsafe` in our dependency trees, and that information should be more readily surfaced by standard tooling. Tools like `cargo geiger`[1] and `siderophile`[2] (FD: my company's tool) bring us a little closer to that.
Excellent -- so your large C codebase is free of security bugs, something no software vendor has managed to accomplish. Ritchie and Thompson themselves didn't manage that feat.
Tell us how you do it, please, so the world may learn.
Oh, that’s actually simple. Enforcing the “malloc once” rule gets you like halfway there, and using custom hardened code for common stuff like string manipulation finishes the job. Things like that are why some C codebases are large in the first place - you have to reinvent your own safe space from scratch.
The funny thing is that it’s actually harder to protect from certain classes of bugs in Rust. For example, you cannot uncouple from the global allocator as easy as you can in C/C++, and if you do, you do it with “unsafe” code. That doesn’t make Rust a bad language, it’s just that you can see some of the inherent flaws only if you had written safe C code previously and then you find out that some basic techniques don’t translate.
The idea of the `unsafe` keyword in Rust isn't that a single use of it anywhere in the dependency chain must make the entire code base suspect. Instead, the point is that encapsulation can be used to create primitives that rely on `unsafe` internally but whose interface doesn't provide any way to violate memory safety; downstream consumers of the interface shouldn't need to worry themselves about upholding memory safety.
Here's a contrived example, imagine a library with the following public function:
pub fn index_or_zero_i_guess_lol_idk(array: Vec<u8>, index: usize) -> u8 {
if index < array.len() {
// SAFETY: array index must be in bounds
unsafe { *array.get_unchecked(index) }
} else {
0 // yolo
}
}
Does a downstream consumer of this library need to do anything to ensure memory safety of their own code? No, because the unsafe `get_unchecked` function has exactly one caller-upheld safety invariant, that the array index is in bounds (documented at https://doc.rust-lang.org/std/primitive.slice.html#safety ), and because this code makes sure to uphold the invariant, the buck stops here. A downstream consumer need not use `unsafe` at all, and thus not give up the guard rails. And even in a single codebase, `unsafe` only "infects" the module that it's in, not the whole codebase, since a module is the unit of encapsulation in Rust. If you make your modules containing `unsafe` as small as possible, then you reduce amount of code that needs audited for safety.
And that's the whole point of `unsafe` in Rust: to reduce the scope of code that can potentially be memory-unsafe. It firmly places responsibility on users of `unsafe` to ensure that they uphold the necessary safety invariants; there's never any question as to which code is responsible for memory unsafety, as it can always be traced back to a single use of the `unsafe` keyword somewhere that has failed to uphold a safety invariant. It doesn't outright eliminate bugs--humans are fallible--but it makes it obvious when they might happen, it makes it easier to have confidence that they won't happen, and when they do happen it's easier to figure out why.
A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety. It is a nice-to-have to know that only the unsafe blocks of code are able to violate memory safety, but once memory is corrupted, errors can manifest anywhere.
Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. In my view, Rust should have a safe keyword to mark functions that are transitively not unsafe, much like the pure keyword in some languages marks transitive functional purity.
> A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety.
That's a silly definition of memory safety. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies. No interior mutability, no allocation, no containers.
The point is these uses of unsafe are checked by the author to ensure they can't introduce unsound behavior through the exposed safe API.
> Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks.
Sort of agree with you here. Unsafe code is rare in C#. But C# has race conditions so some kinds of bugs are more common. But C# isn't as suited for systems programming type of problems or really high performance code.
It is an honest definition of memory safety. The fact that you can not write a non trivial Rust program without unsafe dependencies is not an argument against that definition.
Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe. The virtual machine implementation itself may be unsafe of course, but that is a pretty solid boundary. In Rust, that boundary is easily crossed by anyone, for any reason, to the point where it sips into 50% of dependencies, as this paper shows.
I am not saying that C# evades this issue in principle, obviously it has the same functionality, it is just not used as liberally. Concurrency is orthogonal.
> Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe.
With the obvious exception of JS in the browser, I don’t believe this is true: Python, Ruby, Java, C# all have widely used unsafe native escape hatches that are regularly found in dependency trees. All you have to do is misuse these interfaces, which is significantly easier than in Rust.
Read carefully: The premise is that you can't write a non-trivial program in Rust without using unsafe.
It's true that most language implementations have an escape hatch in the form of a foreign function interface and/or they have native extension modules, nevertheless non-trivial programs can be written without using them. Conversely, using such facilities is often a reason why a program may not be portable between multiple language implementations.
> No, that's not my premise. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies, often the standard library.
Same difference, because the use of unsafe is transitive. There is no distinction between "unsafe-but-part-of-the-standard-library" and "unsafe".
> But this is true of every other programming language I'm aware of. So if you want to define memory safety to be something never achieved in practice, fine, but that's not a useful definition in my books.
I don't. I already made that distinction further up in the thread, you can read that and reply there if you have any objections.
So nothing is memory safe by your definition then.
Languages like C# are more memory safe than Rust because the uses of unsafe code are fewer and more concentrated in the runtime where bugs are more easily caught. I think that's a fair comparison.
> So nothing is memory safe by your definition then.
No. I will now apply maximum charity and explain the distinction once again: Let's take Java. It runs on a VM. The VM has a memory model. There is no way to write memory-unsafe code while staying within that model. Not only can you write non-trivial programs against that model, the vast majority of Java code is written against it and is thus memory safe. That's a fair definition of memory safety.
Of course, in practice, you will likely use an implementation of that language that uses unsafe memory access. You will likely use an implementation that has escape hatches to call into native code - unsafely. That's a reasonable boundary to have while still calling the language memory-safe. Unsafe in C# was a late addition that can usually be disabled without much peril.
Why would memory get corrupted in this example? To put it differently: if the code didn't have the `if index < array.len()` check, clearly things would be different. How would you describe this distinction, other than "memory safety"? (Asking because it's not clear whether you're using an unusual definition of memory safety, or just didn't notice the bounds check in the example.)
> but once memory is corrupted, errors can manifest anywhere
Yes, but those errors can only be caused by code flagged with the unsafe keyword. Which makes debugging, auditing etc. a lot more simple than in, say, C, where it could happen anywhere.
This isn't necessarily a huge help, because those errors could be triggered or manifest at any point where unsafe code is used transitively. In C, not every single line is unsafe. All that the unsafe keyword ultimately does is to force you to aggregate such lines into blocks with the word "unsafe" around them.
You can speculate that this is a significant improvement. I would argue that a disciplined use of a simple language like C would, in practice, be safer than using a complex language like Rust which has "safety" written all over it - but then liberally sprinkles unsafe code into all the gaps.
We can both agree that C++ is the worst of both worlds.
> This isn't necessarily a huge help, because those errors could be triggered or manifest at any point where unsafe code is used transitively.
Right, and that isn’t something Rust or the unsafe keyword set out to solve. Unsafe has absolutely no runtime impact and was never designed to.
Your argument that a disciplined use of C would render this useless is correct, but history has shown us a great many examples of undisciplined use of C, even in projects that start out disciplined. Rust codifies and enforces that discipline, I don’t see anything wrong with that.
You want unsafe to be something it isn’t, but I’m not convinced what you want fits with Rust’s vision. There are many, many safer languages to choose from.
I completely disagree with that. There's no discipline required to slap "unsafe" around a block of code. The discipline must be applied to the block of code itself, there is no way to enforce that. If 30% of dependencies have unsafe code in them, how would I know that the unsafe parts aren't just as sloppily written as the average C code?
Rust has yet to prove itself as a language that actually delivers on its promise of being safer than C, in practice. To that end, we must compare C written with security in mind to Rust written with security in mind. Given that Rust is a rather complex language that encourages abstraction, I'm quite hesitant to drink the cool-aid right away.
> Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks.
But this suffers from selection bias: people don't tend to write the same sort of high-performance libraries in C# as they do in Rust precisely because of how much harder it is to squeeze out every last drop of performance from a managed language (even taking into account C#'s own `unsafe` keyword). If Rust didn't offer `unsafe` and only privileged its own first-party constructs then it could certainly produce a more broadly safe ecosystem, but it would not produce a more useful one. And therein lies the whole point: the goal of Rust is not to produce a safe ecosystem for its own sake, but to provide a more reasonably-safe alternative to C and C++ for the sorts of things that C and C++ get used for.
That's why I would argue for a "safe" keyword. In this sense, the compiler can actually help me write safe code and encourage library authors to do the same.
Either safety really matters and performance can be given up for it, or vice versa. I do not believe that Rust is a "reasonably-safe" alternative to C, because C is already reasonably safe if written with safety in mind, but with the added bonus of being simpler. Rust is reasonably unsafe when written with performance in mind, whereas C++ is just unreasonably unsafe.
This does assume that all `unsafe` usages are wrapped in correctly `safe` guards though, doesn't it?
You can wrap `unsafe` functionality without checking and thus have memory issues, or if the unsafe wrapper hasn't correctly validated all possible entries that can cause memory corruption. Then any consumer of this dependency has a possible security issue with the right inputs to the consumer, which can trigger passing an exploit down to the unsafe portion.
Yes. If your safe code blows up but everything else seems fine, the fault is one of (starting with least likely):
* Your hardware is broken. Did you spill the entire glass of wine on that laptop?
* The Rust compiler has a bug. Or, equivalently, LLVM has a bug and rustc depended on LLVM to be correct. Or, your OS kernel has a bug. These are all unlikely, but they do happen.
* A safe wrapper in a popular library you depend on is not fulfilling its contract and you noticed before anybody else.
* A safe wrapper in your code, or in some lesser known crate you depend on, is not fulfilling its contract. If you just fired Jim for writing LGTM without reading his code reviews, start with any code Jim reviewed that has "unsafe" in it.
Notice that the blame definitely is not on the safe code which relied on a guarantee that wasn't fufilled. This is a human construct layered on top of what Rust's compiler is actually delivering, but it's still important to think this way.
If in hindsight the function Jim didn't review correctly should be marked unsafe because you need to check your parameter wasn't zero when calling it, do exactly that, never write...
// Marked safe, but actually not safe to call with zero
... in the comments for the function, you're making your own life harder.
Your contrived example makes sense, I guess I'm just confused about how one would have faith that code using `unsafe` is actually safe? As in, to me (an avid non-Rust-user), seeing unsafe would mean either A.) I have to audit this codebase to make sure there wasn't any legitimate unsafe behavior, or B.) I just have to totally skip this dependency.
If the argument is "You can use unsafe but do things safely", isn't that the same argument as C? "Just have faith that they did it right"?
The difference is in the quantity of code you have to audit. In a given Rust project perhaps 0.1% of the lines of code may be unsafe, which while technically non-zero is widly less effort to audit than the entire codebase in the case of something like C
In addition to what the sibling comment says, you indeed often can find dependencies with no `unsafe` keyword usage whatsoever, and libraries that do so will tout this as a feature. Fortunately for your purposes the broader Rust population (including library authors) is mostly composed of people who are more-or-less skeptical of wanton `unsafe` usage in the first place (as should be self-evident, as otherwise they'd probably just be using C or C++!).
Yeah I suppose I would've been inclined to just throw out any deps that are built atop unsafe. It seems to kind of totally ruin the intended purpose of Rust as a language (to me, who is uninvolved).
There has to be some unsafe, or else you can't have the parts of the standard library that interact with the OS. I'd at a minimum also include projects with experienced authors and similar needs such as tokio.
For example, you wish to implement a Vector in Rust.
You generally don't want to allocate that Vector over and over, so you request more memory than you need.
That "extra" memory is uninitialized which is a big no-no in Rust. You might be able to initialize that memory, but that will have performance implications that C/C++ programmers would just laugh at.
However, your "invariant" is that you never access that memory until you have a real element that you put there. That's perfectly fine.
Nevertheless, accessing that uninitialized area will always be "unsafe" from the Rust compiler point of view.
Unsafe doesn't remove all the guard rails, but it allows you to do a handful (five) of tremendously unsafe things, for which you then take all responsibility as the compiler cannot analyse them for safety.
* You can dereference raw pointers. Are they pointing at anything at all? At something that type pointer might reasonably point to? Not the compiler's job to check, if you screwed up your program now has Undefined Behaviour like C or C++
* You can access C-style unions. This one says it might be a boolean, a pointer to something, or an integer. You decided to guess it's... a boolean. If you're wrong you get Undefined Behaviour.
* You can mutate (change) static variables. Global cheese flavour is Cheddar? Let's change it to Gorgonzola, no wait Parmesan. Was anybody else using that for anything? No idea, too bad, you might introduce Undefined Behaviour if you didn't think this through.
* You can call functions Rust labelled "unsafe". These functions come with instructions about rules you must follow to use them safely. If you violate any of those instructions, all bets are off, but if you obey the instructions whoever wrote the function (which may be the Rust standard library) promises that was safe.
* You can implement special Traits labelled "unsafe" to implement. These Traits, unlike most Rust traits, have to promise they're correct. If you claim to implement Searcher, a Trait for text processing, and you get it wrong, other people's code may blow up. Whereas I have types like Funhouse<> which claim to implement the trait Eq (promising they understand how equality works) and then as a goof they utterly refuse to do so correctly, that's a safe trait, nothing blows up. My types don't work in a sensible way (Funhouses are simultaneously equal to and not equal to every other Funhouse including themselves, which is stupid), but your program doesn't have Undefined Behaviour if you use these types, they're just annoying.
But, lots of things that would cause Undefined Behaviour in some popular languages are impossible even in unsafe Rust.
Suppose I have a function that takes an array of 1024 bytes. I try to do { let z = array[1040]; } -- That won't compile, this array isn't big enough. If I write unsafe { let z = array[1040]; } it still doesn't compile, didn't I get it, this array isn't big enough.
OK, let's have the function take a parameter k, and at runtime we can set k to 1040 and try that way. If I write { let z = array[k]; } when k is 1040 the program panics. The array isn't big enough. If I write unsafe { let z = array[k]; } it still panics and the compiler even warns me that unsafe isn't doing anything for me here and I should remove it.
Similarly although Rust provides checked arithmetic (ie you can explicitly say "Tell me the answer to x + 100, or, if that would overflow, tell me it didn't work") its default integer arithmetic will panic in debug builds if you overflow and, if you missed that in debug, but it happens in production, you get overflow, which may very well not be what you wanted, but you apparently didn't even know it could happen, so, good luck with that. Either way, no Undefined Behaviour. 255_u8 + 255_u8 is a compile error, a runtime panic or at worst it's 254, but it definitely is not zero, forty-two, or unrelated program misbehaviour.
The difference is that in Rust you can build safe (foolproof) abstractions on top unsafe code. OTOH, in C you can't write a library that returns a pointer and prevents users from misusing it (e.g. ensure they can't use it after your lib frees the data, or improperly share it between threads). In C, the risk of unsafety and misuse of APIs remains a problem for every downstream user.
For example, Rust's String type internally allocates, reallocates, and frees memory. It can have partially uninitialized buffer, and lots of other "unsafe" things.
But there's no way to cause UB when using safe Rust's String. You can't cause use-after-free. You can't cause a data race. You can't overflow the buffer. You can't even break its UTF-8 encoding. That's because String's public interface only exposes safe methods. The unsafe String internals were written and checked for correctness, and from then on they're safe to use for everyone else.
Aren't they saying though, that there could be a bug in the string unsafe code that causes UBs. You can't guarantee that (although this example is unlikely).
There could, and these kind of bugs have happened a few times already, but fortunately they are few and far between, and as the sibling comment mentioned there's some work in progress to formal prove the std lib, and because the std lib is much smaller than the amount of code that uses the stdlib, it's much less work to formally verify the stdlib than it would be to verify for instance all C++ programms using the C++ stdlib.
There's a practical difference in how much of the program is high risk. If all of the risk is concentrated in a few functions, it's easier to focus on reviewing/testing/fuzzing these bits.
Rust changes the bug-spotting game from "Where's Waldo?" to "Is this a Waldo?"
Some code is beyond the ability of a compiler to verify because the necessary information is missing. Rust acknowledges this reality and accommodates such code through unsafe. The presumption inherent in this is that unsafe is used only when deemed necessary. Yes, deemed; a decision made by some flawed soul, as opposed to a compiler. This is a pragmatic policy and it cannot be reconciled by the unreasonable.
> You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword.
The key here: Rust SURFACE AS TYPES this information.
Is similar to this piece of code (imagine is python):
sum(a,b)
what it actually do? WHO KNOWS. But this:
sum(a:int,b:int):int
is informative. And the "type system" can verify its use.
So "unsafe" is part of the type system, type checks that Rust use to COMMUNICATE and CHECK the use of code. So, if I, as user see this:
unsafe fn raw_memory(...)
I know the type system of Rust was bypassed and must TRUST the HUMAN know how correctly code what is inside.
---
The major advantage here, is that the blast area of this stuff is limited and for the rest of regular code, the type system verify the code and remove the need to manually inspect (and make tests!) for a lot of trivial (and not so much1) stuff.
Is a total game-changer, in special for a system language.
What worries me is that when it comes to safety, surely the chain is as strong as the weakest link? You declare one thing unsafe, and then suddenly all bets are off. And maybe you don't even know that what you're doing is unsafe.
But here's an example I pulled from github, which blinks a Raspberry Pi Pico:
// Set GPIO25 to be an output (output enable is set)
p.SIO.gpio_oe_set.write(|w| unsafe {
w.bits(1 << 25);
w
});
Right, but what if you're fiddling with registers that might require atomic operations? Lines such as this seem to be saying "ignore all that, write to the register anyway."
Fearless concurrency, you say: are you sure? How do you know?
Suppose you have an ISR (interrupt service routine) that wants to toggle a GPIO pin. From whence does it borrow the privilege to set the pin?
Surely, at the end of the day, microcontrollers can be characterised as large state machines with global state. That's what they ARE. It's their inherent nature, which you cannot abstract away without it leaking somewhere (at least to do anything useful, anyway).
At least with C/C++ you KNOW you've got a problem, and have to reason about the problem carefully. With Rust, you THINK you've solved the problem, but have you?
" Instead, our interrupt handlers might be called at any time and must know how to access whatever shared memory we are using. At the lowest level, this means we must have statically allocated mutable memory, which both the interrupt handler and the main code can refer to."
"In Rust, such static mut variables are always unsafe to read or write, because without taking special care, you might trigger a race condition, where your access to the variable is interrupted halfway through by an interrupt which also accesses that variable."
RIGHT, but that means that Rust gives me no better assurances than C/C++. Here's their code:
static mut COUNTER: u32 = 0;
#[entry]
fn main() -> ! {
set_timer_1hz();
let mut last_state = false;
loop {
let state = read_signal_level();
if state && !last_state {
// DANGER - Not actually safe! Could cause data races.
unsafe { COUNTER += 1 };
}
last_state = state;
}
}
#[interrupt]
fn timer() {
unsafe { COUNTER = 0; }
}
Then is says "Can you spot the race condition? The increment on COUNTER is not guaranteed to be atomic"
It then solves the problem by showing how to introduce a critical section.
So at the end of the day it's just C with different syntax, and a lot more hoopla. You've made everything mutable and unsafe. Exactly the kind of thing you were trying to avoid in the first place.
> At least with C/C++ you KNOW you've got a problem, and have to reason about the problem carefully. With Rust, you THINK you've solved the problem, but have you?
Having written C and C++ for about a decade and Rust for about 2 years now, I don't think this is accurate: people don't reason any more carefully about C and C++ than other languages, and neither language (even in their safer subsets) encourages methodical safety thinking.
This is in direct contrast to Rust's absolute requirement that you annotate anything that isn't already safe by construction with an `unsafe` keyword. You might be wrong about the safety of whatever you're constructing, but that alone is a significantly more methodical and careful approach to encapsulating the "weak" parts of programs.
Last time you used an higher level interpreted language (like JS), did you have to worry whether or not the machine instructions it was generating were the right ones? Generating instructions at runtime sounds pretty unsafe, no?
You didn't, right? You can just mostly assume the language works and get on with your life. That's how unsafe feels to me when I'm programming in Rust, I never have to worry about it and pretty much never need to use it.
Sometimes you'll run into a problem where you need to use unsafe, but that only happens very very rarely, for example when creating an interface to a C library, but most people will never need that. When that happens, sure, you'll have to be extra careful and make sure everything works. But after you're done with that small piece, you're back into safe Rust.
So, how does it work in practice, if you really need to use unsafe? Suppose you have a C enum with values MyEnum {A, B, C} (but it can be any int in C!), and you want to use it in Rust. You can make an unsafe wrapper that tries to parse the enum, and it can either return None - if it was an invalid integer, or return Some(A), Some(B), Some(C).
But once you get the wrapper right (which is not hard), Rust will guarantee that values of type MyEnum have values A, B or C and nothing else, so you never need to worry about the exceptional case again.
You worry about it once at the boundary and that's it.
Issues with ISRs are different. They'll sorta appear when rust is used inside the Linux kernel, too.
For example, can rust make sure some kind of function is not called at any level when called from certain kind of context (say atomic context).
Can rust help detect/prevent you from accessing some multi-register SFRs or memory locations from main context without making you disable interrupts first in the main context?
And this constraint is only relevant if the SFR or said memory location is ever accessed from some ISR, and the access is not atomic, otherwise disabling interrupts is not necessary.
Do you have to wrap all these accesses and keep track of all this manually just like in C or does rust have some concept of "colored" functions, so that you're forced to wrap accesses to certain memory locations only if these accesses are shared between differently colored functions.
I'm not a rust person either. Just wondering how rust would help me with the most complicated aspects of concurrency in low level programming. Allocating/deallocating memory and keeping track of it is easy compared to this.
What you want sounds similar to Rust's Mutex design pattern. In most languages, there's no link between a mutex and the resources it's protecting, but in the Rust standard library, a Mutex<T> contains the resources (the T), so that the only way to get access to these resources is to lock the mutex, and the borrow checker will not allow accessing them after the mutex is unlocked again.
In your case, you could implement something like an InterruptDisabler<T>, and use it to wrap an object which has the methods to access these registers. When you wanted to access these registers, you would do something like "my_interrupt_disabler.disable_interrupts()", which would disable interrupts and return an InterruptDisablerGuard<T>, which then would you to access the T. Once that InterruptDisablerGuard<T> gets out of scope, the compiler automatically calls the drop() method from the Drop trait on it, and that method could enable interrupts again. You cannot access the inner object without going through the guard, and you cannot get the guard without disabling interrupts.
And that's only one possible implementation. Another one would be to have something like an "interrupts disabled" token which is returned by a "disable interrupts" function, which is consumed (that is, destroyed) by an "enable interrupts" function, and which is !Send and !Sync (so it cannot be passed to another thread). The methods which access these registers would require you to give them a reference to the token, so they could only be called if you somehow obtained the token, which could only be done by disabling interrupts in the current thread. And for when you're within the interrupt handler itself, the low-level code which calls it could manufacture one of these tokens and pass a reference to it to the interrupt handler, so it would be able to access the registers without calling the "disable interrupts" function.
Of course, both of these ideas still require you to decide which registers are safe to call with interrupts enabled, and which registers are not, when creating the objects which represent the register blocks. Rust can help the developer, but it's not a panacea.
It's not about which registers are safe, but about whether the access patterns compiler generates for register access are safe. Say `andl some_sfr,#0xfe` is safe for all SFRs, but `mov r1,some_sfr ; andl r1,#0xfe ; mov some_sfr,r1` is not (not atomic, interrupt can happen in the middle).
Say `some_sfr = 1` is safe but `some_sfr = (some_sfr & 0xf0) | 0x3` is not.
Token thing seems interesting, but generally, no code calls the interrupt handler, HW does that by itself (outside of the control of the language), so there would be no way to pass the token to the handler, it would have to be manufactured in the handler out of thin air (without disabling interrupts, because those are disabled automatically by HW). And I'd have to wrap all the non-atomic language constructs in some functions that would require the token, to have some enforced safety.
I guess the fearless concurrency thing is only for cases where the language controls the creation of threads.
I think most rust programmers are aware enough to know that it's not actually 100% memory safe. But it's memory safe-r. I don't actually know anyone who's done even a little bit of rust actually believe it's 100% safe
The stdlib is full of unsafe, I'd much rather they handle the unsafe parts which have been audited and battle tested and I use their abstractions.
It's almost impossible to avoid unsafe somewhere in the chain. But that's not really unexpected. What's nice is that once you have a safe wrapper, it's impossible to misuse and cause memory errors
In that article, they have three goals:
verify that the pin is correctly configured,
use atomic reads and writes,
make sure only one thread (including interrupts) can write to a pin at a time.
Spoiler alert, the way they avoid interrupts screwing everything up is with `cortex_m::interrupt::free(|_| {/*read/write pin*/})`, which executes the passed closure in an interrupt-free context. The author's solution isn't atomic, but the borrow checker and the fact that they target single threaded MCU means no one else can be writing to the port at the same time.
Looking at your "blink an LED" example this appears to end up in some code from svd2rust and in that code what's going on here is the write takes a closure which modifies the bits to be written (starting from the hardware default).
The unsafe thing being done here is calling w.bits() which is a method on w (the bits we're going to write to this register in the hardware) that allows the caller to set it to any value of their choosing, even values that nobody said are OK for this hardware.
There are two reasons why you might do this. Maybe your hardware was too lazy to provide safe ways to set every value that could be OK. This seems entirely plausible for the Raspberry Pi. Or, maybe you are crazy and you're writing values the hardware maker said never to use. Either way, Rust doesn't know anything about how the Raspberry Pi GPIO unit works and what bit values are OK.
This unsafe code does not write bits to registers. This isn't bypassing some imaginary safeguard that prevents touching stuff simultaneous to an ISR. It's critical to understand that.
The surrounding safe code performs the register write, as an atomic operation (AIUI a single hardware instruction on the Raspberry Pi) using the result of the closure.
> I'm not a Rust programmer, so be gentle with me.
> So at the end of the day it's just C with different syntax, and a lot more hoopla.
So what seems to have happened here is that you're completely ignorant of the topic and you've mistaken that for expertise. That's a bad habit to have developed, and it's important to break it.
That Raspberry Pi Pico code snippet is slightly unusual in that the `unsafe` there is because the library isn't sure that 1<<25 is a safe value to write to this register - for example, if the register was a DMA memory address pointer, it's not safe to allow directly writing arbitrary pointers to it since it would allow memory access outside of Rust's memory model. It's not to do with the potential atomicity requirement.
The exclusivity/atomicity in that case is ensured because the method is called on "p.SIO", an object that can't be accessed from both ISRs and the main code at the same time in safe Rust (because it doesn't "implement Sync"). If both an interrupt and the main thread need to access that peripheral, you need to provide some way of sharing it - either using `unsafe`, or in safe code by using a synchronisation primitive such as a critical-section based Mutex that provides that guarantee.
The book chapter you've linked to starts out by demonstrating what is essentially how you'd write this in C and thus requires unsafe, but it builds towards a safe solution - when using either the Atomic* variables (in the Atomic Access section) or mutexes (in the Mutex section), `unsafe` isn't required any more; the user's code is only safe Rust which provides synchronised access to the shared state between the main thread and the ISR.
In other words, the benefit over C is that it _is_ now possible to use only safe Rust to access memory and peripherals from both interrupts and the main thread, and that safe Rust is itself ensuring you can't cause race conditions. The unsafe option is there as a building block for those safe abstractions.
> So at the end of the day it's just C with different syntax, and a lot more hoopla. You've made everything mutable and unsafe.
Perhaps that chapter isn't getting the right message across then. The goal is to completely avoid applicaton code having to make things mutable and unsafe by providing the right abstractions that allow safe Rust to get the same work done while ensuring there are no races.
I think the difference is the all-or-nothing mentality. Nothing is 100% trustworthy.
Even if you write "unsafe"-free Rust that only relies on the Rust standard library, you need to trust:
- The Rust compiler and everything it uses to be correct (not just memory safe), e.g. LLVM.
- The Rust standard library, and everything it calls in to, e.g. system libraries, the kernel, the CPU.
That's a ton of code. By calling into an additional library that uses the "unsafe" keyword, you're just adding that library to your set of things you need to trust.
Naturally, you probably have higher trust in the Rust compiler and OS kernel then you do in some Rust library on GitHub, but nothing is 100%.
And the GitHub Rust library probably only uses a few small blocks of "unsafe" code. That makes it easier to audit and increase your confidence than if it were written in C.
Plus, the Rust safety guarantees don't cover everything. You're still trusting that the library doesn't have a bug that accidentally writes to the wrong file, or sends the wrong value over the network.
C++ defines a memory model, allowing programmers to "rely" on certain guarantees from the compiler and runtime library. But you're still trusting the C++ compiler, the runtime library, the OS kernel, and the CPU to work the way they're supposed to. That doesn't mean the C++ memory model is useless.
In Rust, the only way to trigger undefined behavior is the presence of unsafe code. So everything outside of manipulating COUNTER in your code block cannot trigger undefined behavior. In C, there aren't any of those guarantees, so any line in the code could potentially trigger undefined behavior. Since we know that only the block manipulating COUNTER can cause undefined behavior to occur, ensuring that the block manipulating COUNTER can not cause undefined behavior to occur is the only place where heavy scrutiny is needed...as opposed to C, where every function call and line of code would need to be scrutinized.
Sure you could throw unsafe everywhere and not check anything, but that misses a major reason to use rust--safe abstractions over unsafe behavior. Once a safe abstraction exists for unsafe behavior, all the heavy lifting of guaranteeing code will not result in undefined behavior is offloaded to the compiler, instead of the programmer having to keep track of it all.
> What worries me is that when it comes to safety, surely the chain is as strong as the weakest link? You declare one thing unsafe, and then suddenly, all bets are off. And maybe you don't even know that what you're doing is unsafe.
Yes, you are correct. If some code or module contains a memory safety or data race bug, then the code which uses it could realize that bug in the form of a crash or race. This paper is not analyzing code that contains a memory safety or data race bug, but instead, code that the Rust compiler cannot prove doesn't have such a bug. That is a big difference.
Let's take your example. The questions you bring up are good ones. Fiddling with registers may indeed require atomic in the presence of multiple threads using that register. Or, if you can guarantee only one thread touches those registers, it may not. In both use cases, I'd like the compiler to help me verify that those conditions are met. A Rust approach to this problem (verified empirically by the paper) would be to create a module, function, or trait that enforces the invariants we need. Such an abstraction might contain a bug that causes the codebase to be unsafe. But when debugging that, we shouldn't have to look over the whole codebase for register writes but rather in just one place.
> Suppose you have an ISR (interrupt service routine) that wants to toggle a GPIO pin. From whence does it borrow the privilege to set the pin?
Assuming we are protecting the pin with atomics, the ISR could call into our library to acquire the lock. The ISR doesn't necessarily need to borrow the privilege from another thread. Our module could provide a safe interface for unsafe access to a global lock.
I would also ask how in C/C++ an ISR performing such an operation would work without data races. I assume it would need to perform a very similar set of operations to the compiled Rust binary. The only difference is that it is harder to find that gpio pin manipulation in the C code. That isn't an accident Rust's explicit goal is for its abstractions to be zero-cost.
> Surely, at the end of the day, microcontrollers can be characterized as large state machines with global state. That's what they ARE. It's their inherent nature, which you cannot abstract away without it leaking somewhere (at least to do anything useful, anyway).
First, this can be said about any computer, no matter how small or large. But I think that you are getting at something important here. Rust is a much more natural fit in user space on an operating system that abstracts away the hardware details than in a micro-controller. And certainly, on a microcontroller, you have to interact often with IO of various kinds, which are global state that Rust considers unsafe. But I think, to the extent, it is possible having the unsafe/safe stratification of code enables you to separate the code which doesn't interact with the outside world and the code which does. I think that is abstractly similar to the user/kernel space divide and Haskell's IO/computation divide. I believe that split helps with code organization and concentrates the attention when such a bug occurs.
> At least with C/C++ you KNOW you've got a problem, and have to reason about the problem carefully.
First, you should know that Rust's safety guarantees extend only to memory safety and data races. Second, I don't think that it is true, in C/C++, that if you have a memory safety or data race bug, you necessarily know that you have one. If you do have one, it could be in any part of the code. Reasoning about a large codebase is much more difficult than a single module with all potentially unsafe code.
> With Rust, you THINK you've solved the problem, but have you?
You may well not have. Here is an example of Rust's core libraries containing a CVE [1]. And here is another popular crate that had a CVE relating to unsafe code [2].
The paper noted the unsafe code was slightly mo...
> We conclude that although the use of the keyword unsafe is limited, the propagation of unsafeness offers a challenge to the claim of Rust as a memory-safe language.
By this definition, is there any memory safe language?
They all boil down to unsafe dependencies. Even if you ignore interpreter abstractions, any language with FFI is unsafe but without any way to audit it.
81 comments
[ 5.2 ms ] story [ 178 ms ] threadTechnically no (only for half of libraries) but effectively yes, if you're comparing to C and C++. The fact that there might be an `unsafe` somewhere in your dependency tree just means that instead of being there being a 0% chance of a memory bug, there's 0.0001%. Whereas a C/C++ codebase of moderate size is virtually guaranteed to have many (orders of magnitude might be off but point stands).
[1]: That's the frequency of this paranoia creeping in that I observed, not of detected memory corruption, but that too I think would be counted in an integer number of percents IME.
https://github.com/rust-fuzz/trophy-case
The point of unsafe is to point you toward the areas you need to audit. Unsafe code can't be proven memory safe by the compiler [...but the rest can be]. It's far easier to audit 10 or 100 LoC of unsafe rust than 10 or 100 thousand of C++.
And even in C++ there are subsets of the language that are supposedly memory safe. But even when writing in those subsets I'm not convinced that the abstractions I'm building on are as well validated as rust.
Rust devs (like myself) aren't illusioned into believing that my code and _everything below it_ is 100% safe.
However i can positively assure you my code is 100% safe. Well, assuming no bug in Rust-lang/compiler. My _programs_ are not guaranteed to be 100% safe unless no unsafe is used, but Rust lets you know for certain what is safe, and what is unsafe.
The point isn't absolute safety in your program. The point is absolute safety where you think there is absolute safety. To strictly know what is unsafe. And to reduce the total lines that are unsafe.
edit: I'm confused by the downvotes, what is incorrect or controversial about this?
The chance is never 0%, and nobody should be thinking that way. Memory safety bugs routinely arise in JITs (e.g. JavaScript engine CVEs). In fact, memory safety issues routinely arise in CPUs (e.g. rowhammer).
In reality, terms like "memory safety" are rarely ever exact, but they're still useful descriptions. Practically, people should be thinking about frequencies, not in black and white.
I've created a memory leak in java. Not a "I forgot to free memory in a map" memory leak but a honest to goodness "This JVM is allocating heap space and losing track of that allocation". Something that required me to install a special malloc in the VM to track down the leak. This was using standard libraries (You can do it too! Go play with the Deflater and Inflater API and you'll find it makes calls off to Zlib. If not properly handled you end up leaking heap allocations).
That was the first and only time I've ran into that issue, but I ran into that issue.
The guardrails rust gives you are essentially the same that the JVM gives you. That is, use only safe code and (barring a compiler bug) you are safe. However, bad things COULD (not will) happen if you use unsafe blocks. Just like bad things can happen in JNI blocks or places where Java makes use of C libraries for functionality.
And that is what makes rust fantastic. It's got memory safety up there with JVM safety without a garbage collector. It even has nifty keywords (unsafe) to keep developers on their toes when they need it. Just like Java's JNI or Pythons C extensions.
In fact, I'd argue that rust unsafe ends up being more safe than either JNI or C extensions primarily because it is something that can be audited very quickly across all rust code bases. Have a memory issue? It's in the unsafe block.
If for new projects what type of project that is not some kernel, browser or some low level library is Rust the best tool for the job? For most type of projects I am thinking Python,Java, C#, C++ still is better .
Right now they're one of our most egregious sources of security vulnerabilities (mostly due to a lot of hands-on bit-twiddling that results in a lot of possibilities for overflow). It's gotten bad enough that a number of system developers (browsers, OSes) have been shifting to a model where codecs are strictly sandboxed, and just get a single binary input blob, and a single output blob, and don't get any other kind of access to the system they're on - it's a scorched earth problem, but it really is that bad.
The thing about trying to write one with Rust is, if you actually 'went with the flow of the language' and didn't just immediately pop into `unsafe`, you'd be able to write a really safe codec.
-but-
The real payoff is that the performance would be surprisingly good - usually a suggestion like the one I just made would absolutely wreck performance, but one of the big things about Rust is that because it lets you describe what your intent is as a programmer in such higher-level terms (despite being a machine-level language), it gives the compiler a lot more information about the scope of your intent, which opens the floodgates to potential optimizations.
For a really easy example - Rust does all variables as "immutable by default". When you're programming in C, and something is compiling your code, there are tons of opportunities where you'd go "oh, well we already calculated this, why don't we just cache it?" But the compiler can't, because it doesn't actually know what you, the human, know - that that thing won't change. Because that knowledge is available to the compiler through Rust, the compiler actually DOES know, so tons of things can get inlined, cached, and otherwise optimized for you which you'd otherwise have to do by hand.
Usually when people do these sorts of things by hand, they'll ace a couple key optimizations (usually things that show up in profiling hotspots) and completely overlook hundreds, even thousands of other potential optimizations.
There are tons of other "high level intent" things that give the compiler more info it can use to optimize, but the "immutability by default" one is a really easy one to explain.
In Rust, if I have two u8 variables and I add them together, Rust will try to figure out if it's sure at compile time that this is an overflow and if so reject it, but otherwise I get a binary which might have an overflow in it.
But in WUFFS when you add those variables together, WUFFS wants you to justify why that can't overflow, or, tell it that you know it does overflow and you want wrapping or whatever. As a result, WUFFS gets to be very confident the resulting (awful, machine generated) C is safe and correct.
As you observed, this sort of confidence also permits a lot of performance tricks. WUFFS has some very impressive (ie better than existing Rust code) benchmark scores for the (much simpler) codecs like JPEG or PNG but I don't think there's any reason it couldn't shoot for MPEG or beyond.
WUFFS is no rival to Rust. It doesn't have IO for example. Which is fine, your codec implementation shouldn't do IO. WUFFS deals in blobs of bytes. Bytes you got over the network or from a file go in, and bytes represented a decoded image, decompressed file, or whatever come out.
You can just start building new parts in rust. Like, if you're building a new audio subsystem, you can do "just that subsystem" in rust, and expose an interface for the rest of the legacy engine code to use.
You've got MSFT, Facebook, & Google all investing heavily into Rust as a replacement for C/C++. It's still early days. Rust's sweet spot for being the best tool is anything that's currently written in C/C++. If you're writing Python, Java, or C# code, then Rust is probably not the language for you (albeit Rust is getting deployed in all these places where perf is needed because it offers the performance of C/C++ with the safety of more managed languages).
At no point will it be "replace everything in Rust" (at least not in the immediate term). It'll be more like "write this component with a thin API in Rust" or "with codebases that have a C++ runtime, work out a way for the peripheries to be written in Rust".
I'll give you an example. We used the venerable addr2line (or even llvm-symbolizer) to symbolicate addresses for an executable. We used it by spinning up a process for each address to symbolicate (which is dumb, but whatever). By writing a wrapper over Gimli-rs/addr2line-rs, we were able to improve our symbolication time 1000x & reduce our memory footprint 3x. The memory footprint improvement is because Gimli is 0-copy. 100x of that speedup is because Gimli's 0-copy is also just faster at parsing the DWARF data. The next 10x is because we only parsed the DWARF once & then used that precomputed information for all addresses. The fact that that Rust tool was correctly engineered as a library + frontend meant this was relatively easy, cheap & painless to integrate into our existing C++ codebase (using the cxx crate).
Like all things, Rust is a tool. Knowing how to find the right tool for the job & deploying it well is 99% of the job. Personally, I've found the Rust libraries I've come across for systems programming to be higher quality than the equivalent C/C++ libraries because the field is green, the userbase is extremely passionate, enthusiastic & early adopteree (i.e. more technically adept), & Rust makes certain kinds of optimizations easier than they were before (so naturally people want to play around with them). Additionally, because these are all rewrites, 90% of the time the improvement is just applying more modern state of the art knowledge to existing APIs/algorithms (e.g. doing 0-copy for symbolication is a "no-brainer" but it's hard to do safely in C++).
So all serious shops are going through this self discovery process but none have decided yet they are weighing the pros and cons daily. Unable to reach that conclusion but moving closer to Rust everyday.
That sounds like someone is hopeful they bet on the right horse.
The question is ultimately a reminder to check your dependencies to make sure they're active and take reasonable steps for validating unsafe code.
> Our results indicate that software engineers use the keyword unsafe in less than 30% of Rust libraries, but more than half cannot be entirely statically checked by the Rust compiler because of Unsafe Rust hidden somewhere in a library's call chain. We conclude that although the use of the keyword unsafe is limited, the propagation of unsafeness offers a challenge to the claim of Rust as a memory-safe language.
I don’t think it’s accurate to say that any use of unsafe necessarily propagates to all code in the call chain. Often safe abstractions are built around the unsafe blocks to effectively limit the scope of unsafeness. It’s not clear from the abstract whether this analysis accounts for such scenarios.
Each week pick a safety wrapper at random, and take it to pieces. Is the safety rationale convincing? Missing altogether? Bogus for some reason?
This seems like it's potentially an interesting exercise, it's educational, and if you find any bugs this way you improve the ecosystem for everybody. Mmm. Am I missing a reason this would be a terrible idea?
Nope, people in the Rust community love seeing these sorts of things. Here's one from just last week where someone went through and fuzzed all the popular HTTP client libraries, which went on to become the third-most upvoted post on /r/rust of all time: https://www.reddit.com/r/rust/comments/oir1g1/ive_tested_rus...
> Our analysis shows that while publicly available Rust libraries rarely use the unsafe keyword (even very popular libraries), most of them are still not Safe Rust, because of unsafe use in dependencies.
I think this might be a misapprehension of safety in Rust by the authors: safety doesn't mean that absence of the `unsafe` keyword in your dependency tree, it means safety by construction through safe wrappers of fundamentally unsafe code.
The Rust standard library is the perfect example of this: it exposes safe interfaces that are built on top of fundamentally unsafe OS primitives and system calls. That doesn't make any code that uses them "unsafe"; it parametrizes the notion of safety on safe abstractions. That's all Rust has ever promised, and it's still significantly better than the status quo.
Other than that, the paper's other observations are excellent: we should be more aware of the presence of `unsafe` in our dependency trees, and that information should be more readily surfaced by standard tooling. Tools like `cargo geiger`[1] and `siderophile`[2] (FD: my company's tool) bring us a little closer to that.
[1]: https://github.com/rust-secure-code/cargo-geiger
[2]: https://github.com/trailofbits/siderophile
You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword.
To my understanding, the idea of unsafe is that you now leave the guard rails that Rust provides and have to think for yourself.
Tell us how you do it, please, so the world may learn.
The funny thing is that it’s actually harder to protect from certain classes of bugs in Rust. For example, you cannot uncouple from the global allocator as easy as you can in C/C++, and if you do, you do it with “unsafe” code. That doesn’t make Rust a bad language, it’s just that you can see some of the inherent flaws only if you had written safe C code previously and then you find out that some basic techniques don’t translate.
Here's a contrived example, imagine a library with the following public function:
Does a downstream consumer of this library need to do anything to ensure memory safety of their own code? No, because the unsafe `get_unchecked` function has exactly one caller-upheld safety invariant, that the array index is in bounds (documented at https://doc.rust-lang.org/std/primitive.slice.html#safety ), and because this code makes sure to uphold the invariant, the buck stops here. A downstream consumer need not use `unsafe` at all, and thus not give up the guard rails. And even in a single codebase, `unsafe` only "infects" the module that it's in, not the whole codebase, since a module is the unit of encapsulation in Rust. If you make your modules containing `unsafe` as small as possible, then you reduce amount of code that needs audited for safety.And that's the whole point of `unsafe` in Rust: to reduce the scope of code that can potentially be memory-unsafe. It firmly places responsibility on users of `unsafe` to ensure that they uphold the necessary safety invariants; there's never any question as to which code is responsible for memory unsafety, as it can always be traced back to a single use of the `unsafe` keyword somewhere that has failed to uphold a safety invariant. It doesn't outright eliminate bugs--humans are fallible--but it makes it obvious when they might happen, it makes it easier to have confidence that they won't happen, and when they do happen it's easier to figure out why.
Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. In my view, Rust should have a safe keyword to mark functions that are transitively not unsafe, much like the pure keyword in some languages marks transitive functional purity.
That's a silly definition of memory safety. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies. No interior mutability, no allocation, no containers.
The point is these uses of unsafe are checked by the author to ensure they can't introduce unsound behavior through the exposed safe API.
> Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks.
Sort of agree with you here. Unsafe code is rare in C#. But C# has race conditions so some kinds of bugs are more common. But C# isn't as suited for systems programming type of problems or really high performance code.
Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe. The virtual machine implementation itself may be unsafe of course, but that is a pretty solid boundary. In Rust, that boundary is easily crossed by anyone, for any reason, to the point where it sips into 50% of dependencies, as this paper shows.
I am not saying that C# evades this issue in principle, obviously it has the same functionality, it is just not used as liberally. Concurrency is orthogonal.
With the obvious exception of JS in the browser, I don’t believe this is true: Python, Ruby, Java, C# all have widely used unsafe native escape hatches that are regularly found in dependency trees. All you have to do is misuse these interfaces, which is significantly easier than in Rust.
It's true that most language implementations have an escape hatch in the form of a foreign function interface and/or they have native extension modules, nevertheless non-trivial programs can be written without using them. Conversely, using such facilities is often a reason why a program may not be portable between multiple language implementations.
But this is true of every other programming language I'm aware of.
So if you want to define memory safety to be something never achieved in practice, fine, but that's not a useful definition in my books.
Same difference, because the use of unsafe is transitive. There is no distinction between "unsafe-but-part-of-the-standard-library" and "unsafe".
> But this is true of every other programming language I'm aware of. So if you want to define memory safety to be something never achieved in practice, fine, but that's not a useful definition in my books.
I don't. I already made that distinction further up in the thread, you can read that and reply there if you have any objections.
Languages like C# are more memory safe than Rust because the uses of unsafe code are fewer and more concentrated in the runtime where bugs are more easily caught. I think that's a fair comparison.
No. I will now apply maximum charity and explain the distinction once again: Let's take Java. It runs on a VM. The VM has a memory model. There is no way to write memory-unsafe code while staying within that model. Not only can you write non-trivial programs against that model, the vast majority of Java code is written against it and is thus memory safe. That's a fair definition of memory safety.
Of course, in practice, you will likely use an implementation of that language that uses unsafe memory access. You will likely use an implementation that has escape hatches to call into native code - unsafely. That's a reasonable boundary to have while still calling the language memory-safe. Unsafe in C# was a late addition that can usually be disabled without much peril.
Yes, but those errors can only be caused by code flagged with the unsafe keyword. Which makes debugging, auditing etc. a lot more simple than in, say, C, where it could happen anywhere.
You can speculate that this is a significant improvement. I would argue that a disciplined use of a simple language like C would, in practice, be safer than using a complex language like Rust which has "safety" written all over it - but then liberally sprinkles unsafe code into all the gaps.
We can both agree that C++ is the worst of both worlds.
Right, and that isn’t something Rust or the unsafe keyword set out to solve. Unsafe has absolutely no runtime impact and was never designed to.
Your argument that a disciplined use of C would render this useless is correct, but history has shown us a great many examples of undisciplined use of C, even in projects that start out disciplined. Rust codifies and enforces that discipline, I don’t see anything wrong with that.
You want unsafe to be something it isn’t, but I’m not convinced what you want fits with Rust’s vision. There are many, many safer languages to choose from.
I completely disagree with that. There's no discipline required to slap "unsafe" around a block of code. The discipline must be applied to the block of code itself, there is no way to enforce that. If 30% of dependencies have unsafe code in them, how would I know that the unsafe parts aren't just as sloppily written as the average C code?
Rust has yet to prove itself as a language that actually delivers on its promise of being safer than C, in practice. To that end, we must compare C written with security in mind to Rust written with security in mind. Given that Rust is a rather complex language that encourages abstraction, I'm quite hesitant to drink the cool-aid right away.
But this suffers from selection bias: people don't tend to write the same sort of high-performance libraries in C# as they do in Rust precisely because of how much harder it is to squeeze out every last drop of performance from a managed language (even taking into account C#'s own `unsafe` keyword). If Rust didn't offer `unsafe` and only privileged its own first-party constructs then it could certainly produce a more broadly safe ecosystem, but it would not produce a more useful one. And therein lies the whole point: the goal of Rust is not to produce a safe ecosystem for its own sake, but to provide a more reasonably-safe alternative to C and C++ for the sorts of things that C and C++ get used for.
Either safety really matters and performance can be given up for it, or vice versa. I do not believe that Rust is a "reasonably-safe" alternative to C, because C is already reasonably safe if written with safety in mind, but with the added bonus of being simpler. Rust is reasonably unsafe when written with performance in mind, whereas C++ is just unreasonably unsafe.
You can wrap `unsafe` functionality without checking and thus have memory issues, or if the unsafe wrapper hasn't correctly validated all possible entries that can cause memory corruption. Then any consumer of this dependency has a possible security issue with the right inputs to the consumer, which can trigger passing an exploit down to the unsafe portion.
* Your hardware is broken. Did you spill the entire glass of wine on that laptop?
* The Rust compiler has a bug. Or, equivalently, LLVM has a bug and rustc depended on LLVM to be correct. Or, your OS kernel has a bug. These are all unlikely, but they do happen.
* A safe wrapper in a popular library you depend on is not fulfilling its contract and you noticed before anybody else.
* A safe wrapper in your code, or in some lesser known crate you depend on, is not fulfilling its contract. If you just fired Jim for writing LGTM without reading his code reviews, start with any code Jim reviewed that has "unsafe" in it.
Notice that the blame definitely is not on the safe code which relied on a guarantee that wasn't fufilled. This is a human construct layered on top of what Rust's compiler is actually delivering, but it's still important to think this way.
If in hindsight the function Jim didn't review correctly should be marked unsafe because you need to check your parameter wasn't zero when calling it, do exactly that, never write...
... in the comments for the function, you're making your own life harder.For fun, here's an example of the Rust stdlib itself working around such things: https://github.com/rust-lang/rust/blob/74ef0c3e404cc72c08b2d...
If the argument is "You can use unsafe but do things safely", isn't that the same argument as C? "Just have faith that they did it right"?
For example, you wish to implement a Vector in Rust.
You generally don't want to allocate that Vector over and over, so you request more memory than you need.
That "extra" memory is uninitialized which is a big no-no in Rust. You might be able to initialize that memory, but that will have performance implications that C/C++ programmers would just laugh at.
However, your "invariant" is that you never access that memory until you have a real element that you put there. That's perfectly fine.
Nevertheless, accessing that uninitialized area will always be "unsafe" from the Rust compiler point of view.
* You can dereference raw pointers. Are they pointing at anything at all? At something that type pointer might reasonably point to? Not the compiler's job to check, if you screwed up your program now has Undefined Behaviour like C or C++
* You can access C-style unions. This one says it might be a boolean, a pointer to something, or an integer. You decided to guess it's... a boolean. If you're wrong you get Undefined Behaviour.
* You can mutate (change) static variables. Global cheese flavour is Cheddar? Let's change it to Gorgonzola, no wait Parmesan. Was anybody else using that for anything? No idea, too bad, you might introduce Undefined Behaviour if you didn't think this through.
* You can call functions Rust labelled "unsafe". These functions come with instructions about rules you must follow to use them safely. If you violate any of those instructions, all bets are off, but if you obey the instructions whoever wrote the function (which may be the Rust standard library) promises that was safe.
* You can implement special Traits labelled "unsafe" to implement. These Traits, unlike most Rust traits, have to promise they're correct. If you claim to implement Searcher, a Trait for text processing, and you get it wrong, other people's code may blow up. Whereas I have types like Funhouse<> which claim to implement the trait Eq (promising they understand how equality works) and then as a goof they utterly refuse to do so correctly, that's a safe trait, nothing blows up. My types don't work in a sensible way (Funhouses are simultaneously equal to and not equal to every other Funhouse including themselves, which is stupid), but your program doesn't have Undefined Behaviour if you use these types, they're just annoying.
But, lots of things that would cause Undefined Behaviour in some popular languages are impossible even in unsafe Rust.
Suppose I have a function that takes an array of 1024 bytes. I try to do { let z = array[1040]; } -- That won't compile, this array isn't big enough. If I write unsafe { let z = array[1040]; } it still doesn't compile, didn't I get it, this array isn't big enough.
OK, let's have the function take a parameter k, and at runtime we can set k to 1040 and try that way. If I write { let z = array[k]; } when k is 1040 the program panics. The array isn't big enough. If I write unsafe { let z = array[k]; } it still panics and the compiler even warns me that unsafe isn't doing anything for me here and I should remove it.
Similarly although Rust provides checked arithmetic (ie you can explicitly say "Tell me the answer to x + 100, or, if that would overflow, tell me it didn't work") its default integer arithmetic will panic in debug builds if you overflow and, if you missed that in debug, but it happens in production, you get overflow, which may very well not be what you wanted, but you apparently didn't even know it could happen, so, good luck with that. Either way, no Undefined Behaviour. 255_u8 + 255_u8 is a compile error, a runtime panic or at worst it's 254, but it definitely is not zero, forty-two, or unrelated program misbehaviour.
For example, Rust's String type internally allocates, reallocates, and frees memory. It can have partially uninitialized buffer, and lots of other "unsafe" things.
But there's no way to cause UB when using safe Rust's String. You can't cause use-after-free. You can't cause a data race. You can't overflow the buffer. You can't even break its UTF-8 encoding. That's because String's public interface only exposes safe methods. The unsafe String internals were written and checked for correctness, and from then on they're safe to use for everyone else.
[1]: the most famous bug in the standard library exposing an unsound construction is the “leakpocalypse”: https://github.com/rust-lang/rust/issues/24292
Rust changes the bug-spotting game from "Where's Waldo?" to "Is this a Waldo?"
Pragmatic techniques mystify some.
Some code is beyond the ability of a compiler to verify because the necessary information is missing. Rust acknowledges this reality and accommodates such code through unsafe. The presumption inherent in this is that unsafe is used only when deemed necessary. Yes, deemed; a decision made by some flawed soul, as opposed to a compiler. This is a pragmatic policy and it cannot be reconciled by the unreasonable.
The key here: Rust SURFACE AS TYPES this information.
Is similar to this piece of code (imagine is python):
what it actually do? WHO KNOWS. But this: is informative. And the "type system" can verify its use.So "unsafe" is part of the type system, type checks that Rust use to COMMUNICATE and CHECK the use of code. So, if I, as user see this:
I know the type system of Rust was bypassed and must TRUST the HUMAN know how correctly code what is inside.---
The major advantage here, is that the blast area of this stuff is limited and for the rest of regular code, the type system verify the code and remove the need to manually inspect (and make tests!) for a lot of trivial (and not so much1) stuff.
Is a total game-changer, in special for a system language.
What worries me is that when it comes to safety, surely the chain is as strong as the weakest link? You declare one thing unsafe, and then suddenly all bets are off. And maybe you don't even know that what you're doing is unsafe.
But here's an example I pulled from github, which blinks a Raspberry Pi Pico:
Right, but what if you're fiddling with registers that might require atomic operations? Lines such as this seem to be saying "ignore all that, write to the register anyway."Fearless concurrency, you say: are you sure? How do you know?
Suppose you have an ISR (interrupt service routine) that wants to toggle a GPIO pin. From whence does it borrow the privilege to set the pin?
Surely, at the end of the day, microcontrollers can be characterised as large state machines with global state. That's what they ARE. It's their inherent nature, which you cannot abstract away without it leaking somewhere (at least to do anything useful, anyway).
At least with C/C++ you KNOW you've got a problem, and have to reason about the problem carefully. With Rust, you THINK you've solved the problem, but have you?
"Change My View", as they say on Reddit.
*Update 1* OK, I'm going to press my point on this. I'm looking at the Embedded Rust Book (please remember, I'm not a Rust programmer) at https://docs.rust-embedded.org/book/concurrency/index.html
I'm considering the issue of interrupts here.
" Instead, our interrupt handlers might be called at any time and must know how to access whatever shared memory we are using. At the lowest level, this means we must have statically allocated mutable memory, which both the interrupt handler and the main code can refer to."
"In Rust, such static mut variables are always unsafe to read or write, because without taking special care, you might trigger a race condition, where your access to the variable is interrupted halfway through by an interrupt which also accesses that variable."
RIGHT, but that means that Rust gives me no better assurances than C/C++. Here's their code:
Then is says "Can you spot the race condition? The increment on COUNTER is not guaranteed to be atomic"It then solves the problem by showing how to introduce a critical section.
So at the end of the day it's just C with different syntax, and a lot more hoopla. You've made everything mutable and unsafe. Exactly the kind of thing you were trying to avoid in the first place.
Having written C and C++ for about a decade and Rust for about 2 years now, I don't think this is accurate: people don't reason any more carefully about C and C++ than other languages, and neither language (even in their safer subsets) encourages methodical safety thinking.
This is in direct contrast to Rust's absolute requirement that you annotate anything that isn't already safe by construction with an `unsafe` keyword. You might be wrong about the safety of whatever you're constructing, but that alone is a significantly more methodical and careful approach to encapsulating the "weak" parts of programs.
Last time you used an higher level interpreted language (like JS), did you have to worry whether or not the machine instructions it was generating were the right ones? Generating instructions at runtime sounds pretty unsafe, no?
You didn't, right? You can just mostly assume the language works and get on with your life. That's how unsafe feels to me when I'm programming in Rust, I never have to worry about it and pretty much never need to use it.
Sometimes you'll run into a problem where you need to use unsafe, but that only happens very very rarely, for example when creating an interface to a C library, but most people will never need that. When that happens, sure, you'll have to be extra careful and make sure everything works. But after you're done with that small piece, you're back into safe Rust.
So, how does it work in practice, if you really need to use unsafe? Suppose you have a C enum with values MyEnum {A, B, C} (but it can be any int in C!), and you want to use it in Rust. You can make an unsafe wrapper that tries to parse the enum, and it can either return None - if it was an invalid integer, or return Some(A), Some(B), Some(C).
But once you get the wrapper right (which is not hard), Rust will guarantee that values of type MyEnum have values A, B or C and nothing else, so you never need to worry about the exceptional case again.
You worry about it once at the boundary and that's it.
For example, can rust make sure some kind of function is not called at any level when called from certain kind of context (say atomic context).
Can rust help detect/prevent you from accessing some multi-register SFRs or memory locations from main context without making you disable interrupts first in the main context?
And this constraint is only relevant if the SFR or said memory location is ever accessed from some ISR, and the access is not atomic, otherwise disabling interrupts is not necessary.
Do you have to wrap all these accesses and keep track of all this manually just like in C or does rust have some concept of "colored" functions, so that you're forced to wrap accesses to certain memory locations only if these accesses are shared between differently colored functions.
I'm not a rust person either. Just wondering how rust would help me with the most complicated aspects of concurrency in low level programming. Allocating/deallocating memory and keeping track of it is easy compared to this.
In your case, you could implement something like an InterruptDisabler<T>, and use it to wrap an object which has the methods to access these registers. When you wanted to access these registers, you would do something like "my_interrupt_disabler.disable_interrupts()", which would disable interrupts and return an InterruptDisablerGuard<T>, which then would you to access the T. Once that InterruptDisablerGuard<T> gets out of scope, the compiler automatically calls the drop() method from the Drop trait on it, and that method could enable interrupts again. You cannot access the inner object without going through the guard, and you cannot get the guard without disabling interrupts.
And that's only one possible implementation. Another one would be to have something like an "interrupts disabled" token which is returned by a "disable interrupts" function, which is consumed (that is, destroyed) by an "enable interrupts" function, and which is !Send and !Sync (so it cannot be passed to another thread). The methods which access these registers would require you to give them a reference to the token, so they could only be called if you somehow obtained the token, which could only be done by disabling interrupts in the current thread. And for when you're within the interrupt handler itself, the low-level code which calls it could manufacture one of these tokens and pass a reference to it to the interrupt handler, so it would be able to access the registers without calling the "disable interrupts" function.
Of course, both of these ideas still require you to decide which registers are safe to call with interrupts enabled, and which registers are not, when creating the objects which represent the register blocks. Rust can help the developer, but it's not a panacea.
Say `some_sfr = 1` is safe but `some_sfr = (some_sfr & 0xf0) | 0x3` is not.
Token thing seems interesting, but generally, no code calls the interrupt handler, HW does that by itself (outside of the control of the language), so there would be no way to pass the token to the handler, it would have to be manufactured in the handler out of thin air (without disabling interrupts, because those are disabled automatically by HW). And I'd have to wrap all the non-atomic language constructs in some functions that would require the token, to have some enforced safety.
I guess the fearless concurrency thing is only for cases where the language controls the creation of threads.
The stdlib is full of unsafe, I'd much rather they handle the unsafe parts which have been audited and battle tested and I use their abstractions.
It's almost impossible to avoid unsafe somewhere in the chain. But that's not really unexpected. What's nice is that once you have a safe wrapper, it's impossible to misuse and cause memory errors
In that article, they have three goals: verify that the pin is correctly configured, use atomic reads and writes, make sure only one thread (including interrupts) can write to a pin at a time.
Spoiler alert, the way they avoid interrupts screwing everything up is with `cortex_m::interrupt::free(|_| {/*read/write pin*/})`, which executes the passed closure in an interrupt-free context. The author's solution isn't atomic, but the borrow checker and the fact that they target single threaded MCU means no one else can be writing to the port at the same time.
The unsafe thing being done here is calling w.bits() which is a method on w (the bits we're going to write to this register in the hardware) that allows the caller to set it to any value of their choosing, even values that nobody said are OK for this hardware.
There are two reasons why you might do this. Maybe your hardware was too lazy to provide safe ways to set every value that could be OK. This seems entirely plausible for the Raspberry Pi. Or, maybe you are crazy and you're writing values the hardware maker said never to use. Either way, Rust doesn't know anything about how the Raspberry Pi GPIO unit works and what bit values are OK.
This unsafe code does not write bits to registers. This isn't bypassing some imaginary safeguard that prevents touching stuff simultaneous to an ISR. It's critical to understand that.
The surrounding safe code performs the register write, as an atomic operation (AIUI a single hardware instruction on the Raspberry Pi) using the result of the closure.
> I'm not a Rust programmer, so be gentle with me.
> So at the end of the day it's just C with different syntax, and a lot more hoopla.
So what seems to have happened here is that you're completely ignorant of the topic and you've mistaken that for expertise. That's a bad habit to have developed, and it's important to break it.
The exclusivity/atomicity in that case is ensured because the method is called on "p.SIO", an object that can't be accessed from both ISRs and the main code at the same time in safe Rust (because it doesn't "implement Sync"). If both an interrupt and the main thread need to access that peripheral, you need to provide some way of sharing it - either using `unsafe`, or in safe code by using a synchronisation primitive such as a critical-section based Mutex that provides that guarantee.
The book chapter you've linked to starts out by demonstrating what is essentially how you'd write this in C and thus requires unsafe, but it builds towards a safe solution - when using either the Atomic* variables (in the Atomic Access section) or mutexes (in the Mutex section), `unsafe` isn't required any more; the user's code is only safe Rust which provides synchronised access to the shared state between the main thread and the ISR.
In other words, the benefit over C is that it _is_ now possible to use only safe Rust to access memory and peripherals from both interrupts and the main thread, and that safe Rust is itself ensuring you can't cause race conditions. The unsafe option is there as a building block for those safe abstractions.
> So at the end of the day it's just C with different syntax, and a lot more hoopla. You've made everything mutable and unsafe.
Perhaps that chapter isn't getting the right message across then. The goal is to completely avoid applicaton code having to make things mutable and unsafe by providing the right abstractions that allow safe Rust to get the same work done while ensuring there are no races.
Even if you write "unsafe"-free Rust that only relies on the Rust standard library, you need to trust:
- The Rust compiler and everything it uses to be correct (not just memory safe), e.g. LLVM.
- The Rust standard library, and everything it calls in to, e.g. system libraries, the kernel, the CPU.
That's a ton of code. By calling into an additional library that uses the "unsafe" keyword, you're just adding that library to your set of things you need to trust.
Naturally, you probably have higher trust in the Rust compiler and OS kernel then you do in some Rust library on GitHub, but nothing is 100%.
And the GitHub Rust library probably only uses a few small blocks of "unsafe" code. That makes it easier to audit and increase your confidence than if it were written in C.
Plus, the Rust safety guarantees don't cover everything. You're still trusting that the library doesn't have a bug that accidentally writes to the wrong file, or sends the wrong value over the network.
C++ defines a memory model, allowing programmers to "rely" on certain guarantees from the compiler and runtime library. But you're still trusting the C++ compiler, the runtime library, the OS kernel, and the CPU to work the way they're supposed to. That doesn't mean the C++ memory model is useless.
Sure you could throw unsafe everywhere and not check anything, but that misses a major reason to use rust--safe abstractions over unsafe behavior. Once a safe abstraction exists for unsafe behavior, all the heavy lifting of guaranteeing code will not result in undefined behavior is offloaded to the compiler, instead of the programmer having to keep track of it all.
Yes, you are correct. If some code or module contains a memory safety or data race bug, then the code which uses it could realize that bug in the form of a crash or race. This paper is not analyzing code that contains a memory safety or data race bug, but instead, code that the Rust compiler cannot prove doesn't have such a bug. That is a big difference.
Let's take your example. The questions you bring up are good ones. Fiddling with registers may indeed require atomic in the presence of multiple threads using that register. Or, if you can guarantee only one thread touches those registers, it may not. In both use cases, I'd like the compiler to help me verify that those conditions are met. A Rust approach to this problem (verified empirically by the paper) would be to create a module, function, or trait that enforces the invariants we need. Such an abstraction might contain a bug that causes the codebase to be unsafe. But when debugging that, we shouldn't have to look over the whole codebase for register writes but rather in just one place.
> Suppose you have an ISR (interrupt service routine) that wants to toggle a GPIO pin. From whence does it borrow the privilege to set the pin?
Assuming we are protecting the pin with atomics, the ISR could call into our library to acquire the lock. The ISR doesn't necessarily need to borrow the privilege from another thread. Our module could provide a safe interface for unsafe access to a global lock.
I would also ask how in C/C++ an ISR performing such an operation would work without data races. I assume it would need to perform a very similar set of operations to the compiled Rust binary. The only difference is that it is harder to find that gpio pin manipulation in the C code. That isn't an accident Rust's explicit goal is for its abstractions to be zero-cost.
> Surely, at the end of the day, microcontrollers can be characterized as large state machines with global state. That's what they ARE. It's their inherent nature, which you cannot abstract away without it leaking somewhere (at least to do anything useful, anyway).
First, this can be said about any computer, no matter how small or large. But I think that you are getting at something important here. Rust is a much more natural fit in user space on an operating system that abstracts away the hardware details than in a micro-controller. And certainly, on a microcontroller, you have to interact often with IO of various kinds, which are global state that Rust considers unsafe. But I think, to the extent, it is possible having the unsafe/safe stratification of code enables you to separate the code which doesn't interact with the outside world and the code which does. I think that is abstractly similar to the user/kernel space divide and Haskell's IO/computation divide. I believe that split helps with code organization and concentrates the attention when such a bug occurs.
> At least with C/C++ you KNOW you've got a problem, and have to reason about the problem carefully.
First, you should know that Rust's safety guarantees extend only to memory safety and data races. Second, I don't think that it is true, in C/C++, that if you have a memory safety or data race bug, you necessarily know that you have one. If you do have one, it could be in any part of the code. Reasoning about a large codebase is much more difficult than a single module with all potentially unsafe code.
> With Rust, you THINK you've solved the problem, but have you?
You may well not have. Here is an example of Rust's core libraries containing a CVE [1]. And here is another popular crate that had a CVE relating to unsafe code [2].
The paper noted the unsafe code was slightly mo...
By this definition, is there any memory safe language?
They all boil down to unsafe dependencies. Even if you ignore interpreter abstractions, any language with FFI is unsafe but without any way to audit it.