So it's your contention that the comment should read, "At least my usecase has C completely disconnected from the internet. I have no idea how [using memory-safe languages] is going to work at the OS level."
I struggle to understand the comment. Do these two sentences relate to one another or is the second one a non-sequitur?
I mean it's obvious why it works. The less code that has to be heavily scrutinized for memory issues, the more the few people that can scrutinize such code can focus on the parts that still need that level of scrutiny.
As other commenters have said, you can do it a bit at a time, converting non-core kernel code to Rust at the edges. Another route to memory (and other) safety would be the C -> WASM -> C route that effectively rewrites/transpiles code to operate within its own sandbox, which also has the advantage that it could at some later date support modules written in languages other than C.
I realize this is beating a dead horse, but it really is a shame that microkernels didn't win for human-facing Unix systems. This issue was recognized and basically solved decades ago, and we're still almost there.
Yeah, I've wondered a bit if the Spectre class of vulnerabilities was discovered a couple decades sooner if it would have changed the course of history there.
I do wonder what this landscape could have looked like if all of the effort had gone behind a Mach-based microkernel. I read all of the back and forth between Tanenbaum and Torvalds. At the time, I was just so excited to see "free and open" winning.
I should specify that the XNU Kernel from Darwin/OSX took a lot of my attention when it first premiered.
If you're willing to split your workload into VMs in the first place I think application-level segmentation like Qubes starts to become a more interesting idea that microkernels.
There is a lot of flexibility that you get from microkernels outside of isolation that you don't get from virtual machines. I'm not suggesting use one GNU/Hurd VM for one application (similar to what you would get with Qubes). I'm suggesting using one GNU/Hurd VM for many applications.
I consider servers to be human-facing systems. I more meant to account for the fact that L4 is used in Qualcomm modems and Minix is used for Intel ME. I would not consider either of those as human-facing.
What's the uptake of Rust as far as Linux kernel dev goes? Is it being positively welcomed and seeing a rise in usage or is it still at PoC stage? Any ongoing rewrites of important subsystems?
It may not be much at the moment, but the thing that I find encouraging is that there seems to be very little pushback against it; the sanity of the Rust approach is very much accepted now.
Last I checked still very early days with most of the work/experimentation happening in the driver space. However, it doesn't seem like its facing a lot of resistence and Torvalds seems reasonably positive about it atm.
To really be real if air-gapping is a significant mitigation for your risk profile then you are likely the target of one of those rare APT's. You should probably also be using a memory safe language if you think you need air gapping.
wait for gov contract projects mandate them in the fine prints, it's a signal that is serious enough for anyone interested in doing any software-related business with gov, for them, this is nearly the same as 'mandated'
I for one would prefer that government projects set this kind of requirement. I don't want my government software implemented in C++. If they are the customer they should require that the projects do everything they can to eliminate bugs caused by memory unsafety. Using a Memory Safe language is one of the easiest ways to do so.
But, on the other hand it’s not. The government is a virtual monopsony. One customer has unappealing requirements and you can choose not to serve them. The federal government saying don’t hire c devs will hurt c devs. C devs will be understandably dismayed by this.
But this differs from the above statement that specifically referred to "government-related business". The "business" part implies a contract but it doesn't mean all non-govt contracts need to follow suit.
I don't understand the point you're trying to make. You're the person who wrote it would be mandated, and now you're admitting it's a ridiculous position to take.
If you have some point to make please do so earnestly. Layering in levels of irony makes any point you're trying to make difficult to understand or follow, even if labelled with /s.
This is going off-topic, but there is a style of internet arguing that I have come to seriously dislike. It is one where instead of someone making a point, they make the point they wish to detract and simply flag that they are being ironic. In doing so they don't actually advance the point they're trying to make, they assume that the audience already understands and is sympathetic to that point, so they simply put up a target to scoff at.
I'm not sure if that's what you're doing here, or if you're just struggling to make a point about worrying about government intrusion into private business.
I guess the GP is trying to say that those languages will enter the compliance checklist of some unavoidable rule, and that recommendation will turn into an effective mandate.
What is a quite real possibility. For example, there are plenty of places out there that can't stop expiring passwords every 1 or 3 months because it's in one of those lists. But I do agree that complaining about the recommendation because of this is completely out of topic, the focus should be on the rule that actually mandates it.
Well, there are groups that want a government mandate for this, though.
If you don't know, Consumer Reports is paid by groups interested in encouraging the government to apply regulations to certain areas. A bike helmet manufacturer may pay them to create a report, host events, and otherwise lobby on their behalf to e.g. create regulations about people needing to use bike helmets.
It is my understanding that many Rust advocates, security researchers, and members of the Internet Society are effectively advocating/lobbying for partial government mandates of 'memory-safe languages'[0]:
> It’s not yet possible for government procurement to only buy memory-safe software. For example, you can’t say routers must be memory-safe top to bottom because no such products currently exist. But it may be possible for the government to say that newly developed custom components have to be memory-safe to slowly shift the industry forward.
> This would require some type of central coordination and trust in that system. The government could ask for a memory safety road map as part of procurement. The map would explain how the companies plan to eliminate memory-unsafe code in their products over time. The carrot approach for memory safety may include not just decreased future costs in cybersecurity, but also reliability and efficiency.
Ok fine, then to quote the parent to whom I was responding, “Rust advocates.”
I am willing to bet that most of those “Rust advocates” are programmers who code in Rust but I’m fine with not calling them that. I agree that good programmers should be able to work in different languages, operating systems, countries.
My point is that even in a world where the government mandates that certain applications that would otherwise be written in C++ are instead written in Rust or Swift we wouldn't see some massive loss of work for people who currently program in C++.
Ok, that’s a fair point. Imagine you’re Bjarne Stroustrup, sure he doesn’t consider himself a c++ programmer, and he isn’t worried about his employment prospects, but he still has a vested interest in this issue.
We may see that the 'next generation' of 'better' programming languages (the next Swift, the next Rust, etc.) cannot easily/readily/practically become certified for government use, though, leading to less innovation long term.
For example, if this had already happened we may find today that Java is certified for use but that Rust is simply not allowed, while maybe Swift is because of Apple's backing of it.
Next thing they'll be giving requirements for people building bridges, houses, and gas and electricity fittings.
Seriously, I think the time has long since passed software needs regulating. It's a major part of modern society, and as far as I'm aware, most people aren't opposed to building standards in principle.
And many buildings were built deliberately poorly to make a quick buck - or caught fire too easily or fell down in minor earthquakes (or worse damage other property/people). Peoples homes are a major financial commitment and can ruin people if they're not up to scratch.
The regulations are to ensure the 10% of bad builders/developers don't ruin peoples lives.
And yet, poorly build buildings are still being build, but at least now it is much harder and much more expensive for a person to build their own house, and for a small construction companies to compete with giant monopolies. Yay!
"Poorly build [sic] buildings" depends on context. Even low-quality builds in the US are relatively safe. It's disingenuous to imply they are of the same quality as in countries where building codes are effectively non-existent.
Regulation hinders progress and makes things more expensive. But no regulation raises costs and hinders progress too: it basically creates a situation of very low trust, and low trust is extremely expensive for customers/buyers who are not domain experts. This also makes them overcautious and conservative. One needs a balance and lots of nuance to make a reasonably well functioning market/system.
The case against regulation on software business is not that "regulation is bad". It's that programming and software is very new and rapidly evolving area of human activity. It's not nearly as well understood as building houses. Written and unwritten standards and best practices are constantly changing. The field is subject to very strong fashion-driven "crazes".
(Just look at how many new languages are still being created. Most don't become as widespread as C++ or Java or Python, but many do find their niche, and very many are in use to some extent. This indicates that "language Holy Graal" is nowhere to be seen yet.)
In software, there is very little consensus between domain experts on most issues. This is very unlike construction and house building, where some new materials ant techniques are introduced too, but the basic principles are well understood, calculable, and where agreement among experts is usually quite achievable.
So arguably it's nearly impossible to create good regulation at this stage, at least outside of certain special niches. This is very different from building codes and stuff.
Of course there is also bad regulation. The bureaucracy likes to expand their control indefinitely, wants to regulate things that should not be regulated, thus (if unchecked) creating very bad regulation. Well, that's the case for checks and balances, for the society to fight back. But "this regulation is bad and needs to be changed" is a much more mature position than "we need no f*g regulation!", in my opinion...
Define "figured out." Did people know how to cobble together a structure? Of course. Did they inherently know all the best practices that reasonably balance safety, cost, and timeliness? Probably not.
The same can be applied to software. An ability to cobble together a "Hello World" does not necessarily mean I want you programming a controls system on a nuclear power plant.
The wild west had a lot less danger and death than Hollywood makes it out to seem. Likewise, I'm going to miss the internet and computing as we know it now when it's regulated to shit like everything else. Nothing nice ever lasts.
Complete safety, or actual freedom. Pick one. You can't have both.
That seems more realistic, because I'd argue you can't really have "Complete Safety" or "Actual Freedom" (by which I am guessing some people would interpret 'actual' as 'complete').
>Complete safety, or actual freedom. Pick one. You can't have both.
False dichotomies aren't helpful and underscore lazy thinking. The real world is full of nuance, and so should our policies. Using a risk-based approach is probably more appropriate than an all-or-nothing policy.
If software had been regulated like engineering we'd still be writing horrible OO-heavy enterprise Java style code like in the 90s and UML would be mandatory.
We'd probably be using Ada. The government used to mandate that contractors write software in Ada, but they stopped because of the pushback they got; now they also use C++.
They'll "regulate" a bunch of controls to make corporations more efficient at making money, while limiting their liability when they get hacked and increasing the barrier to compete with them.
If we want a safer internet, make them carry insurance against data breaches and fine them a fixed amount, say $1500, for each identity they leak, paid immediately upon proof of pwnage.
I agree that it is not a mandate yet. NSA does have some interesting insight and expertise they acquire in this area.
Time and time again, we see experts make recommendations, then legislation and rules make it mandatory.
A burning example is most of NIST special publications. NIST makes no rules, mandates and such. Yet, mandates (e.g.,DFARS, DEAR) point to the recommendation as the requirement.
Right now there are two of these playing out in the cybersecurity field - zero trust and passwordless authentication.
So those who down this comment, you are right, it is not a mandate. Those who up this comment, you are right, it is likely to become a mandate.
My impression of historical Python is that is an old, partially arcane language that due to D.S/AI is now popular; I would initially think it would be no better than those other interpreted. Python does these things well... mainly due to pandas/dataframes/polars.
Language wise, Python is basically just Perl with its arms cut off and bunch of makeup added. It's just very popular in the scientific community, so they have to add it.
I don’t know how you can possibly come away with this conclusion after studying both languages. Maybe if your point of reference is an ancient version of Python?
Yes it is and current Python is basically just an object oriented system hacked on top of ancient Python. The underlying elements beneath the syntax is basically all the same.
No offense to Ruby, but if I were going to push people to change to a new language I would not recommend Ruby. It had its time of popularity and I feel like it's just a niche language now, surpassed by others.
From a memory safety perspective, I would say the same about python. The libraries that make it popular are all written in c++ to my knowledge. Anyone not already using python usually has better tools to choose from these days.
> if I were going to push people to change to a new language I would not recommend Ruby
None of the alternatives except Go and Swift is new.
Ruby has turned into a robust boring technology, it does not mean it’s unworthy.
The Ruby team is still pushing the languages features and speeding up the runtime. Take a look at Ruby Ractors for example.
I do have to agree on that Ruby is more of a niche language now thanks to Rails but I’d say Ruby works very well for scripting and gluing together different systems.
It’s the only scripting language listed — I suspect scripting languages weren’t the focus, but that it got special consideration and added to the list to not place the AI/ML R&D communities in an awkward spot. Of course they should move to something besides Python, but network effects give Python powerful inertia, and AI/ML is a strategic area.
Really, the only memory unsafe languages still in use are C and C++.
If it weren't for the behemoth of legacy code we'd really have this problem more-or-less licked. Unfortunately, that behemoth is still rampaging across the landscape.
"Rewrite it in Rust" gets a bit of pushback, perhaps even justified, but at this point in time I'll take anything that just reduces that behemoth in size. The journey of a thousand miles begins with a single step, an elephant is eaten one bite at a time, etc. Rust is just one of the easier and more effective options for a legacy codebase, with the unusual advantage of being able to slip in incrementally. Almost every other language requires a true rewrite.
In summary, Ada tries to be memory safe by default -- as far as that can be done without requiring automatic memory management and garbage collection -- but deliberate use of "unchecked" language features can break memory safety.
In other words, if you go out of your way to use unsafe features, and don't use the features that compensate, Ada is memory unsafe. This has become the goto dismissal of Ada, apparently more popular than "eww...a BEGIN..END language" and "designed by committee/government tainted".
Ada isn't, Ada/SPARK is. That's a subset of Ada, and while it is the main draw of the language for new projects the majority of extant Ada code predates SPARK.
Fortran does not end up where I'm too worried about its security. C and C++ does. At the scale I'm talking about I'm not sure we'd even say Fortran is "in use".
Assembly is in use, yes, but in 2023 I feel there is generally an understanding of the risks and I haven't seen the "write everything in assembly" crew in about 15 years. The problem is that there's still too many programmers blithely using C and C++ without realizing the risks and thinking they can cowboy through the problems. For every line of vulnerable, dangerous assembly I bet there's thousands of lines of C or C++.
There is also the problem that there have been some big bugs that got through even static analysis and fuzz testing, but I'd still be at least reasonably satisfied if all the critical software in C and C++ would be supported by those tools. Interpreters and compilers have had non-zero error rates too.
At the scale I'm talking about, Ada is a non-entity as well. It isn't used. "But it is! I'm a professional Ada programmer!" says someone reading this to themselves. In which case I would say, you darned well know what I mean and don't pretend otherwise just to try to score useless internet points. Ada is not a relevant force on the programming world. That may be sad, but it's true.
But we do not plan on rewriting SQLite, that much I can say :)
(My reading is that the GP points to the exemplary achievement that SQLite has reached a close to (security) bugfree, at what I consider a nearly superhuman effort)
Plus everything that needs to directly interface with the above languages. So many Python libraries that are one "funny integer" away from a nightmare debugging session.
Why are C and C++ considered the same, in these conversations?
C++ at least has tools to make life significantly more safe. I can write a buffer overflow in any language, and on the scale of difficulty, ASM-C-C++-Rust-Python covers my experience (from easiest to fuck up to hardest).
Yet nobody is calling for us to rewrite everything in python. Why is the line drawn at Rust? It's perfectly simple to trash memory in Rust.
Rust is memory safe by default, with unsafety as an optional feature that you basically never need to use unless you’re writing extremely low-level code, need absolute maximum performance, or are interfacing with libraries written in other languages.
C++ is unsafe by default.
Of course it’s just as easy to write bugs in unsafe Rust as it is in C++ (actually, it’s probably even easier), but defaults matter.
No I've seen libraries that need the user to use it by default. The wgpu library is one example. It's not even that low level. Rust stuff is a little too safe that it influences code organization and modularity as well.
This is a common conception, and I agree to a point. However, interfaces matter. At the interface to _literally any_ system call, unsafe starts to creep out. Either in the wrapper implementation, or in the interface _to_ the system, or even leaking through the wrapper to the caller.
At that point, if we have to re-wrap everything in rust to hide the unsafety of the interfaces to the system (sockets, shared mem, etc etc), then why not just write safe cpp wrappers?
Yes, people are writing memory overflows in their own code, but I'd argue 99% of the critical security bugs are actually in the unsafe interfaces. And we don't really need a new language to fix that. We just need new interfaces.
I love Rust, but using it for anything nontrivial makes the "safe" patina really fade. You're quickly writing what feels like C, with MaybeUninit<X> all over.
C++ makes it very difficult to write safe interfaces. You can't expose references, nor spans, nor variants, no shared_ptrs to things that can't be thread-safely overwritten, nor any standard library containers nor a lot of other things. And even if you only use whatever few interfaces remain safe, the interfaces you create are unsafe by default too. As a result, these unsafe interfaces are everywhere.
I'd contend that using Rust for anything nontrivial results in MaybeUninit & co being common.
> At the interface to _literally any_ system call, unsafe starts to creep out. Either in the wrapper implementation, or in the interface _to_ the system, or even leaking through the wrapper to the caller.
It’s quite rare to have to make syscalls directly in Rust, just like it is in c++. Most code in any large enough system is related to the internal logic of the system, not to its interface with the outside world. And when you _do_ need to interface with the outside world, you can use a wrapper (lots of the standard library is basically wrappers around syscalls; this is true in any language). And no, in Rust unsafety doesn’t typically “leak through” interfaces, unless those interfaces are buggy.
> why not just write safe cpp wrappers?
There’s no such thing. It’s not possible to write a safe interface to c++ code in the sense that that term is used by the Rust community. In Rust, “safe interface” means: assuming there are no bugs in the underlying code, and the client code never invokes `unsafe`, using the interface cannot cause undefined behavior. This is impossible to guarantee in c++.
> I love Rust, but using it for anything nontrivial makes the "safe" patina really fade. You're quickly writing what feels like C, with MaybeUninit<X> all over.
This is not true at all in my experience. I work on Materialize, surely one of the more non-trivial Rust programs that exists. We use very little unsafe/MaybeUninit/C-like code. Do you have an example of a codebase you’re thinking of that does this?
OK this is reasonable. Perhaps my experience skews towards the lower-level a bit too much. And it's also reasonable I'm misusing the language given it's not my day job.
To answer your question, I'm referring to much of the networking code in socket2 / socket, which uses MaybeUninit when doing non-standard stuff like forming your own packets. (RAW)
Yep, I definitely buy that if you're doing very low-level stuff, C or C++ might be more ergonomic than Rust. But I don't think that covers most of the real-world use of C++.
I'm not too familiar with `socket2` but normally in Rust to construct a buffer with arbitrary bytes in safe code you would first zero it out and then write it. Using `MaybeUninit` there is presumably just a micro-optimization to avoid having to memset things to zero.
And that's the problem, I do have to make syscalls directly quite often, and so I dislike Rust immensely. There are literally dozens of us at least, but the only people ever talking about Rust on the internet always like to drag C into the conversation for whatever reason even though they are always C++ programmers.
That's fair! If you are doing something low-level enough that the bulk of the work is interfacing directly with a C library (or with the kernel, in the case of syscalls) then C might make more sense than Rust.
I feel I have to disagree with your (implied) contention that it's feasible to write an API in C++ that, no matter what its inputs are, cannot ever exhibit undefined behaviour.
Because that's what "safe" in Rust means. No memory safety errors, no undefined behaviour.
You're more right than wrong, but I want to push back just a little. You can write a buffer overflow in safe rust if you store multiple things in the same array and work with indices rather than slices. Of course the risk is bounded by what shares an array, and it's more awkward than doing it any of several right ways. You won't write a buffer overflow in safe rust... but you can if you want to.
This is a bit like saying "you can write a buffer overflow in any turing-complete language, because you can write a C emulator, and then write the buffer overflow in C"
People are calling for the use of languages like java or python when it is appropriate. Rust is just specifically mentioned (along with Swift, to some degree) when it comes to applications that have a couple of fundamental requirements that prevent the use of other languages. These might be requirements like no pausing for GC or the ability to run without a VM.
Rust (and Swift) are viable languages for solving most problems that people usually reach for C or C++ to solve today and both make it considerably more difficult, by default, to introduce the most common class of serious security vulnerabilities in the modern world.
I don't think C and C++ are that different. I agree that C++ gives you tools to make safer abstractions, but it still gives little tools to enforce these abstractions. For example std::shared_ptr being easy to use is a great improvement as in many cases you can just use it rather than trying to prove that you don't need it so that you don't need to bother implementing your own reference counting.
In C++ vec[999] is a buffer overflow and you can index any pointer even if it isn't supposed to be an array. There are so many easy mistakes that can be made and aren't obvious to a reviewer. Maybe with a very strong linter you can consider C++ very distinct from C, but by default I don't think it is that different.
> I can write a buffer overflow in any language
Try doing it in JavaScript? If so the Mozilla security team would appreciate a private disclosure. Of course it is possible in any non-sandboxed Turing complete language, but there is a huge difference between the default accessor of the most used container type allowing it vs needing to use functions in the `sun.misc.Unsafe` package or wrapping your code in an `unsafe` block. Making code that may cause a buffer overflow explicit is a night and day difference. It means that you can't do it via a typo in the vast majority of your code, and it will grab the attention of your reviewer very quickly. Isolating the part of the code that can cause buffer overflows to a small part greatly raises the attention that is given to those areas, and greatly reduces the chance of them occurring.
I don't think that Java or Rust prevent all buffer overflows, but I also don't think that it is possible to write C or C++ without them. Sure, it is possible to be careful and avoid most of the buffer overflows most of the time, but we and our reviewers are just human so we will never prevent all of the buffer overflows all of the time.
I don't think that this recommendation is under the impression that "memory safe languages" will prevent all buffer overflows, but the idea is that they will greatly reduce the number. In many situations, I would guess the majority of them, this is a good tradeoff.
What I don't understand is the excitement for using Rust vs. using garbage collected languages like Golang, at least for high-level applications (performant or low-level systems applications are excepted here.) My experience is that an experienced programmer can be a lot more productive quickly with Golang, since they don't need to climb the Rust borrow-checking learning curve. Rust doesn't even free you from the need for a runtime or standard library.
Rust certainly performs runtime bounds-checking as well as some other tasks, so there is runtime code (even if it's just compiled into executables.) If you want features like async (standard in many language runtimes) you're also going to have to pull in some kind of external runtime dependency. And everyone doing high-level web-style development seems to drag in something like tokio.
> Rust certainly performs runtime bounds-checking as well as some other tasks, so there is runtime code (even if it's just compiled into executables.)
I don't think I've ever seen anyone reference "C with bounds checks enabled" as "having a runtime". Does having stack probes also imply having a runtime? I guess I'd be less surprised if it had been worded as "some mitigations/features have a runtime cost".
> If you want features like async (standard in many language runtimes) you're also going to have to pull in some kind of external runtime dependency.
Yes, you can add a runtime to your application (if you need to use async/await). It has an additional cost over not doing that, but the "promise" is that it is "zero (additional) cost (over what you'd end up with if you wrote the functionality by hand)".
For sure C programs have a runtime. I'm debugging an issue right now that is to do with Windows not shipping VCRUNTIME140_1.DLL on out-of-the-box or old versions of Win10, so in that case it's very clear because you can make C programs that won't start due to a missing runtime library.
The runtime isn't all that large but every OS has one. On UNIX it's spread over libc, libpthread, libgcc, libm and so on.
On Linux stack probes usually have some support code in libgcc and/or glibc, if I recall correctly.
(not the parent) You're not wrong but you're not right either. There's basically a colloquialism where many developers say "runtime" to mean "large runtime" and "no runtime" to mean "small runtime," but that doesn't mean that crt0, the "c runtime starting from zero", doesn't exist, just that we've ended up in a place where this gets confusing to talk about because nobody uses the same definitions.
So if it's idiosyncratic really depends on what audience you're talking to.
It's the correct definition and I don't know of anyone familiar with OS design that would claim C doesn't have a runtime. It very much does. So does C++. Every language does, except assembly.
It may feel like C doesn't have a runtime if you're only familiar with UNIX, because the C runtime is guaranteed to come with the OS there whereas other language runtimes are optional and thus more visible. But every language has a runtime.
>Rust doesn't even free you from the need for a runtime
I always thought that Rust was the only memory-safe language that doesn't need a runtime (beyond the libc that every language links to when running on Unix-like OSes). Maybe you could define what you mean by runtime.
From the article: "In contrast, Rust's explicitness in this area not only made things simpler for us but also more correct. If you want to set a file permission code in Rust, you have to explicitly annotate the code as Unix-only. If you don't, the code won't even compile on Windows. This surfacing of complexity helps us understand what our code is doing before we ever ship our software to users."
> My experience is that an experienced programmer can be a lot more productive quickly with Golang, since they don't need to climb the Rust borrow-checking learning curve.
Will somebody please tell me why everyone seems obsessed with optimizing for programmers going from zero to minimally productive?
I have been using Ruby for twenty years, Rust for eight, golang for nine, and C for twenty-six. Most programmers will use a language for dramatically longer than a year, so why are the first three months such a singular point of focus?
The code I wrote in the first three months of using every one of these languages was bug-ridden, unidiomatic, unnecessarily difficult to maintain, and generally terrible. Ironically, Ruby was probably the least bad in this regard. Go and Rust were probably about the same, but I’d frankly give Rust the slight edge here. C was the inarguably the worst, but it was also my first language.
Subjectively and retroactively comparing things a year in, I’d wager my Rust was of the best quality (readability, ease of maintenance, speed of development, bugs per “unit of functionality”), followed by Go, Ruby, and then C. At five years, the quality of my Rust code blows everything else out of the water. My C was still terrible (partly because it was C, partly because it was still my first language). But I’d say Ruby edged out Go at this point for me.
Obviously this is not only anecdata but wildly guesstimated looking back and comparing learning curves on languages at completely different points in my experience as a programmer. I’ll happily admit that Rust pulling so far ahead so quickly is as likely to do with it building off the knowledge of prior decades of professional software engineering. And that my personal experience with any of these languages is of course unique to me and my circumstances.
But it just seems wild to me that people seem to focus on “getting a new person up to speed as fast as possible” to the exclusion of apparently everything else.
Conversion friction. Very important. Arguably the reason why Haskell is not 10-100 times more popular than it currently is; the conversion friction is just too much, and even if all the tooling was perfect and the libraries were perfect and the documentation was perfect it would still have too high a conversion friction to attract a community the size of Go or C# or something.
I can sympathize somewhat with this argument. But it’s also kind of circular to me.
Go being easy to pick up and learn is certainly a virtuous cycle insofar as it helps bootstrap a large community. And that’s absolutely happened!
But that is—in my mind—more of an explanation for why Go has become so popular so quickly more than it is a compelling argument for the language itself. Haskell having conversion friction might explain its lack of adoption, and that’s certainly a great argument in a discussion about why or why not to adopt it for yourself or your team! But it seems like an overvalued axis on which people seem to evaluate languages on their own.
As a counterexample: PHP classically had a reputation as being a language that was very easy for beginners to pick up. And it’s even memory safe! But it also had a reputation for having poor long-term prospects for projects written it as well as being a limiting factor in the growth of engineers using it (note: I make no claims as to the fairness of this reputation, nor to its applicability on “modern” PHP).
PHP is arguably even easier to learn than Go. So why is it that virtually nobody jumps in these discussions trumpeting that?
I’m writing a sibling comment to answer the parent’s question directly rather than the meta-argument from my original reply.
> What I don't understand is the excitement for using Rust vs. using garbage collected languages like Golang… since they don't need to climb the Rust borrow-checking learning curve.
Because, in my experience, climbing that learning curve has made me a better programmer more than nearly any other change in my long career. And that benefit has extended to code in every language I write.
The borrow checker isn’t just some hurdle to get in your way; it’s trying to tell you (awkwardly at times and perhaps less helpfully than one would wish) something fundamental about the way you think about and design programs. Internalizing that lesson can bring significant benefits on designing systems with clean boundaries that are easier to test, easier to reason about, and easier to compose.
Besides that, Rust greatly assists you (through features other than the borrow checker) in building software that is correct. This means it will tell you in a much wider variety of scenarios when future code invalidates previous assumptions. This is invaluable for projects that we expect to survive for a long time since the time a project is maintained will dwarf the time it’s under active development. And it will almost certainly be maintained by someone without the full context of the original developer(s). This is true even if the maintainer is the same person who wrote it in the first place, since our mental model of a program bitrots far faster than the program itself.
In practice, this aligns with my personal experience. Go projects end up with a lot of implicit assumptions that are silently violated by future work and expose bugs. They crash on nil pointer derefs. They accrue a multitude of linting tools that usually paper over some of the language’s shortcomings, but only in common cases. And they become painful to maintain as the original developers move on to other projects, with new changes grafted haphazardly into dozens of touch points instead of cleanly in one or two places. Yes, you can “easily” follow what any particular function does, but to do so you have to parse out and mentally model every minute detail, rather than being able to reason at a high level.
> Why are C and C++ considered the same, in these conversations?
Conjunction is not equality. They are both memory unsafe. Then you can argue from that over how memory unsafe they are in practice (using the right practices, using the right language subset).
What makes you think so? Most Rust programmers and programmers from other languages, agree that this is not possible. I might be missing something, but can you give an example of such simple methods to trash memory in Rust, asking from a curiosity standpoint?
I don’t think most Rust programmers agree it’s impossible at all.
There’s always unsafe. I can make a pointer to anywhere by hand and write to it. That would involve some very intentional work, but I could do it if I wanted to.
I would assume they mean through the use of unsafe, which is true, but in practice unsafe code is less common than people that don't write Rust seem to think and tools like Miri help a lot to write unsafe that doesn't write to memory locations you weren't meant to.
Perhaps they aren't writing Rust because those are the people that need to write unsafe code. Chicken and egg. I'm sure it you forced all the C programmers to switch to Rust you would see a lot more use of unsafe.
But there are plenty of projects out there that are written in Rust and have to deal directly with hardware and syscalls. Hubris, a kernel written in Rust has 94 files referencing unsafe[1] out of 414 total .rs files[2]. This is as "bad" a ratio as you're gonna encounter in a project. There are many valid reasons one can have to not use Rust. "I need a lot of unsafe" is not really one.
Because "significantly more safe than C", while true, is also irrelevant. I want safe, not "safer than grotesquely unsafe". Unfortunately, for all the advances C++ has made, it is still in the "unsafe at any speed" class. It is difficult to escape the foundation of unsafety the entire C++ edifice rests on.
(At least, without further support. I consider "C/C++ with high quality static analysis" to be de facto distinct languages, and while I would favor something else even so, high-quality use of a high-quality static analyzer is enough to calm me down. Things have still crept through that level of care, but then, interpreters and compilers for safe languages have had safety errors in them before too.)
This is particularly true because it's just C and C++ that are memory unsafe. If we still in 1980, we could be arguing about the gradients of unsafety, but in 2023, we don't need to. Unsafety is not necessary at scale.
As for why people aren't asking to rewrite in Python, I partially answered that in my post. You can actually incrementally rewrite in Rust. You can't incrementally rewrite software in Python. There is also plenty of software that can be written in C, but simply can't be written in Python because it would be too slow. (Rewriting it in Python but oh no wait I'll just write the slow bits in C is a no-op, practically.)
As for trashing memory in Rust, by perfectly reasonable convention we generally understand that unsafe is unsafe, and that while languages can't avoid having it, having it does not necessarily make the entire rest of the language just as bad as C. I can crash Haskell with a straight-up, genuine null pointer exception with the Unsafe module in a single line of code. We do not thereby call Haskell an "unsafe" language where it is trivial to trash memory. Stock Rust is far safer than C++, to the point of being not only a qualitative change, but I'd contend, multiple such qualitative changes.
Separating safe C/C++ from everyday C/C++ is not fair, in my opinion, but I get your point: If it can be abused it will be, either by accident, inexperience, or maliciousness.
Once you separate C/C++ into safe and unsafe cateogries, and admit that Rust has unsafe uses that are "just so much harder to use", we're clearly defining a gradient:
Sure you can use safe C++ with some effort, but the libraries you use most likely still use unsafe C/C++. For Rust, I expect that the libraries are much safer in general.
People writing C are the people that really do have to make syscalls directly, or use weird calling conventions, or whatever all the time. I see Rust replacing C++ but I have a hard time seeing it replace C because the people that desired and/or could tolerate safety, like you said, are already not writing C for the most part. That group has been firmly C++ for a long time.
Rust is a little too safe imo. I want a rust with just shared pointers and no move semantics. I guess go would be it? But go is too opinionated with a bunch of stupid go specific philosophies like the weird error handling and the stupid packaging rules.
Go is also opinionated with concurrency. So that's an issue too.
> If it weren't for the behemoth of legacy code we'd really have this problem more-or-less licked. Unfortunately, that behemoth is still rampaging across the landscape.
It is not only or even mostly legacy. I'm a systems programmer (in classical sense, not "but my web service is soooo highly loaded and scalable that I will call it systems programming!") and from what I see on the job people start new projects in C and C++ all the time.
Why do they choose C/C++? Is it just what they and their colleagues already know and nobody wants to be the one to push for change? Easier integration with other C/C++ stuff?
From my experience reasons differ for C and C++ programmers.
C programmers are often more experienced people who are used to "simple" language that gets out of the way. They don't want to invest time into learning tricky language like Rust with all the intricacies of its type system, borrow checker, etc. Something simpler like Zig might work for them, but it is not on the table at the moment.
C++ programmers tend to be people who spent hundreds if not thousands of hours learning its ugly corner cases, reading Meyers and Alexandrescu books, that kind of thing. Sunken cost is immense, they whole careers are built on being "C++ experts" and they dread to abandon it and have to learn another very complex language from scratch.
And managers often don't see value in investing time into switching projects to new language. From their PoV it is more like programmers just want to play with a new toy instead of doing "real work".
Kind of interesting that they didn't include it given python is there. (JS is not even mentioned in the document.) I guess they don't class it as a general purpose programming language? It's not as if we have other options in the browser, particularly, and while node can do a lot of stuff, it's got a clearly intended purpose. Or... maybe they consider it unsafe somehow, but that seems less likely.
Write javascript engines in memory safe languages. I'd vote for rust as rust and javascript's APIs are pretty similar in style, structure, consistency and security/other issues that are not memory safety.
On that note, try valgrind on existing javascript engines, you might be "entertained". (I certainly was, but that was some years back.
Most javascript engines are JIT-based and that's hard to make safe, you'd need a complete proof that the emitted assembly is correct. It's similar to the problem of proving correctness for any compiler.
Oh, I've used such tests on other JIT based languages, as well as non-JIT. None made valgrind show quite so much show. I'm not sure I've ever seen less "memory safe" code bases.
How are Rust and JS APIs similar, and why would it matter?
AFAIK the only competitive JS engines written in memory safe languages are GraalJS and other JS-on-the-JVM runtimes. GraalJS has the advantage of being fully up to date, not having any memory unsafe code in it (the JIT compiler that makes it fast is a separate module, also written in a memory safe language, and the JS impl does not have low level code in it). And you can run it on SubstrateVM which is a virtual machine also written in a memory safe language, although of course small parts like the GC need to use unsafe features.
It also has other useful features like sandboxing and the ability to interop with other languages like Python or Java. Plus, it can actually sandbox native code as well because the "languages" that you can run on GraalVM include both wasm and more usefully LLVM bitcode, in which each individual C/C++ allocation becomes GC-managed and bounds checked.
So in terms of memory safety the Graal team are way ahead there.
(disclosure: I recently started part time work with the GraalVM team, but was a long term supporter before that)
Rust and JS - er in short, there's a lot more to security than "memory safety". But then I've been in groups burned by NPM hacks in the past. I've tried Rust some but it looks like the same problems yet again. I'm wary, and going to give it time to mature - it's not interesting enough on its own - especially for someone more used to embedded programming spaces (C centric spaces, for good reasons).
As for the second - good to know! Seriously appreciate knowing that - not sure if/when I'll need it myself, but it's good to hear, and good that it's visible here!
Neither Ada nor Rust are completely memory-safe... and they are partially unsafe in completely different ways. But I guess people prefer the Rust explicitness.
Ada is safe if you never free memory explicitly. The story for reclaiming memory without GC was always a little weird, basically pool allocation by type. But it does bounds checking, counted strings, and has a reasonably rich type system that allows variants and things in a safe way.
Ada has Controlled types, so the memory reclaiming story can be similar to C++/Rust RAII. What it's missing compared to Rust is the lifetime and borrow checking.
"SQL/PSM (SQL/Persistent Stored Modules) is an ISO standard mainly defining an extension of SQL with a procedural language for use in stored procedures... SQL/PSM is derived, seemingly directly, from Oracle's PL/SQL. Oracle developed PL/SQL and released it in 1991, basing the language on the US Department of Defense's Ada programming language. However, Oracle has maintained a distance from the standard in its documentation. IBM's SQL PL (used in DB2) and Mimer SQL's PSM were the first two products officially implementing SQL/PSM. It is commonly thought that these two languages, and perhaps also MySQL/MariaDB's procedural language, are closest to the SQL/PSM standard. However, a PostgreSQL addon implements SQL/PSM (alongside its other procedural languages like the PL/SQL-derived plpgsql), although it is not part of the core product."
It is odd since Ada is DoD's (bastard?) child and NSA is DoD but a little digging you get this from 1997 (with a singular mention of the word "safety"):
tldr; seems to be that the software development world has changed from the days that DoD was the "dominant" software developer, and Ada in the interim did not get adopted by the commercial sector (with safety critical exceptions in aerospace, etc. noted).
* many GC'd languages like Go, C#, Java make it harder to leak memory, while languages where reference counting is more prevalent (Python, Rust) it can be easier to leak memory due to circular references.
* Languages with VMs like C#/Java/Python may be easier to sandbox or execute securely, but since native code is often called into it breaks the sandboxing nature.
* Formally-verified C code (like what aerospace manufacturers write) is safer than e.g. Rust.
* For maximum safety, sandboxing becomes important - so WASM begins to look appealing for non-safety-critical systems (like aerospace) as it allows for applying memory/CPU constraints too in addition to restricting all system access.
Go is memory safe, that post does not means anything in real life scenario.
Do you have a single example in the last 14 years of memory safety exploit using the Go runtime? I'm talking about public and known exploit not ctf and the like.
Go was released in 2009 and I've never heard about any exploit and what not , by the way this is known and by design it's not new. It's all about the multi word for interface.
I mean if in 14 years there was nothing it's a proof that it's not an issue.
Even the attacker ack that it's not a threat.
"As said before, while a fun exercise it's pretty useless in the current Go threat mode"
Wow, that's super interesting. As you say, it's a contrived CTF example, but I'm pretty shocked that it's possible to read and write arbitrary process memory without importing any packages (especially unsafe, of course).
I'm also surprised that a fix has been theorized at least as far back as 2010[1], but not implemented. Is adding one layer of internal pointer redirection for interfaces, slices, and strings really that much of a performance concern?
It's not safe when using goroutines to access shared mutable data (and most Go code does this). If you stick to message passing a.k.a. "share data by communicating" you don't run into memory unsafety. But this kind of design is more vulnerable to other concurrency issues, viz. race conditions and deadlocks.
That's fine. Safety with concurrency doesn't exist in any language. Only rust is special in that it tries to provide safety with concurrency as well. I haven't seen any other language besides rust actually do this.
The reason is obvious. There's a high cost to this type of safety. Rust is hard to use and learn and many times it's safety forces users to awkwardly organize code.
And there's still the potential for race conditions even though the memory is safe, you don't have full safety.
In this case it's not memory unsafe. It is guaranteed to crash the program (or get caught). It's closer to a NullReferenceException than it is to reading from a null pointer in C. There's no memory exploitation you can pull off from this bug being in a Go program, but you could in a C program
It's only guaranteed because of the operating system's sandboxing.
> It's closer to a NullReferenceException than it is to reading from a null pointer in C.
No, it's exactly the same as a null pointer dereference in C, because it is literally reading from a null pointer in Go as well. In Java, the compiler inserts null checks before every single dereference and throws an exception for null references.
> There's no memory exploitation you can pull off from this bug being in a Go program, but you could in a C program
Provided the OS sends a SEGV signal for null pointer dereferences, I don't see there being a difference in security between C and Golang in this respect. It's a bigger problem when you're running without an operating system.
> In Java, the compiler inserts null checks before every single dereference and throws an exception for null references.
Doesn't OpenJDK install a SIGSEGV handler, and generate the exception from that on a null dereference?
(AFAIK, a lot of runtimes for GC'd languages that support thread-based parallelism do so anyway, because they can use mprotect to do a write barrier in hardware.)
> Doesn't OpenJDK install a SIGSEGV handler, and generate the exception from that on a null dereference?
I thought I had read that they explicitly don't do that, but I can't find it anymore. You may be right. I should have checked before saying that.
> (AFAIK, a lot of runtimes for GC'd languages that support thread-based parallelism do so anyway, because they can use mprotect to do a write barrier in hardware.)
That's true. I guess those implementations must do something more advanced than "throw a NullPointerException if the program segfaults," given their garbage collector runtimes also rely on that signal.
When this happens it'll cause a deoptimization and recompilation of the code to include the null check rather than rely on the signal handler repeatedly.
In huge number of cases the null dereference is not from accessing 0x0 but some offset to it (ie. accessing a struct member or array element that's not the first one). Of course in practice most of the offsets are below the limit where nothing is ever mapped (on Linux vm.mmap_min_addr and seems 64k by default for me) but it's still very possible to have such dereference to not segfault in C. That should not be possible in Go/Java (if it is, it would almost certainly be considered a bug in the compiler/VM).
Why isn't it possible in Go? If you can use pointers to structures in both Go and C, and you can access the fields of a structure through a pointer in both, then I don't understand why reading a structure field through a null pointer wouldn't cause the dereference of an address like 0x8 in both languages.
Unbounded/large offsets are the critical part. Minimum unit where memory protection can be set is one page (4096 bytes on x86), so compiler could reasonably assume that offsets 0-4095 are always safe to dereference (in the sense that SIGSEGV is guaranteed, which can be then turned into a NullPointerException in the SIGSEGV signal handler) without a NULL check. For anything larger or array accesses, add a explicit check for NULL before dereference.
Conspiracy: The NSA has compromised the runtimes of those languages to run arbitrary payloads and spy on everything that runs through them. I mean, "catch terrorists".
I’ve only used Java in the last ten years. I helped deal with the log4j incident at a few companies. We specifically had to patch systems that were running newer versions of Java and older versions of Spring. The exploit relied on a new method of adding code to the JVM at runtime that newer versions of Spring had locked down to prevent people from using.
I’ve never seen an explanation for why this mechanism was added or what it was supposed to enable — besides enabling new exploits.
Every time I’ve seen Java used for a safety critical application the justification has been entirely based on the fact that it has cryptographic libraries that are widely certified for safety by enterprises. The security people on our side were… resigned.
Perhaps, but reviewing code for memory safety issues is a more well defined task than general code generation so LLMs can be more easily trained to get better at it.
Encouraging. I hope the increasing drumbeat of awareness and advocacy around this reaches a critical mass soon. I'm not naive enough to think it will solve all security problems, but it will reduce the attack surface so, so much.
There's an important difference between an opaque "trust us" recommendation where it's broadly impossible to verify the claim (e.g., Dual_EC_DRBG), and one such as this which is fairly anodyne and merely intended to put more weight behind getting people to move forward in their choice of implementation languages.
The NSA's split offensive/defensive responsibility is bad but that doesn't affect recommendations such as this.
Sure. The problem is that externally one is left guessing which team something came from. I agree that this is a decent recommendation, but it would be much easier if those two responsibilities (offensive vs defensive) were two distinct agencies.
You can think of the NSA as being red team / blue team.
Red team wants to hack everyone with a negative value add for America, blue team wants Americans (mostly companies) to be as safe as possible from digital attacks.
What is the memory-safe subset of Java? Java is memory safe by definition. Sadly, the definition might not give people what they want, which is bug-free code.
Grew up on C++ and the paradigm (& loved it), but then moved to interpreted for use cases.
I think the uncoolness of them is due to OOP methodology... programmers today seem to want to be easier to get moving, but then spend more time debugging OR the debugging has been "placed onto" DevOps / use of Cheap fast hardware. IMHO
High speed fiance is mostly done in Java and Haskell, because mistakes are expensive. (And yeah, it's one of the very few fields that Haskell enters the list.)
I advise training programmers instead of throwing them in front of a screen without any training. Companies these days provides no training at all. When I was hired over 40 years ago, I spent plenty of time being trained for my first 3 months.
Now, nothing, and you if you want to train a new person, you do it on your own time.
> absolutely correct, in fact now some new hires are expected to do a few git-commit the first day(looking at you, Meta).
Meta have a phenomenally good training program, called Bootcamp for basically everyone in engineering, so they're probably not the example you're looking for.
I recall meta expects new hires to do first commit the first day, not sure how its training will help in this scenario, this is in no way saying meta has no good training, it just does not add up here though.
What kind of commit though? We expect a first commit on the first day. The commit adds you to the maintainers list in readme and grants you some new roles after merging. It is not a "solve this user's issue" kind of commit.
How is that possible? It takes over a week just to get a new employee configured. Laptop, IDE, permissions to the hundred needed things, etc. Let alone opening up an unfamiliar codebase.
Unless it is a “hello world” commit to confirm tooling is operational, I have strong doubts.
> How is that possible? It takes over a week just to get a new employee configured. Laptop, IDE, permissions to the hundred needed things, etc. Let alone opening up an unfamiliar codebase.
When I started at FB in 2013, I had all the laptop/permissions things done by end of the first day.
At the time, the presumption was that you'd push a really small change by the end of the week, but all of the tasks available tended to be pretty easy (note I wasn't in eng then, so all my info is second hand here).
Unlike 40 years ago these commits will be reviewed by an experienced colleague and automatically unit and integration tested, so there's no benefit anymore to training people separately before being allowed to touch the real codebase.
Back in the day people with SVN/CVS commit access would push directly to the main branch after testing locally, and testing would be done as part of preparing a release, so it was more necessary to have a "safety" period where new people could get used to the new codebase.
Startups are likely to fail, big tech is likely to fire the developer (or they quit), and poorly run companies just don't know how to train. End result is it just doesn't happen much anymore.
Employers complain they can't find people with the skills they need yet they refuse to provide training for those skills as it's cheaper to hire someone that was trained in the skills they need by another employer.
That doesn't seem like a contradiction at all. As you say, they want to hire trained workers rather than providing training themselves. Also, complaining is free while hiring good people is not...
Sure, but even the best programmers can write buffer overflows and other memory issues. Not that average shops have any interest in actually training people, though.
Best programmers tend to work on the most complex software though. There are small and relatively simple C/C++ programs around which have no know memory issues (but a person with minimal viable knowledge can introduce new bugs even there and that's where training would be useful).
Sure and for every small and simple application there’s are hundreds of terrible apps. I actually think memory safety is a pretty poor measure (in my isolated world I rarely see C code).
I see plenty of php logic which is far more likely to be publicly exposed and generally of a terrible quality.
"intentional training" (my term) is one of the most under utilized levers to pull for a team or organization to level up. Your observation aligns with my anecdotal experiences with teams, and it's sad to be honest.
The pushback I've received is something I'd categorize as "cart before the horse" - executives want to know exactly what will be achieved with such effort before approving any expenditure.
The teams I've been involved with - training has generally come organically and grass roots.
Training and technical security measures shouldn't be treated as alternatives. Even trained programmers produce bugs and security issues, if at least some of them can be prevented by an additional safety net, then it would be stupid not to use it.
or that this hype about the tech replacing programmers is not going to pan out. so maybe it is you who didn't pick up on something? to be able to provide commentary on a joke does not mean the joke was missed
I feel like each company/organization has such a unique DevOps toolset that all the "training" goes into figuring out which IDE you're allowed to use, which git client to install, what even are the processes/policies for installing stuff on your work computer, what's the workflow, etc etc etc. All the bandwidth is used on that and there's not much left over for the actual art of coding.
It's plausible that in many settings this is actually most efficient from the point of view of the company.
Do training on the things that can cause trouble longer term (inconsistent tooling taking over, security issues, company-specific things you cannot find online) and just hope people pick up the rest as they go. Good workers will usually manage. The people deciding how to train new workers maybe don't even know what kinds of "main" knowledge a given worker would need. Rather than assigning an expensive senior worker to training them, they might hope someone good willing will do it in their spare time.
I assume that includes leaders taking responsibility for the organization they are paid to lead. That includes investing in their coworkers' "strategic awesomeness" via training.
I think you do not appreciate that some people have jobs that don't give them adequate support to be awesome.
I just watched the most recent Last Week Tonight, where they went in depth on Freight Trains [1]. Specifically they talked about 3 mile long trains that currently only have two operators - one conductor and one engineer - and the industry is trying to cut it down to one.
That's just an analogy for software development. But I've definitely seen people be over-worked and under-supported.
Well 40 years ago stuff was absolutely laden with exploits... if you're talking about code written 40 years ago that we still use today? Most of that stuff is battle tested and came from teams that were highly disciplined to begin with - the good code floats.
My dad is in his 60s, started his company with some friends in the 90s. None of them got super rich, but they are all doing well, and all slowly going into pension. The company grew to a mid-size level, but actually most of the early employees stayed, and I think the average tenure is still probably > 10 years. Switching jobs every 3 years is weird, it is pretty much the time it takes to learn the ins and outs deeply enough to be an expert. There's some sense to it, as after learning all you can, it might be better for your career to search for new challenges. There's also the nonsense that switching that often generally yields significantly higher pay.
It's impossible to stay long with companies. The nascence of "shareholder value" as an overriding concept means that if you're far more likely to be laid off, or forced to job-hop to maintain purchasing power, than to be able to stay with one business for years and still have a successful career. By companies' own policies, it has become untenable for them to train. So it's all on them. They've got to fix the issue (before the government does it for them).
As someone who has worked in C/C++ on teams for a while. My personal opinion is that safe C++ programmers never use pointers - put everything on the stack and use (usually const) references.
But I've really got to ask - at that point do you really want to program in C++ anymore or is it just better to use a safer language that removes pointers entirely?
Edited for correctness - I was indeed talking about C++ and references are relevant only in C++. C absolutely has stack allocations which can make things a fair bit safer but if you're passing around collections it's usually done by pointer just to avoid copy actions.
And this still permits oodles of memory safety issues. You can have lifetime problems with stack-use-after-return. You can have bounds checking issues on stack allocated arrays. You can have data leakage through use of uninitialized memory.
And while there are some kinds of applications where never performing heap allocations is viable, this design is simply not an option for a huge number of applications.
It is basically impossible to write general purpose software like compilers, word processors, and layout engines without doing heap allocations. That means either pointers or references, which are difficult to distinguish if you do not engage in pointer arithmetic.
Any C++ program that does not do heap allocations either uses arrays as a substitute for the same thing or isn't a general purpose application.
There are categories of software for which C++ is still the clear best choice due to its superior expressiveness as a systems language. There are also other types of safety beyond memory safety that are better supported in C++ than other systems languages. As a practical matter I haven't seen a pointer issue in years, working from a C++17 foundation, so the expressiveness and other safety benefits are more valuable than a lack of a borrow checker or garbage collector.
I use safer languages when appropriate but there are cases where they would introduce more bugs than they address.
C++20 has very powerful compile-time metaprogramming and generics facilities. As a consequence, you can codegen complex systems of data structures and algorithms that are verified to be both optimal and safe at compile-time, essentially from a specification. The ability to automagically infer and codegen use-case specific types means that there are almost no naked primitive ‘int’ and similar types anywhere, the types are restricted to what they need to express and little more. While there is some ability to formally verify the correctness of these type interactions at compile-time, it is limited as a practical matter. The real strength is the ability verify the properties pretty exhaustively at runtime, given sufficiently good testing.
If you implement something like a database kernel almost entirely in user space, which is what you want to do if you care about performance, you have a new set of problems that languages like Rust were not designed to solve. Ownership, mutability, and lifetimes are not resolvable for most objects at compile-time, they can only be resolved dynamically at runtime. With C++, you can make these look like ordinary objects with ordinary behaviors, with metaprogramming abstractions under the hood arbitrating the chaos. Things that look unsafe are actually guaranteed safe by the underlying abstractions. In this type of software, almost your entire address space is “unsafe” in some way, so it is enormously useful to be able to overlay a safety model that does not rely on the compiler, since the compiler cannot solve these cases.
I use both C++ and Rust in production environments. There are cases where C++ used well is legitimately safer, theoretically and empirically, which is why I still use C++ for some things. Rust may become stronger at dealing with these cases but it still has many years to go. (Also, Rust async story is a hot mess, and you really need async for scalable, performant systems software.)
"In attempts to mitigate the dangers of memory unsafe code in C and C++,
many software manufacturers invest in training programs for their developers. Many of
these training programs include tactics designed to reduce the prevalence of memory
unsafe vulnerabilities produced by those languages. Additionally, there are numerous
commercial and industry trade association training programs. Further, various organizations
and universities offer trainings and a professional certificate for demonstrating knowledge of
secure coding practices in C and C++.
While training can reduce the number of vulnerabilities a coder might introduce, given how
pervasive memory safety defects are, it is almost inevitable that memory safety
vulnerabilities will still occur. Even the most experienced developers write bugs that can
introduce significant vulnerabilities. Training should be a bridge while an organization
implements more robust technical controls, such as memory safe languages."
I can see why that would sometimes work, but the better preconfigured vim installations have a long-press of the power button remapped to a 911 call, in case the user has fallen unconscious onto the keyboard and needs medical attention.
>"In attempts to mitigate the dangers of memory unsafe code in C and C++, many software manufacturers invest in training programs for their developers
I have seen that too over the years and even been to a couple when I went to other companies, 99% of the time these classes are worthless and consists of marketing drivel. Half the class contains statements like "You can do it the hard way, or the easy way if you purchase ....".
One instructor even stated "You do not need to worry about running out of memory, allocate as much as you want". That class was 20+ years ago. After that I told my manager I will not go to any more vendor classes.
The training I am referring to is by experienced peers and goes on for many months where your work is reviewed in detail, 1 line at a time with you there. That use to happen decades ago, now no more.
Depends on the company these days. In my initial role, I was given a couple days to "read over the codebase" and was then expected to start producing results. Most of my peers had weeks or months-worth if training before they started contributing.
If (security) training worked, we wouldn’t have these issues.
Even in highly regulated, controlled environments memory safety is still an issue. I have seen bugs in these environments where I know for a fact that the developer has had 40-80 hours of training on memory safety and they still pop up.
If these people have occasional issues, the rest of the world has no effin chance.
It doesn't take long to find comments on HN about how there are plenty of people out there that can do a good interview but can't code their way out of a paper bag. The standard solution to this, and I've been on the receiving end of this, is to have someone as a contractor for 3+ months before converting them to full time employee.
I'm afraid your approach would mean hiring someone and spending time and money on them before you're sure they'll work out in your organization. Few want to train contractors and firing FTEs after 3 months and change is a dim prospect as well.
> Memory safety vulnerabilities are coding errors affecting software’s memory management code in which memory can be accessed, written, allocated, or deallocated in unintended ways.
How can you do any of this without software running in the local machine?
A specially crafted networking package or file can cause a vulnerable application to get confused enough to interpret input meant to represent data as binary code to be executed. If you have an application that doesn't interact with any inputs, yes, you can avoid a lot of exploits. If the vulnerable application runs in a server, you might be temporarily safe in your local machine, until that server starts being used as a delivery mechanism against all its clients.
To take a very simple example (which won't work on modern OS with ASLR):
1. You have a service that takes some user input
2. The service allocates a buffer on the stack for 20 bytes (the max input allowed)
3. Service doesn't actually validate that the user input is < 20 bytes
4. It writes the input into the buffer, but keeps writing past the end of the buffer.
5. It eventually overwrites the return pointer so now the user input can control where the execution jumps to on return
6. In the user input, the malicious user includes some shell code to do any arbitrary thing they want.
7. The overwrite the return pointer to jump back into the shell code and now they are executing arbitrary code in the server process.
This sort of thing was embarrassingly easy to do on old linux kernels before ASLR was implemented. Now it is dramatically harder because there are all sorts of countermeasures in place to prevent it (ASLR, stack canaries, executable vs non-executable memory, etc).
"Undefined behavior" in general is a nightmare. After memory safety, the next target should be the enforcement of underflow/overflow trapping. With the exception of the intentional use to implement modular arithmetic, underflow/overflow should always be an error condition.
Not everything is a server facing malicious actors. Forcing underflow/overflow trapping would absolutely harm optimisation and performance for use-cases where security is a non-issue.
I want to agree, but I’ve seen too many cases where code written for a narrowly scoped purpose finds its way into a totally different environment where the original assumptions no longer hold. Maybe it’s worth a tax on constrained use cases in the name of having fewer time bombs out there?
The main problem with signed overflow being UB is that it's an opportunity for compilers to blatantly do adversarial language lawyering against the programmer in order to enable optimisations.
When it comes to unsigneds, there's no such problem (and this is in fact the real reason why anything that can be unsigned in C/C++ should be unsigned).
Yeah. Back when machine architectures were churning so fast that compiler flexibility allowed portability that otherwise wouldn't have happened, UB freedom was a net good. Now that those arbitrary architecture choices have stabilized, the good is no more, and the evil of optimization rules lawyering has replaced it. UB needs to go.
>> the next target should be the enforcement of underflow/overflow trapping
Trapping is nonsense - in production anyway. You might use it in development to catch errors. But in production there is no way to "fix" an overflow if it happens and is detected, so you'd be looking to crash on trap which in some code is preferable to silent data corruption.
I do lots of fixed-point math for embedded motor control. Representing rotor angle as a signed 16bit number is deeply engrained in me these days because taking differences to get a signed delta just works, and I never have to worry about rollover because it behaves exactly the way I want. ;-) While this is technically undefined behavior, I've never run across a compiler that worked differently.
most consumer operating systems are inherently non-deterministic which I put right in there next to undefined behavior (some people say it is a different thing).
It is a different thing. Non determinism is different to undefined behaviour.
Undefined behaviour can cause miscompiles which can break the expected logic of a program. In the worse case, maybe the compiler optimises away your password validation because it has decided that the failure branch is undefined behaviour.
Non determinism can lead to logic bugs, but it can't magically introduce new logic into the program like UB can as part of optimising
Rust is already most of the way there. The default math operators still do implicit underflow and overflow, but the behavior is actually well defined: debug builds panic on overflow and release builds wrap on overflow. There is no way to get a C-style "demons fly out my nose" overflow.
There's also explicit arithmetic functions that let you choose your overflow behavior:
* Checked: return None on overflow
* Wrapping: two's compliment wrapped overflow (the "overflowing" variant gives you a carry bit)
Do you mean the C compiler's optimization passes can do strange things to code paths that overflow? The last I checked, C would wrap on overflow too. Rust uses the same compiler backend as C, so it could very well have the same behavior emerge from compiler optimization passes.
> it could very well have the same behavior emerge from compiler optimization passes.
It can not, because "compiler optimization passes" is not the root cause of this behavior: it is that it is undefined behavior in the language itself. This is what gives the compiler license to make those transformations. In Rust, it is not undefined behavior, and therefore the compiler does not have the right to make those transformations for Rust code.
Just because Rust uses LLVM does not mean that suddenly it inherits C's semantics. This has actually happened (well, more specifically, C++'s semantics, C was accidentally inheriting them as well IIRC) at least one time before, but that is a bug in LLVM that was fixed. Now each language has the proper semantics here, and it all works just fine. (I am referring to the behavior with regards to infinite loops with no side effects.)
Having it as default on todays hardware is a non-starter. Few would pay say a 200% perf penalty for arithmetic unless it's on a security critical path. So opt-in it is. And almost all languages already support that type of opt in at the global level or smaller scope.
You can have underflow/overflow trapping today if you want it in C++. Most people don't use it, at least not in release builds, because of the performance cost.
Even with `-ftrapv`, evaluating abs8(-128) will produce -128, because 127 is the maximum value for a signed 8-bit integer (so trying to get 128 wraps around to -128), but this isn't caught. However, both Rust and Ada do catch this. This article highlights this difference between C and Ada:
I wasn’t necessarily assuming ‘-ftrapv’. For reliable software in C++ it is pretty common to have alternative integer type implementations that provide different fully defined behaviors and guarantees than the default integer types. This became legitimately transparent around C++17 IIRC. It is more or less drop-in and mostly produces optimal codegen. Also a good way to eliminate irritating integer behaviors inherited from C.
It is easy enough to crank out custom integer types in C++ these days which are arbitrarily safe that there isn’t much excuse for not doing it, particularly since generics and metaprogramming does most of the work. Outside of interfacing with syscalls, there isn’t much use for C primitives beyond size_t.
> For reliable software in C++ it is pretty common to have alternative integer type implementations that provide different fully defined behaviors and guarantees than the default integer types. This became legitimately transparent around C++17 IIRC. It is more or less drop-in and mostly produces optimal codegen. Also a good way to eliminate irritating integer behaviors inherited from C.
I didn't know this. I can imagine you can create your own integer types as wrappers over existing primitives, and define them so that, e.g., on signed overflow, the program aborts or wraps (and your signed integers are actually a wrapper class over unsigned integers, but with operator overloading so that they act like signed integers). Is that what you mean? If not, could you show me what you mean?
Yes, this is one of the advantages of pervasive operator overloading in C++. You can create types that behave like primitive integral types for all practical purposes but which implement alternative behaviors like trapping overflow, blocking integer promotion, defining shift operators greater than or equal to the bit width, creating alternative casting rules, etc. However, it has not always been possible to implement this transparently in C++.
This did not work well in older versions of C++ because the limited type inference meant that many common cases around different type interactions required explicit handling that made it clear you were not using native types. Around C++17 it became possible to define integer types that were almost entirely indistinguishable from the native types in ordinary C++ code.
C++17 was important because it meant a lot of safety in the code base could become automagic via the type system, especially for primitive types. The code looks the same whether you are using it or not. That was a huge capability change. C++20 then generalized it to arbitrarily complex types. It is difficult to overstate how much this improves the conciseness and safety of non-trivial code.
I think the question we have to ask is why it has such a performance cost, and how we can circumvent it.
There are likely multiple necessary changes for it to be fast. One part is implementing the check quickly, this could be hardware assisted. Another is addressing all the optimizations the compiler does by asuming that there is no overflow, and figuring out alternative ways to get the compiler to emit fast code.
Confused about Python in particular considering a lot of powerful and common dependencies in the ecosystem starting out from numpy and friends all have C/++ components to them for performance improvements. Surely this is a vector to be considered?
Python itself is memory-safe, but you're right that a good chunk of the packaging ecosystem uses memory unsafe languages for performance and interop.
The Python Software Foundation noted this in our response to the US Government RFI on open source security. There are efforts to make using Rust easier as a systems language for Python packages and some security-critical packages like 'cryptography' have migrated to Rust.
That's the irony in all of this. The Fort is sprinkling their wisdom down upon the masses and proclaiming that C++ is unsafe while arguing for a language that's a wrapper for c++ developers...Reminds me of John Gall's book about systems "The system itself does not do what it says it is doing."
If you have no users then go for it. However if you do then you are doing those users a disservice by exposing them to unnecessary risk.
Maybe the risk is mitigated in other ways. Your software runs as a cli and not a service. It doesn't process outside input. It is run in an ironclad sandbox.
But honestly, if you think the way C++ with RAII want's you to then you should already be following the rules that Rust want's you to follow for the most part.
What I'm saying is that if you are sufficiently smart then you could concievably write perfectly safe C++. But since almost no one in practice has ever been able to be sufficiently smart consistently enough to ship C or C++ code that doesn't have memory safety issues, I don't trust anyone to do so and would rather people stop writing stuff in them when there are better languages to use where I don't have to trust you as much.
RAII is great (especially as a developer’s aid), but I don’t think it’s had a huge effect on memory unsafety: it’s still very easy to UAF in a RAII-only codebase, thanks to all of the flexibility C++ allows in constructors and destructors (and unintuitive behavior in moved-out-of objects).
Rust is generally perceived as having many of the same developer kindnesses as C++, while having same-class performance. Learning it has a bit of a curve, but I found that it eventually clicked (I have a C and C++ professional background).
RAII is great, but unfortunately it breaks down when the resource it manages leaks out of the RAII type.
E.g. Whenever handing out the raw pointer stored in a unique_ptr, the code receiving that raw pointer can delete it, which will very likely lead to a use after free and certainly to a double free issue.
I used to work for this agency and your description is the most apt way to characterize the dichotomy between SID (SIGINT) and IAD (information assurance) I can think of.
Bounds checking will sure solve some of the problems, but it will not solve all of them, perhaps not even most of them. You'd still got: UAF, iterator invalidation, etc.. And not even all code can be bounds checked, you need to know the length for that (which means using new types such as std::span or having to annotate your functions with a compiler attribute, to be able to also bound-check pointers).
Bounds checking is also frustratingly expensive because of C++'s horrible aliasing rules meaning that compilers struggle to soundly eliminate redundant bounds checks.
Can someone explain why we can't double-down on C++ and, through compiler wizardry and reduction in toolset (say, strings can only be fixed-size at 32 chars, 64, or 256 long. No raw pointers, allocator zeroes out all freed memory), achieve a memory-safe language?
Obviously, making certain concessions would be a deal-breaker for some, but it might be viable for legacy codebases. If you were to try to make C++ memory-safe, where does it begin to break down?
I think this is, realistically, the easiest way to solve the problem. That, or a totally different language that is (somehow) a drop-in replacement for c/c++.
I'm in no way an expert in any aspect that relates to this problem, but it's reasonable to believe that it would be but a mildly challenging task if companies like Google, meta, Microsoft and the likes joined forces.
A year of concentrated efforts might be sufficient to rid us with this problem once and for all.
Btw I don’t know how this works in most languages, but circular references can be still a problem even with these restrictions. (If I remember well dangling references can be more or less handled, but correct me if I’m wrong. I remember that JavaScript definitely had problem with those)
I was only tossing it out there as a token example. You can manipulate the heap in a minor way if you continually request and free memory in irregular ways (called mem fragmentation). Fixing the size (might) be one step towards hardening the system against OOM attacks. overall, I'm just wondering why we can't make C++ safe by starting at the lowest layers, mem-proofing them, and working up from there (and limiting/discarding some features along the way if necessary).
"Ban a ton of widely used C++ features" creates a problem that is smaller than but still comparable to "rewrite it in rust." You also still have some features of modern defensive C++ that can still be abused and cannot be removed from the language or made safe without something resembling the Rust borrow checker.
Getting these features in the C++ language itself will be absolutely impossible, both because of massive performance implications and well as nightmarishly large ABI breaks on everything.
There are various efforts to do what you are describing. I think that they are often a good idea, but they absolutely will not solve the problem conclusively.
I assume it's because they view private industry as a US asset, and don't want Russia or China hacking them (to steal IP, or gather information, etc.).
Promoting cyber security in the civilian workforce is also part of the DoD's 2023 "Cyber Strategy".
The Department will take action to foster a culture of cybersecurity and cyber awareness. We will establish an expectation that senior military and civilian leaders possess a baseline fluency in cybersecurity issues. The Department will develop, fund, and implement technical curricula across different levels of professional military and civilian education, emphasizing General Officer and Senior Executive Service leadership courses.
The framework they're working with here isn't about specific recommendations like this. It is about non-memory safe languages, mitigations you can try to add safety, and a framework for moving towards memory safe languages over time. It's at a higher level of abstraction than "specific language with specific technique."
In the PDF the closest things they talk about to sanitizers are SAST/DAST tooling, which isn't exactly the same thing, and compiler mitigations, which aren't exactly the same thing, but I also don't believe this is intended to be fully comprehensive at this level of detail.
How about a new drop-in replacement language for c/c++?
I'm in no way an expert in any aspect that relates to this problem, but it's reasonable to believe that it would be but a mildly challenging task if companies like Google, meta, Microsoft and the likes joined forces.
A year of concentrated efforts might be sufficient to rid us with this problem once and for all. Billions of lines of codes would gain (some) safety instantly.
That's basically how Rust is being treated by those companies.
But it's not going to be "drop-in", lots of stuff will have to be refactored to work with Rust's memory model.
See, memory management is a hard problem and the complexity involved in solving it has to live somewhere.
In C and C++, that complexity lives in the programmer's head.
In Java, Python, Go, etc. that complexity lives in the program's runtime in the form of garbage collection.
In Rust, that complexity lives in the compiler - with some caveats, Rust code is safe as long as it compiles and passes the borrow checker.
However, the ownership-based memory model of Rust is not always trivial to port other code to. Modern, idiomatic C++ written with RAII and smart pointers should be relatively easy to port to Rust, but most of the critical systems code out there is not like that at all. Most of the critical systems code out there is written in C or old-style C++ with C-style strings & arrays and raw pointers to heap memory all over the place.
So regardless of what your C/C++ language replacement looks like, it will take some serious effort to replace that kind of code.
Fully agree. The best time to stop giving the next generation of coders the gift of unsafe memory is today. C and C++ should be known as 'legacy' languages.
It is generally speaking difficult to make an efficient implementation of the compiler and/or the virtual machine for many memory safe languages without writing it in a more efficient, statically compiled language like C, C++, or Rust. And that is to say nothing of software like operating system kernels and browser engines. So perhaps Rust will gradually take over the world there.
Simplified, this means it's generally difficult to safely make things go fast. And there's nothing wrong with that. The sooner we realize it the better.
Until I see professionals get funding to build, maintain, document, and create training materials for Rust bindings, I'm going to continue to assume there will not be movement in that sector. I don't think industry is going to throw away the C++ ecosystem and build greenfield projects on a foundation of hobby projects.
The industry did not abandon C and C++ for Ada in the past, likely it will do the same to Rust. Not to mention C and C++ are both evolving and their toolchains are getting much better nowadays.
After using Rust for a while, I actually decided to stay with to c/c++ for the rest of my career.
I’m not op of course, but from a career perspective you have to make choices based on what will give you the most opportunities in the future.
Right now Rust is technically safer language but it doesn’t have the same amount of job postings and career opportunities as C++. That’s just an inertia problem.
The conundrum is if you jump in early and risk the language becoming yet another language or do you wait and see.
I haven’t coded C++ in 4 years, but my understanding is that the language is evolving to answer the threat Rust poses to it.
Only because there weren't affordable compilers, everyone was cloning UNIX, and there weren't goverment mandates for the industry at large.
There are still COBOL and Fortran developers, C and C++ won't go away tomorrow, even if the goverment says so, but it will be harder to use them in security clearance contexts, similar to hazardous goods.
Importantly, if you look at lists of "C/C++" security holes, you find it is almost all C security holes. So when they write "C/C++" they are hoping you won't notice they are lying to you.
The residue of holes in C++ code is about the same as JS, Python, etc. So just eliminating C and C-like constructs gets you to the same level of security as the other languages.
The way you have phrased this is very disingenuous.
> Importantly, if you look at lists of "C/C++" security holes, you find it is almost all C security holes. So when they write "C/C++" they are hoping you won't notice they are lying to you.
As these "C-isms" are part of the base language, and you cant 'opt out', this means the base language is still included in the lists. Most C++ applications still linked (and used symbols from) the system libc runtime.
I do not write C++ with any regularity, because I like my C footguns and dont want additional layers of footguns in my code. last time I tried writing C++, it allowed you to use after free, access raw memory pointers with no validation, iterate off the end of arrays and integer overflow and underflow, all without using "C" style constructs.
Maybe it has changed, maybe C++ is now not as insane as it used to be and smart auto magic pointers solve everything. I'd love to read about how modern C++ solves these issues, do you have any resources ?
modern-ish C++ has some tools that can make it easier, but fundamentally it's still unsafe, it's still proven pretty difficult to write a large C++ codebase without memory safety issues. (that said I would still personally prefer C++ to C, though the recent updates to the standard contain at best very small improvements from my point of view)
You are more optimistic than me. I would guess that if Khronos does not support Rust by that point, anyone in industry who is affected will adopt Carbon or whatever other path of least resistance brings them into compliance, and whatever that option is will take off.
426 comments
[ 4.2 ms ] story [ 309 ms ] threadI have no idea how this is going to work at the OS level.
I struggle to understand the comment. Do these two sentences relate to one another or is the second one a non-sequitur?
There will always need to be unsafe ops in highest performance, low level coding.
But in the vast majority of use cases, there only need to be a few.
So huge security win being able to isolate those vs "somewhere in the entire codebase."
I do wonder what this landscape could have looked like if all of the effort had gone behind a Mach-based microkernel. I read all of the back and forth between Tanenbaum and Torvalds. At the time, I was just so excited to see "free and open" winning.
I should specify that the XNU Kernel from Darwin/OSX took a lot of my attention when it first premiered.
Had the servers been running microkernels instead, updating services without full restarts would have been trivial in almost all cases.
It's just a better approach overall. I curse the path dependency that led to the current situation.
[0]: https://github.com/search?q=repo%3Atorvalds%2Flinux++languag...
https://en.m.wikipedia.org/wiki/Stuxnet
C#, Go, Java, Python, Rust & Swift
Even that mandate did not appear to fundamentally change the landscape beyond government work.
If you have some point to make please do so earnestly. Layering in levels of irony makes any point you're trying to make difficult to understand or follow, even if labelled with /s.
This is going off-topic, but there is a style of internet arguing that I have come to seriously dislike. It is one where instead of someone making a point, they make the point they wish to detract and simply flag that they are being ironic. In doing so they don't actually advance the point they're trying to make, they assume that the audience already understands and is sympathetic to that point, so they simply put up a target to scoff at.
I'm not sure if that's what you're doing here, or if you're just struggling to make a point about worrying about government intrusion into private business.
What is a quite real possibility. For example, there are plenty of places out there that can't stop expiring passwords every 1 or 3 months because it's in one of those lists. But I do agree that complaining about the recommendation because of this is completely out of topic, the focus should be on the rule that actually mandates it.
If you don't know, Consumer Reports is paid by groups interested in encouraging the government to apply regulations to certain areas. A bike helmet manufacturer may pay them to create a report, host events, and otherwise lobby on their behalf to e.g. create regulations about people needing to use bike helmets.
It is my understanding that many Rust advocates, security researchers, and members of the Internet Society are effectively advocating/lobbying for partial government mandates of 'memory-safe languages'[0]:
> It’s not yet possible for government procurement to only buy memory-safe software. For example, you can’t say routers must be memory-safe top to bottom because no such products currently exist. But it may be possible for the government to say that newly developed custom components have to be memory-safe to slowly shift the industry forward.
> This would require some type of central coordination and trust in that system. The government could ask for a memory safety road map as part of procurement. The map would explain how the companies plan to eliminate memory-unsafe code in their products over time. The carrot approach for memory safety may include not just decreased future costs in cybersecurity, but also reliability and efficiency.
[0] https://advocacy.consumerreports.org/wp-content/uploads/2023...
I am willing to bet that most of those “Rust advocates” are programmers who code in Rust but I’m fine with not calling them that. I agree that good programmers should be able to work in different languages, operating systems, countries.
For example, if this had already happened we may find today that Java is certified for use but that Rust is simply not allowed, while maybe Swift is because of Apple's backing of it.
Worked out well enough with Ada in 1978. https://en.m.wikipedia.org/wiki/Ada_(programming_language)#H...
Seriously, I think the time has long since passed software needs regulating. It's a major part of modern society, and as far as I'm aware, most people aren't opposed to building standards in principle.
The regulations are to ensure the 10% of bad builders/developers don't ruin peoples lives.
Regulation hinders progress and makes things more expensive. But no regulation raises costs and hinders progress too: it basically creates a situation of very low trust, and low trust is extremely expensive for customers/buyers who are not domain experts. This also makes them overcautious and conservative. One needs a balance and lots of nuance to make a reasonably well functioning market/system.
The case against regulation on software business is not that "regulation is bad". It's that programming and software is very new and rapidly evolving area of human activity. It's not nearly as well understood as building houses. Written and unwritten standards and best practices are constantly changing. The field is subject to very strong fashion-driven "crazes".
(Just look at how many new languages are still being created. Most don't become as widespread as C++ or Java or Python, but many do find their niche, and very many are in use to some extent. This indicates that "language Holy Graal" is nowhere to be seen yet.)
In software, there is very little consensus between domain experts on most issues. This is very unlike construction and house building, where some new materials ant techniques are introduced too, but the basic principles are well understood, calculable, and where agreement among experts is usually quite achievable.
So arguably it's nearly impossible to create good regulation at this stage, at least outside of certain special niches. This is very different from building codes and stuff.
Of course there is also bad regulation. The bureaucracy likes to expand their control indefinitely, wants to regulate things that should not be regulated, thus (if unchecked) creating very bad regulation. Well, that's the case for checks and balances, for the society to fight back. But "this regulation is bad and needs to be changed" is a much more mature position than "we need no f*g regulation!", in my opinion...
Building a building without regulation is easy. Building a city that doesn't burn to the ground every time someone knocks over an oil lamp is not.
The same can be applied to software. An ability to cobble together a "Hello World" does not necessarily mean I want you programming a controls system on a nuclear power plant.
Complete safety, or actual freedom. Pick one. You can't have both.
Or, maybe pick a point on a spectrum.
That seems more realistic, because I'd argue you can't really have "Complete Safety" or "Actual Freedom" (by which I am guessing some people would interpret 'actual' as 'complete').
False dichotomies aren't helpful and underscore lazy thinking. The real world is full of nuance, and so should our policies. Using a risk-based approach is probably more appropriate than an all-or-nothing policy.
If we want a safer internet, make them carry insurance against data breaches and fine them a fixed amount, say $1500, for each identity they leak, paid immediately upon proof of pwnage.
Time and time again, we see experts make recommendations, then legislation and rules make it mandatory.
A burning example is most of NIST special publications. NIST makes no rules, mandates and such. Yet, mandates (e.g.,DFARS, DEAR) point to the recommendation as the requirement.
Right now there are two of these playing out in the cybersecurity field - zero trust and passwordless authentication.
So those who down this comment, you are right, it is not a mandate. Those who up this comment, you are right, it is likely to become a mandate.
Though at this point, this is a recommendation, not a mandate.
My impression of historical Python is that is an old, partially arcane language that due to D.S/AI is now popular; I would initially think it would be no better than those other interpreted. Python does these things well... mainly due to pandas/dataframes/polars.
<btw, Perl? ;) >
They should've instead just said Python and languages alike?
None of the alternatives except Go and Swift is new. Ruby has turned into a robust boring technology, it does not mean it’s unworthy.
The Ruby team is still pushing the languages features and speeding up the runtime. Take a look at Ruby Ractors for example.
I do have to agree on that Ruby is more of a niche language now thanks to Rails but I’d say Ruby works very well for scripting and gluing together different systems.
If it weren't for the behemoth of legacy code we'd really have this problem more-or-less licked. Unfortunately, that behemoth is still rampaging across the landscape.
"Rewrite it in Rust" gets a bit of pushback, perhaps even justified, but at this point in time I'll take anything that just reduces that behemoth in size. The journey of a thousand miles begins with a single step, an elephant is eaten one bite at a time, etc. Rust is just one of the easier and more effective options for a legacy codebase, with the unusual advantage of being able to slip in incrementally. Almost every other language requires a true rewrite.
Ada, Fortran, assembly?
http://www.ada-auth.org/standards/rm12_w_tc1/html/RM-13-9-1....
https://old.reddit.com/r/ada/comments/mme3jk/is_ada_memory_s...
In other words, if you go out of your way to use unsafe features, and don't use the features that compensate, Ada is memory unsafe. This has become the goto dismissal of Ada, apparently more popular than "eww...a BEGIN..END language" and "designed by committee/government tainted".
No pointer direct arithmetic, though. If you allocate the memory via allocate, you can inquire if it's been deallocated.
https://www.adacore.com/about-spark
Hasn't it always been? I'm no expert but always assumed that it was used basically military and avionics, and perhaps other safety critical equipment.
Assembly is in use, yes, but in 2023 I feel there is generally an understanding of the risks and I haven't seen the "write everything in assembly" crew in about 15 years. The problem is that there's still too many programmers blithely using C and C++ without realizing the risks and thinking they can cowboy through the problems. For every line of vulnerable, dangerous assembly I bet there's thousands of lines of C or C++.
There is also the problem that there have been some big bugs that got through even static analysis and fuzz testing, but I'd still be at least reasonably satisfied if all the critical software in C and C++ would be supported by those tools. Interpreters and compilers have had non-zero error rates too.
At the scale I'm talking about, Ada is a non-entity as well. It isn't used. "But it is! I'm a professional Ada programmer!" says someone reading this to themselves. In which case I would say, you darned well know what I mean and don't pretend otherwise just to try to score useless internet points. Ada is not a relevant force on the programming world. That may be sad, but it's true.
Duplicating the DO-178B certification that it has obtained in an endorsed language will be an incredible burden for any who attempt it.
(My reading is that the GP points to the exemplary achievement that SQLite has reached a close to (security) bugfree, at what I consider a nearly superhuman effort)
Plus everything that needs to directly interface with the above languages. So many Python libraries that are one "funny integer" away from a nightmare debugging session.
C++ at least has tools to make life significantly more safe. I can write a buffer overflow in any language, and on the scale of difficulty, ASM-C-C++-Rust-Python covers my experience (from easiest to fuck up to hardest).
Yet nobody is calling for us to rewrite everything in python. Why is the line drawn at Rust? It's perfectly simple to trash memory in Rust.
C++ is unsafe by default.
Of course it’s just as easy to write bugs in unsafe Rust as it is in C++ (actually, it’s probably even easier), but defaults matter.
At that point, if we have to re-wrap everything in rust to hide the unsafety of the interfaces to the system (sockets, shared mem, etc etc), then why not just write safe cpp wrappers?
Yes, people are writing memory overflows in their own code, but I'd argue 99% of the critical security bugs are actually in the unsafe interfaces. And we don't really need a new language to fix that. We just need new interfaces.
I love Rust, but using it for anything nontrivial makes the "safe" patina really fade. You're quickly writing what feels like C, with MaybeUninit<X> all over.
I'd contend that using Rust for anything nontrivial results in MaybeUninit & co being common.
It’s quite rare to have to make syscalls directly in Rust, just like it is in c++. Most code in any large enough system is related to the internal logic of the system, not to its interface with the outside world. And when you _do_ need to interface with the outside world, you can use a wrapper (lots of the standard library is basically wrappers around syscalls; this is true in any language). And no, in Rust unsafety doesn’t typically “leak through” interfaces, unless those interfaces are buggy.
> why not just write safe cpp wrappers?
There’s no such thing. It’s not possible to write a safe interface to c++ code in the sense that that term is used by the Rust community. In Rust, “safe interface” means: assuming there are no bugs in the underlying code, and the client code never invokes `unsafe`, using the interface cannot cause undefined behavior. This is impossible to guarantee in c++.
> I love Rust, but using it for anything nontrivial makes the "safe" patina really fade. You're quickly writing what feels like C, with MaybeUninit<X> all over.
This is not true at all in my experience. I work on Materialize, surely one of the more non-trivial Rust programs that exists. We use very little unsafe/MaybeUninit/C-like code. Do you have an example of a codebase you’re thinking of that does this?
To answer your question, I'm referring to much of the networking code in socket2 / socket, which uses MaybeUninit when doing non-standard stuff like forming your own packets. (RAW)
I'm not too familiar with `socket2` but normally in Rust to construct a buffer with arbitrary bytes in safe code you would first zero it out and then write it. Using `MaybeUninit` there is presumably just a micro-optimization to avoid having to memset things to zero.
Because that's what "safe" in Rust means. No memory safety errors, no undefined behaviour.
Not in safe Rust.
As I said, you won't write a buffer overflow in rust, but unpacking why can be interesting and it doesn't end at "bounds checks".
Rust (and Swift) are viable languages for solving most problems that people usually reach for C or C++ to solve today and both make it considerably more difficult, by default, to introduce the most common class of serious security vulnerabilities in the modern world.
In C++ vec[999] is a buffer overflow and you can index any pointer even if it isn't supposed to be an array. There are so many easy mistakes that can be made and aren't obvious to a reviewer. Maybe with a very strong linter you can consider C++ very distinct from C, but by default I don't think it is that different.
> I can write a buffer overflow in any language
Try doing it in JavaScript? If so the Mozilla security team would appreciate a private disclosure. Of course it is possible in any non-sandboxed Turing complete language, but there is a huge difference between the default accessor of the most used container type allowing it vs needing to use functions in the `sun.misc.Unsafe` package or wrapping your code in an `unsafe` block. Making code that may cause a buffer overflow explicit is a night and day difference. It means that you can't do it via a typo in the vast majority of your code, and it will grab the attention of your reviewer very quickly. Isolating the part of the code that can cause buffer overflows to a small part greatly raises the attention that is given to those areas, and greatly reduces the chance of them occurring.
I don't think that Java or Rust prevent all buffer overflows, but I also don't think that it is possible to write C or C++ without them. Sure, it is possible to be careful and avoid most of the buffer overflows most of the time, but we and our reviewers are just human so we will never prevent all of the buffer overflows all of the time.
I don't think that this recommendation is under the impression that "memory safe languages" will prevent all buffer overflows, but the idea is that they will greatly reduce the number. In many situations, I would guess the majority of them, this is a good tradeoff.
Could you elaborate on this? Rust doesn't have a runtime (beyond what C has), and am having trouble understanding what you meant to say about stdlibs.
I don't think I've ever seen anyone reference "C with bounds checks enabled" as "having a runtime". Does having stack probes also imply having a runtime? I guess I'd be less surprised if it had been worded as "some mitigations/features have a runtime cost".
> If you want features like async (standard in many language runtimes) you're also going to have to pull in some kind of external runtime dependency.
Yes, you can add a runtime to your application (if you need to use async/await). It has an additional cost over not doing that, but the "promise" is that it is "zero (additional) cost (over what you'd end up with if you wrote the functionality by hand)".
The runtime isn't all that large but every OS has one. On UNIX it's spread over libc, libpthread, libgcc, libm and so on.
On Linux stack probes usually have some support code in libgcc and/or glibc, if I recall correctly.
So if it's idiosyncratic really depends on what audience you're talking to.
It may feel like C doesn't have a runtime if you're only familiar with UNIX, because the C runtime is guaranteed to come with the OS there whereas other language runtimes are optional and thus more visible. But every language has a runtime.
I always thought that Rust was the only memory-safe language that doesn't need a runtime (beyond the libc that every language links to when running on Unix-like OSes). Maybe you could define what you mean by runtime.
https://vercel.com/blog/turborepo-migration-go-rust
Will somebody please tell me why everyone seems obsessed with optimizing for programmers going from zero to minimally productive?
I have been using Ruby for twenty years, Rust for eight, golang for nine, and C for twenty-six. Most programmers will use a language for dramatically longer than a year, so why are the first three months such a singular point of focus?
The code I wrote in the first three months of using every one of these languages was bug-ridden, unidiomatic, unnecessarily difficult to maintain, and generally terrible. Ironically, Ruby was probably the least bad in this regard. Go and Rust were probably about the same, but I’d frankly give Rust the slight edge here. C was the inarguably the worst, but it was also my first language.
Subjectively and retroactively comparing things a year in, I’d wager my Rust was of the best quality (readability, ease of maintenance, speed of development, bugs per “unit of functionality”), followed by Go, Ruby, and then C. At five years, the quality of my Rust code blows everything else out of the water. My C was still terrible (partly because it was C, partly because it was still my first language). But I’d say Ruby edged out Go at this point for me.
Obviously this is not only anecdata but wildly guesstimated looking back and comparing learning curves on languages at completely different points in my experience as a programmer. I’ll happily admit that Rust pulling so far ahead so quickly is as likely to do with it building off the knowledge of prior decades of professional software engineering. And that my personal experience with any of these languages is of course unique to me and my circumstances.
But it just seems wild to me that people seem to focus on “getting a new person up to speed as fast as possible” to the exclusion of apparently everything else.
Go being easy to pick up and learn is certainly a virtuous cycle insofar as it helps bootstrap a large community. And that’s absolutely happened!
But that is—in my mind—more of an explanation for why Go has become so popular so quickly more than it is a compelling argument for the language itself. Haskell having conversion friction might explain its lack of adoption, and that’s certainly a great argument in a discussion about why or why not to adopt it for yourself or your team! But it seems like an overvalued axis on which people seem to evaluate languages on their own.
As a counterexample: PHP classically had a reputation as being a language that was very easy for beginners to pick up. And it’s even memory safe! But it also had a reputation for having poor long-term prospects for projects written it as well as being a limiting factor in the growth of engineers using it (note: I make no claims as to the fairness of this reputation, nor to its applicability on “modern” PHP).
PHP is arguably even easier to learn than Go. So why is it that virtually nobody jumps in these discussions trumpeting that?
> What I don't understand is the excitement for using Rust vs. using garbage collected languages like Golang… since they don't need to climb the Rust borrow-checking learning curve.
Because, in my experience, climbing that learning curve has made me a better programmer more than nearly any other change in my long career. And that benefit has extended to code in every language I write.
The borrow checker isn’t just some hurdle to get in your way; it’s trying to tell you (awkwardly at times and perhaps less helpfully than one would wish) something fundamental about the way you think about and design programs. Internalizing that lesson can bring significant benefits on designing systems with clean boundaries that are easier to test, easier to reason about, and easier to compose.
Besides that, Rust greatly assists you (through features other than the borrow checker) in building software that is correct. This means it will tell you in a much wider variety of scenarios when future code invalidates previous assumptions. This is invaluable for projects that we expect to survive for a long time since the time a project is maintained will dwarf the time it’s under active development. And it will almost certainly be maintained by someone without the full context of the original developer(s). This is true even if the maintainer is the same person who wrote it in the first place, since our mental model of a program bitrots far faster than the program itself.
In practice, this aligns with my personal experience. Go projects end up with a lot of implicit assumptions that are silently violated by future work and expose bugs. They crash on nil pointer derefs. They accrue a multitude of linting tools that usually paper over some of the language’s shortcomings, but only in common cases. And they become painful to maintain as the original developers move on to other projects, with new changes grafted haphazardly into dozens of touch points instead of cleanly in one or two places. Yes, you can “easily” follow what any particular function does, but to do so you have to parse out and mentally model every minute detail, rather than being able to reason at a high level.
Conjunction is not equality. They are both memory unsafe. Then you can argue from that over how memory unsafe they are in practice (using the right practices, using the right language subset).
What makes you think so? Most Rust programmers and programmers from other languages, agree that this is not possible. I might be missing something, but can you give an example of such simple methods to trash memory in Rust, asking from a curiosity standpoint?
There’s always unsafe. I can make a pointer to anywhere by hand and write to it. That would involve some very intentional work, but I could do it if I wanted to.
But I did disclaim that it had to be somewhat intentional.
[1]: https://github.com/search?q=repo%3Aoxidecomputer%2Fhubris+un...
[2]: https://github.com/search?q=repo%3Aoxidecomputer%2Fhubris++l...
(At least, without further support. I consider "C/C++ with high quality static analysis" to be de facto distinct languages, and while I would favor something else even so, high-quality use of a high-quality static analyzer is enough to calm me down. Things have still crept through that level of care, but then, interpreters and compilers for safe languages have had safety errors in them before too.)
This is particularly true because it's just C and C++ that are memory unsafe. If we still in 1980, we could be arguing about the gradients of unsafety, but in 2023, we don't need to. Unsafety is not necessary at scale.
As for why people aren't asking to rewrite in Python, I partially answered that in my post. You can actually incrementally rewrite in Rust. You can't incrementally rewrite software in Python. There is also plenty of software that can be written in C, but simply can't be written in Python because it would be too slow. (Rewriting it in Python but oh no wait I'll just write the slow bits in C is a no-op, practically.)
As for trashing memory in Rust, by perfectly reasonable convention we generally understand that unsafe is unsafe, and that while languages can't avoid having it, having it does not necessarily make the entire rest of the language just as bad as C. I can crash Haskell with a straight-up, genuine null pointer exception with the Unsafe module in a single line of code. We do not thereby call Haskell an "unsafe" language where it is trivial to trash memory. Stock Rust is far safer than C++, to the point of being not only a qualitative change, but I'd contend, multiple such qualitative changes.
Once you separate C/C++ into safe and unsafe cateogries, and admit that Rust has unsafe uses that are "just so much harder to use", we're clearly defining a gradient:
Not really. See for example desktop Linux (i.e. Gnome).
Go is also opinionated with concurrency. So that's an issue too.
It is not only or even mostly legacy. I'm a systems programmer (in classical sense, not "but my web service is soooo highly loaded and scalable that I will call it systems programming!") and from what I see on the job people start new projects in C and C++ all the time.
C programmers are often more experienced people who are used to "simple" language that gets out of the way. They don't want to invest time into learning tricky language like Rust with all the intricacies of its type system, borrow checker, etc. Something simpler like Zig might work for them, but it is not on the table at the moment.
C++ programmers tend to be people who spent hundreds if not thousands of hours learning its ugly corner cases, reading Meyers and Alexandrescu books, that kind of thing. Sunken cost is immense, they whole careers are built on being "C++ experts" and they dread to abandon it and have to learn another very complex language from scratch.
And managers often don't see value in investing time into switching projects to new language. From their PoV it is more like programmers just want to play with a new toy instead of doing "real work".
On that note, try valgrind on existing javascript engines, you might be "entertained". (I certainly was, but that was some years back.
AFAIK the only competitive JS engines written in memory safe languages are GraalJS and other JS-on-the-JVM runtimes. GraalJS has the advantage of being fully up to date, not having any memory unsafe code in it (the JIT compiler that makes it fast is a separate module, also written in a memory safe language, and the JS impl does not have low level code in it). And you can run it on SubstrateVM which is a virtual machine also written in a memory safe language, although of course small parts like the GC need to use unsafe features.
It also has other useful features like sandboxing and the ability to interop with other languages like Python or Java. Plus, it can actually sandbox native code as well because the "languages" that you can run on GraalVM include both wasm and more usefully LLVM bitcode, in which each individual C/C++ allocation becomes GC-managed and bounds checked.
So in terms of memory safety the Graal team are way ahead there.
(disclosure: I recently started part time work with the GraalVM team, but was a long term supporter before that)
As for the second - good to know! Seriously appreciate knowing that - not sure if/when I'll need it myself, but it's good to hear, and good that it's visible here!
"SQL/PSM (SQL/Persistent Stored Modules) is an ISO standard mainly defining an extension of SQL with a procedural language for use in stored procedures... SQL/PSM is derived, seemingly directly, from Oracle's PL/SQL. Oracle developed PL/SQL and released it in 1991, basing the language on the US Department of Defense's Ada programming language. However, Oracle has maintained a distance from the standard in its documentation. IBM's SQL PL (used in DB2) and Mimer SQL's PSM were the first two products officially implementing SQL/PSM. It is commonly thought that these two languages, and perhaps also MySQL/MariaDB's procedural language, are closest to the SQL/PSM standard. However, a PostgreSQL addon implements SQL/PSM (alongside its other procedural languages like the PL/SQL-derived plpgsql), although it is not part of the core product."
https://en.wikipedia.org/wiki/SQL/PSM
So depends on your definition of "dead"
https://nap.nationalacademies.org/read/5463/chapter/3
tldr; seems to be that the software development world has changed from the days that DoD was the "dominant" software developer, and Ada in the interim did not get adopted by the commercial sector (with safety critical exceptions in aerospace, etc. noted).
Go isn't memory safe when using goroutines. See: Golang data races to break memory safety: https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...
* many GC'd languages like Go, C#, Java make it harder to leak memory, while languages where reference counting is more prevalent (Python, Rust) it can be easier to leak memory due to circular references.
* Languages with VMs like C#/Java/Python may be easier to sandbox or execute securely, but since native code is often called into it breaks the sandboxing nature.
* Formally-verified C code (like what aerospace manufacturers write) is safer than e.g. Rust.
* For maximum safety, sandboxing becomes important - so WASM begins to look appealing for non-safety-critical systems (like aerospace) as it allows for applying memory/CPU constraints too in addition to restricting all system access.
Do you have a single example in the last 14 years of memory safety exploit using the Go runtime? I'm talking about public and known exploit not ctf and the like.
> Is it possible to achieve arbitrary code execution on any Go version, even with PIE, and with no package import at all, just builtins? Yes!
Whether it's capture the flag is irrelevant, IMO, because anything that's allowed by the compiler will emerge given enough complexity.
1: https://blog.stalkr.net/2022/01/universal-go-exploit-using-d...
I mean if in 14 years there was nothing it's a proof that it's not an issue.
Even the attacker ack that it's not a threat.
"As said before, while a fun exercise it's pretty useless in the current Go threat mode"
Or bash before shellshock
I'm also surprised that a fix has been theorized at least as far back as 2010[1], but not implemented. Is adding one layer of internal pointer redirection for interfaces, slices, and strings really that much of a performance concern?
[1] https://research.swtch.com/gorace
The reason is obvious. There's a high cost to this type of safety. Rust is hard to use and learn and many times it's safety forces users to awkwardly organize code.
And there's still the potential for race conditions even though the memory is safe, you don't have full safety.
> It's closer to a NullReferenceException than it is to reading from a null pointer in C.
No, it's exactly the same as a null pointer dereference in C, because it is literally reading from a null pointer in Go as well. In Java, the compiler inserts null checks before every single dereference and throws an exception for null references.
> There's no memory exploitation you can pull off from this bug being in a Go program, but you could in a C program
Provided the OS sends a SEGV signal for null pointer dereferences, I don't see there being a difference in security between C and Golang in this respect. It's a bigger problem when you're running without an operating system.
Doesn't OpenJDK install a SIGSEGV handler, and generate the exception from that on a null dereference?
(AFAIK, a lot of runtimes for GC'd languages that support thread-based parallelism do so anyway, because they can use mprotect to do a write barrier in hardware.)
I thought I had read that they explicitly don't do that, but I can't find it anymore. You may be right. I should have checked before saying that.
> (AFAIK, a lot of runtimes for GC'd languages that support thread-based parallelism do so anyway, because they can use mprotect to do a write barrier in hardware.)
That's true. I guess those implementations must do something more advanced than "throw a NullPointerException if the program segfaults," given their garbage collector runtimes also rely on that signal.
This is exactly what they do: https://shipilev.net/jvm/anatomy-quarks/25-implicit-null-che...
When this happens it'll cause a deoptimization and recompilation of the code to include the null check rather than rely on the signal handler repeatedly.
;)
Java it's a pest fest for exploits.
From those, I'd choose Go, C# (and not totally sure because of AOT/JIT's) and Rust.
Sure, if you haven't used it since the nineties and pay zero attention to new development.
I’ve never seen an explanation for why this mechanism was added or what it was supposed to enable — besides enabling new exploits.
Every time I’ve seen Java used for a safety critical application the justification has been entirely based on the fact that it has cryptographic libraries that are widely certified for safety by enterprises. The security people on our side were… resigned.
The NSA's split offensive/defensive responsibility is bad but that doesn't affect recommendations such as this.
There's nothing wrong with a recommendation to use open, auditable tools. Be skeptical, but also I would expect this message to come from them.
Red team wants to hack everyone with a negative value add for America, blue team wants Americans (mostly companies) to be as safe as possible from digital attacks.
This seems one of those blue team suggestions.
Absolutely not in the memory-safe subset of Java though.
I think the uncoolness of them is due to OOP methodology... programmers today seem to want to be easier to get moving, but then spend more time debugging OR the debugging has been "placed onto" DevOps / use of Cheap fast hardware. IMHO
Their podcast titled "Signals and Threads" is quite interesting.
Actually, I have no idea if FPGAs in HFT are still a thing.
Now, nothing, and you if you want to train a new person, you do it on your own time.
no wonder so many bugs here and there.
Meta have a phenomenally good training program, called Bootcamp for basically everyone in engineering, so they're probably not the example you're looking for.
Unless it is a “hello world” commit to confirm tooling is operational, I have strong doubts.
When I started at FB in 2013, I had all the laptop/permissions things done by end of the first day.
At the time, the presumption was that you'd push a really small change by the end of the week, but all of the tasks available tended to be pretty easy (note I wasn't in eng then, so all my info is second hand here).
Back in the day people with SVN/CVS commit access would push directly to the main branch after testing locally, and testing would be done as part of preparing a release, so it was more necessary to have a "safety" period where new people could get used to the new codebase.
That doesn't seem like a contradiction at all. As you say, they want to hire trained workers rather than providing training themselves. Also, complaining is free while hiring good people is not...
A: What if you don't and they stay?
"I'd rather use a threading library written by one great programmer than by ten average programmers."
I see plenty of php logic which is far more likely to be publicly exposed and generally of a terrible quality.
The pushback I've received is something I'd categorize as "cart before the horse" - executives want to know exactly what will be achieved with such effort before approving any expenditure.
The teams I've been involved with - training has generally come organically and grass roots.
Do training on the things that can cause trouble longer term (inconsistent tooling taking over, security issues, company-specific things you cannot find online) and just hope people pick up the rest as they go. Good workers will usually manage. The people deciding how to train new workers maybe don't even know what kinds of "main" knowledge a given worker would need. Rather than assigning an expensive senior worker to training them, they might hope someone good willing will do it in their spare time.
I just watched the most recent Last Week Tonight, where they went in depth on Freight Trains [1]. Specifically they talked about 3 mile long trains that currently only have two operators - one conductor and one engineer - and the industry is trying to cut it down to one.
That's just an analogy for software development. But I've definitely seen people be over-worked and under-supported.
[1]: https://www.youtube.com/watch?v=AJ2keSJzYyY
(I actually don't know what the norm was. My dad was an EE for the defense industry, and IIRC he worked for 3 different employers over his career.)
But I've really got to ask - at that point do you really want to program in C++ anymore or is it just better to use a safer language that removes pointers entirely?
How do you manage heap allocations without ever touching a pointer?
And while there are some kinds of applications where never performing heap allocations is viable, this design is simply not an option for a huge number of applications.
Any C++ program that does not do heap allocations either uses arrays as a substitute for the same thing or isn't a general purpose application.
I use safer languages when appropriate but there are cases where they would introduce more bugs than they address.
Such as? Genuinely curious.
If you implement something like a database kernel almost entirely in user space, which is what you want to do if you care about performance, you have a new set of problems that languages like Rust were not designed to solve. Ownership, mutability, and lifetimes are not resolvable for most objects at compile-time, they can only be resolved dynamically at runtime. With C++, you can make these look like ordinary objects with ordinary behaviors, with metaprogramming abstractions under the hood arbitrating the chaos. Things that look unsafe are actually guaranteed safe by the underlying abstractions. In this type of software, almost your entire address space is “unsafe” in some way, so it is enormously useful to be able to overlay a safety model that does not rely on the compiler, since the compiler cannot solve these cases.
I use both C++ and Rust in production environments. There are cases where C++ used well is legitimately safer, theoretically and empirically, which is why I still use C++ for some things. Rust may become stronger at dealing with these cases but it still has many years to go. (Also, Rust async story is a hot mess, and you really need async for scalable, performant systems software.)
I have seen that too over the years and even been to a couple when I went to other companies, 99% of the time these classes are worthless and consists of marketing drivel. Half the class contains statements like "You can do it the hard way, or the easy way if you purchase ....".
One instructor even stated "You do not need to worry about running out of memory, allocate as much as you want". That class was 20+ years ago. After that I told my manager I will not go to any more vendor classes.
The training I am referring to is by experienced peers and goes on for many months where your work is reviewed in detail, 1 line at a time with you there. That use to happen decades ago, now no more.
Edit.
Idk why downvotes
Chrome and Windows code bases had 70% of cves due to mem issues and their engineers arent some unqualified and lacking of training opportunities ppl
Even in highly regulated, controlled environments memory safety is still an issue. I have seen bugs in these environments where I know for a fact that the developer has had 40-80 hours of training on memory safety and they still pop up.
If these people have occasional issues, the rest of the world has no effin chance.
I'm afraid your approach would mean hiring someone and spending time and money on them before you're sure they'll work out in your organization. Few want to train contractors and firing FTEs after 3 months and change is a dim prospect as well.
https://www-users.york.ac.uk/~ss44/joke/c.htm
#joke #hilarious
> Memory safety vulnerabilities are coding errors affecting software’s memory management code in which memory can be accessed, written, allocated, or deallocated in unintended ways.
How can you do any of this without software running in the local machine?
Honestly asking.
1. You have a service that takes some user input
2. The service allocates a buffer on the stack for 20 bytes (the max input allowed)
3. Service doesn't actually validate that the user input is < 20 bytes
4. It writes the input into the buffer, but keeps writing past the end of the buffer.
5. It eventually overwrites the return pointer so now the user input can control where the execution jumps to on return
6. In the user input, the malicious user includes some shell code to do any arbitrary thing they want.
7. The overwrite the return pointer to jump back into the shell code and now they are executing arbitrary code in the server process.
This sort of thing was embarrassingly easy to do on old linux kernels before ASLR was implemented. Now it is dramatically harder because there are all sorts of countermeasures in place to prevent it (ASLR, stack canaries, executable vs non-executable memory, etc).
To get a sense of what a more modern real world exploit looks like, give https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i... a read.
When it comes to unsigneds, there's no such problem (and this is in fact the real reason why anything that can be unsigned in C/C++ should be unsigned).
Trapping is nonsense - in production anyway. You might use it in development to catch errors. But in production there is no way to "fix" an overflow if it happens and is detected, so you'd be looking to crash on trap which in some code is preferable to silent data corruption.
I do lots of fixed-point math for embedded motor control. Representing rotor angle as a signed 16bit number is deeply engrained in me these days because taking differences to get a signed delta just works, and I never have to worry about rollover because it behaves exactly the way I want. ;-) While this is technically undefined behavior, I've never run across a compiler that worked differently.
Undefined behaviour can cause miscompiles which can break the expected logic of a program. In the worse case, maybe the compiler optimises away your password validation because it has decided that the failure branch is undefined behaviour.
Non determinism can lead to logic bugs, but it can't magically introduce new logic into the program like UB can as part of optimising
There's also explicit arithmetic functions that let you choose your overflow behavior:
* Checked: return None on overflow
* Wrapping: two's compliment wrapped overflow (the "overflowing" variant gives you a carry bit)
* Saturating: return maximum value on overflow
It can not, because "compiler optimization passes" is not the root cause of this behavior: it is that it is undefined behavior in the language itself. This is what gives the compiler license to make those transformations. In Rust, it is not undefined behavior, and therefore the compiler does not have the right to make those transformations for Rust code.
Just because Rust uses LLVM does not mean that suddenly it inherits C's semantics. This has actually happened (well, more specifically, C++'s semantics, C was accidentally inheriting them as well IIRC) at least one time before, but that is a bug in LLVM that was fixed. Now each language has the proper semantics here, and it all works just fine. (I am referring to the behavior with regards to infinite loops with no side effects.)
https://borretti.me/article/signed-integers-asymmetrical
I know Rust catches it because after I read that article, I tested it with Rust in debug mode.
It is easy enough to crank out custom integer types in C++ these days which are arbitrarily safe that there isn’t much excuse for not doing it, particularly since generics and metaprogramming does most of the work. Outside of interfacing with syscalls, there isn’t much use for C primitives beyond size_t.
I didn't know this. I can imagine you can create your own integer types as wrappers over existing primitives, and define them so that, e.g., on signed overflow, the program aborts or wraps (and your signed integers are actually a wrapper class over unsigned integers, but with operator overloading so that they act like signed integers). Is that what you mean? If not, could you show me what you mean?
This did not work well in older versions of C++ because the limited type inference meant that many common cases around different type interactions required explicit handling that made it clear you were not using native types. Around C++17 it became possible to define integer types that were almost entirely indistinguishable from the native types in ordinary C++ code.
C++17 was important because it meant a lot of safety in the code base could become automagic via the type system, especially for primitive types. The code looks the same whether you are using it or not. That was a huge capability change. C++20 then generalized it to arbitrarily complex types. It is difficult to overstate how much this improves the conciseness and safety of non-trivial code.
There are likely multiple necessary changes for it to be fast. One part is implementing the check quickly, this could be hardware assisted. Another is addressing all the optimizations the compiler does by asuming that there is no overflow, and figuring out alternative ways to get the compiler to emit fast code.
The Python Software Foundation noted this in our response to the US Government RFI on open source security. There are efforts to make using Rust easier as a systems language for Python packages and some security-critical packages like 'cryptography' have migrated to Rust.
If no friendlier safe high speed programming language appears, I rather use C/C++ and trade safety for friendliness.
Maybe the risk is mitigated in other ways. Your software runs as a cli and not a service. It doesn't process outside input. It is run in an ironclad sandbox.
But honestly, if you think the way C++ with RAII want's you to then you should already be following the rules that Rust want's you to follow for the most part.
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p26...
Rust is generally perceived as having many of the same developer kindnesses as C++, while having same-class performance. Learning it has a bit of a curve, but I found that it eventually clicked (I have a C and C++ professional background).
E.g. Whenever handing out the raw pointer stored in a unique_ptr, the code receiving that raw pointer can delete it, which will very likely lead to a use after free and certainly to a double free issue.
```
class Foo;
Foo get_foo() { std::cout << "whoops, I forgot to return\n"; }
int main() { const auto foo = get_foo(); }
```
One wolf that wants to help ensure that American computers are secure
And one wolf that wants to hoard as many 0-days as possible
Obviously, making certain concessions would be a deal-breaker for some, but it might be viable for legacy codebases. If you were to try to make C++ memory-safe, where does it begin to break down?
There might be a better path for memory-safe language interoperable with C++ (Carbon?) but this will require significant effort.
I'm in no way an expert in any aspect that relates to this problem, but it's reasonable to believe that it would be but a mildly challenging task if companies like Google, meta, Microsoft and the likes joined forces.
A year of concentrated efforts might be sufficient to rid us with this problem once and for all.
Btw I don’t know how this works in most languages, but circular references can be still a problem even with these restrictions. (If I remember well dangling references can be more or less handled, but correct me if I’m wrong. I remember that JavaScript definitely had problem with those)
Getting these features in the C++ language itself will be absolutely impossible, both because of massive performance implications and well as nightmarishly large ABI breaks on everything.
There are various efforts to do what you are describing. I think that they are often a good idea, but they absolutely will not solve the problem conclusively.
As a C++ programmer, I really want to find a spare week or two to learn Rust.
Promoting cyber security in the civilian workforce is also part of the DoD's 2023 "Cyber Strategy".
The Department will take action to foster a culture of cybersecurity and cyber awareness. We will establish an expectation that senior military and civilian leaders possess a baseline fluency in cybersecurity issues. The Department will develop, fund, and implement technical curricula across different levels of professional military and civilian education, emphasizing General Officer and Senior Executive Service leadership courses.
https://media.defense.gov/2023/Sep/12/2003299076/-1/-1/1/202...
They're also trying to promote cyber security in open source software: https://www.nsa.gov/Press-Room/Press-Releases-Statements/Pre...
That's not to say you're wrong about the NSA probably having backdoors in critical software like the Rust compiler.
In the PDF the closest things they talk about to sanitizers are SAST/DAST tooling, which isn't exactly the same thing, and compiler mitigations, which aren't exactly the same thing, but I also don't believe this is intended to be fully comprehensive at this level of detail.
I'm in no way an expert in any aspect that relates to this problem, but it's reasonable to believe that it would be but a mildly challenging task if companies like Google, meta, Microsoft and the likes joined forces.
A year of concentrated efforts might be sufficient to rid us with this problem once and for all. Billions of lines of codes would gain (some) safety instantly.
But it's not going to be "drop-in", lots of stuff will have to be refactored to work with Rust's memory model.
See, memory management is a hard problem and the complexity involved in solving it has to live somewhere.
In C and C++, that complexity lives in the programmer's head.
In Java, Python, Go, etc. that complexity lives in the program's runtime in the form of garbage collection.
In Rust, that complexity lives in the compiler - with some caveats, Rust code is safe as long as it compiles and passes the borrow checker.
However, the ownership-based memory model of Rust is not always trivial to port other code to. Modern, idiomatic C++ written with RAII and smart pointers should be relatively easy to port to Rust, but most of the critical systems code out there is not like that at all. Most of the critical systems code out there is written in C or old-style C++ with C-style strings & arrays and raw pointers to heap memory all over the place.
So regardless of what your C/C++ language replacement looks like, it will take some serious effort to replace that kind of code.
You can't do "drop-in replacement" and "memory safety" at the same time.
- continuing accumulation of documentation and utility libraries from Khronos
- excellent learning materials from the community
- tons of legacy code
all using C++.
https://github.com/KhronosGroup/Vulkan-Utility-Libraries
https://github.com/cg-tuwien/VulkanLaunchpad
https://cescg.org/our-services/an-introduction-to-vulkan/
Until I see professionals get funding to build, maintain, document, and create training materials for Rust bindings, I'm going to continue to assume there will not be movement in that sector. I don't think industry is going to throw away the C++ ecosystem and build greenfield projects on a foundation of hobby projects.
After using Rust for a while, I actually decided to stay with to c/c++ for the rest of my career.
Care to expand?
Right now Rust is technically safer language but it doesn’t have the same amount of job postings and career opportunities as C++. That’s just an inertia problem.
The conundrum is if you jump in early and risk the language becoming yet another language or do you wait and see.
I haven’t coded C++ in 4 years, but my understanding is that the language is evolving to answer the threat Rust poses to it.
There are still COBOL and Fortran developers, C and C++ won't go away tomorrow, even if the goverment says so, but it will be harder to use them in security clearance contexts, similar to hazardous goods.
The residue of holes in C++ code is about the same as JS, Python, etc. So just eliminating C and C-like constructs gets you to the same level of security as the other languages.
> Importantly, if you look at lists of "C/C++" security holes, you find it is almost all C security holes. So when they write "C/C++" they are hoping you won't notice they are lying to you.
As these "C-isms" are part of the base language, and you cant 'opt out', this means the base language is still included in the lists. Most C++ applications still linked (and used symbols from) the system libc runtime.
I do not write C++ with any regularity, because I like my C footguns and dont want additional layers of footguns in my code. last time I tried writing C++, it allowed you to use after free, access raw memory pointers with no validation, iterate off the end of arrays and integer overflow and underflow, all without using "C" style constructs.
Maybe it has changed, maybe C++ is now not as insane as it used to be and smart auto magic pointers solve everything. I'd love to read about how modern C++ solves these issues, do you have any resources ?
It is no accident that Khronos has security critical certified subsets of OpenGL and Vulkan.
Right now the only C++ wannabe replacement that is as far ahead as Rust is Circle, and it is a big political issue with the community, unfortunately.
Forget about Carbon, Hylo or Cpp2.
My only interactions with compiled languages all have legacy constraints, or involve someone else's project.
ESP32 seems to have lots of Rust support work happening, so I can very easily imagine switching in a few years though.
If I had a reason to write a new performance-critical non-embedded app I'd probably use Rust.
I was hopeful we'd be transitioning by now.... it's at least another decade out.