51 comments

[ 0.21 ms ] story [ 4.0 ms ] thread
The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way as your original code would, right? You can optimize on "I already checked the value of X" if you can prove that nothing could have changed X.

Well, it sounds like a lot of compilers are making unjustified assumptions about what the outside world is allowed to affect or observe. Maybe with the encouragement of specs, maybe not.

> The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way

Well, in C/C++, as soon as your program has one UB bug, the compiler has absolutely no obligation whatsoever.

This is not quite correct in C. ISO C at least requires that observable behavior until this point is preserved.
I don't think this is true. You're going to have to point to the exact place in the spec that says this, because optimizations in the presence of UB in most compilers make absolutely no assumptions.
It follows from the definition of UB:

undefined behavior: "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this document imposes no requirements" The C++ spec at some point changed this to explicitly allow changing anything in the program not just the specific behavior implied by the "for which". The C spec never did this.

Because people were confused about this, in C23 we added the following note. "Note 3 to entry: Any other behavior during execution of a program is only affected as a direct consequence of the concrete behavior that occurs when encountering the erroneous or non-portable program construct or data. In particular, all observable behavior (5.1.2.4) appears as specified in this document when it happens before an operation with undefined behavior in the execution of the program."

Compilers mostly follow this. GCC has bugs related to volatile. Clang often follows the C++ standard, where it is different from C, so probably does not conform to the standard here (as for some other things).

The new C++ standard will have UB "barrier", i.e. std::observable that will limit the effect of UB to things before this barrier.

This is important. Undefined Behaviour is a behaviour, and so if we can ensure the behaviour doesn't happen, our problem is averted. For example if there's UB when a zero size file is loaded by our software, we can instruct operators to check the file has a non-zero size and we're preventing whatever horrible UB would arise.

We can even rope off whole parts of the software. If the Postscript printing code has UB, simply instruction operators only to use the HP inkjet printers for which we know Postscript is not used can prevent this UB from happening.

Well yes, that's what UB means. It's a singularity, you run into it and there's no longer a specified requirement for the behavior that will follow.

Rust also has UB, btw, https://doc.rust-lang.org/reference/behavior-considered-unde..., so I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you.

If you want to write code for a VAX, then you can use the K&R C compiler where it had defined outcomes for everything. If you want to write portable C code for modern CPUs then it's fair to ask what the C language standard is supposed to define for each of those CPUs and OSes and ABIs.

And a bunch of people were nice enough to do that for you and I, but because they are not deities, there are things that they had to leave out to make the language useful, so they did.

> I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you.

They did. Most other languages, the vast majority of which are also memory safe languages, go out of their way to do the opposite, and give meaning even to erroneous programs. Some things slip through the cracks, and generally language designers and implementers work hard to get rid of UB.

C/C++ is the only ecosystem that has fully embraced UB as a way of life. They are the only compilers that make full use of "UB is bad and cannot ever happen" as a core tenet in how optimizations are designed. Rust UB is at least a little different. Rust UB can only be the result of unsafe code and is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster.

C/C++ have a surprisingly large set of UB, too. Thankfully, the rest of the software world is rising up and the committees are starting to make things like gasp signed arithmetic overflow into defined behavior.

But don't hold your breath.

> Rust UB […] is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster.

This isn't true. Rust UB is meant to only be possible to trigger via `unsafe` code, but if you do trigger it, the compiler is free to do whatever it wants, and in practice it will make full use of that freedom. Rustc uses LLVM, it shares most of its optimizations with Clang!

C++26 and C++29 are in the process of turning a lot of that UB into erroneous behaviour, basically what safer languages have been doing for ages.

However, how many years it will take until those versions become widely deployed across major compilers, and used by developers?

> Rust also has UB, btw

Not at all the same. C and C++ are full of hazards and I get the impression it’s genuinely difficult to avoid entirely in normal code bases, and typically impossible to avoid statically. Whereas in Rust it’s all gated behind the unsafe keyword, and if you don’t use it (and most code bases never need to use it), you cannot encounter UB; and that scoping makes it far easier to control and handle correctly.

I think this is a bit exaggerated. I mostly find it not difficult to avoid UB in C. There are mainly five areas where you can have problems: type safety issues, signed integer overflow, out-of-bounds accesses, use-after-free, and race conditions. Type safety is generally not a problem if you avoid unsafe casts (and casts are easy to screen for just like "unsafe"), signed overflow one can protect against via sanitizers or one can rule it out statically, and out-of-bounds accesses you can avoid by using safe buffer and string abstractions and never doing open-coded pointer arithmetic.

Use-after-free and race conditions are the areas where Rust has a clear advantage. Here one needs to have a clear strategy and enforce it manually (or using tools, but we lack good open-source tools for this). Valgrind and similar tools also help.

As someone who coded C++ (15+ years) and later Rust (about 4 years now) for my dayjob: there are more than those you listed I have seen commonly. Unaligned accesses is a perrenial favourite, as is ODR violations and reliance on the undefined order of static constructors between translation units. In C++ I didn't see much of unsafe casts, except related to enums (always specify an underlying type to mitigate this).

Rust protects against all of these, but if you think Rust is only about memory safety, I don't believe you have seriously tried it. It does a lot of things in std API design as well to steer you away from bugs. Some examples:

- The pervasive use of Result and Option makes it impossible to forget to handle (or forward) the error case.

- Because of usage of RAII (C++ has this too, but not as pervasively, C doesn't except using some very new GCC extension) it is very hard to forget to free resources such as files, sockets, database connections, mutexes, etc.

- Enums can carry payload in their variants (C devs: think tagged unions, but safe, C++ devs: think std::variant but with match/case rather than bulky visitor pattern), which means you can make API designs that cannot represent invalid states.

- The typestate pattern is a bit hard to explain briefly, but it allows a state machine with types at compile time, to make sure you dont misuse an API. For example it can be used to prevent forgetting setting required fields in a builder before building. Or in embedded microcontrollers to make sure you can't hand out the same GPIO pin to different parts of the code base.

I often find that my code in Rust works first try, while that almost never happen in C++ for non-trivial code. It is what all those Haskell devs were talking about all these years, but in a systems language (no GC is critical to my day job in hard realtime control systems) and without the incomprehensible abstract math lingo.

Almost but not quite. You are promised that safe Rust doesn't have Undefined Behaviour, but that's cultural, not technological. The technology is just enabling Rust's culture to deliver what they promised, but it would be very easy to purposefully (indeed there are known bugs where it does happen, look for "Rust soundness bug" if you want examples) inject UB which happens in your safe Rust.

The extent to which this is about culture should not be underestimated, to me that's the most hilarious part of Bjarne Stroustrup's big rant on memory safety a few years ago. The C word appears exactly once in his slides, in a quote from somebody else about what needs fixing. But Bjarne never addresses this once, even though it's the actual problem.

Sure Rust is better on it, my point is that it is there, despite that community making it a Big Freakin' Deal to have all the memory safe they could design into the compiler and language.

If even they had to add escape hatches despite the presences of powerful language primitives like types, traits, borrow-checking, maybe the people charged with making it all work with 80s compiler technology weren't the literal Antichrist for also having UB as Rust does.

For what it's worth, C and C++ are much different in terms of hazard, so when you bucket them together it makes me wonder how familiar you are with the actual risk of UB in practice.

Nowadays it generally stems from doing weird things.... but no one is making you do weird things, any more than people are making you use Rust's unsafe keyword.

If you don't want UB to mean "Absolutely anything might happen" which necessarily has to include "... forever" then you need Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead.

To some extent in C and even more in C++ there's a much worse problem, IFNDR [Ill-formed No Diagnostic Required]. Programs which the language specification insists mean nothing at all, but your tools won't (in many cases can't) notice so the result might do anything. It's not Undefined Behaviour, your program never had any defined behaviour at all.

> Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead

It’s worth being clear here that this is not what Fil-C does, it still has UB, and can still explode in many of the same ways as C and all (after all, it’s a clang fork). Fil-C takes one particular class of allocation related bugs and UB off the table, but leaves many of them behind.

Whatever happens for the other UB remains bounded memory safely in Fil-C. (according to a definition of memory safety that excludes protection of subobject bounds, but Rust also redefines memory safety to what the Rust compiler can do, e.g. excludes memory leaks).
I'm not so sure you can make such a strong statement about what happens when UB is invoked. The presence of UB allows the compiler to make all kinds of weird assumptions, and it seems very unlikely that there exists no series of allowable transforms that results in a pointer capabilities check being elided or similar.
Presumably Martin intends the usual caveat that this is subject to bugs. So, "there might be a bug in the compiler" isn't interesting. There are bugs in LLVM, bugs in Rust's trait resolution, bugs in Javascript implementations, obviously code has bugs and that's generally not very interesting - we can fix bugs.

Are you claiming that there must be such transforms or only that it is likely that bugs exist which isn't interesting.

Neither. I'm claiming that UB allows a transform that violates or ignores the additional guards put in place by Fil-C, and by the definition of UB that is not a bug in the compiler, as any behavior is allowable.

To assert that something specific always happens under UB is counter to the definition of UB. Fil-C carefully defines away UB for some operations, but to make full guarantee of safety, even by their definition and modulo bugs, I believe it necessary to fully remove UB.

Maybe it was unclear that Fil-C is a compiler. So the thing doing those transforms you're worried about is Fil-C.

Fil-C gets to look at some code which has UB if variable z is 9 and go "OK, lets check whether z is 9, and if so...". The result, of course, is much, much slower than executables from a typical modern C compiler, but since "It always does X" is in fact a permitted implementation of "Undefined Behaviour" this is a compliant C implementation, at least in most observable respects.

Leaking isn't unsafe. Even if you have a linear type system, and so you can't leak in this useless technical sense, it makes no practical difference, the end user doesn't care that your program which gradually bloats by 1GB per hour until it blows up doesn't technically "leak" memory because it was just caching some data with no limits, whereas my program which grows by 800MB per hour and thus blows up slightly less often does technically "leak" memory because it didn't keep the references needed to free that data. To the end user these programs are both leaky garbage.
This depends on the definition. Rust originally included leaks in their definition of unsafe and later changed the definition when they found out that they can not reliably prevent leaks.

Leaks could reasonably be considered unsafe as they can cause a program to crash due to resource exhausting, even where the actually used memory is limited.

If you do not consider leaks as a problem, there is a trivial way to avoid use-after-free and double-free: Simply never free any memory. (Which in some scenarios is exactly what people do.)

> Leaks could reasonably be considered unsafe as they can cause a program to crash due to resource exhausting, even where the actually used memory is limited.

You can make this claim about any resource, there's no reason to single out memory here. You can run out of file descriptors, inodes, connections to a remote database, disk space, anything - including CPU time.

And notice that it wasn't the leak that you've now said was unsafe, it was the use itself. The program didn't blow up "because of a leak", it blew up because we exceeded some arbitrary resource threshold which may have been invisible to us.

> If you do not consider leaks as part of the problem, there is a trivial way to avoid use-after-free and double-free: Simply never free any memory

Indeed. And that's exactly what we see in some domains and if you've solved the other issues (e.g. bounds misses, type confusion) you've now got memory safe programs. Most general purpose software can't be written this way, but there is a whole heck of a lot of software out there which could be.

> The later would fulfill your definition of "holding references to be able to free them".

And that former would be characterized as "leak everything" and indeed that's entirely safe and, just as I said, to an end user this is a distinction which makes absolutely no difference.

Yes, other resource leaks can cause similar problems, this just does not have much to do with memory, so calling it "memory safety" would be strange. On the other hand, including "memory leaks" under "memory safety" makes a lot of sense, which is also why Rust did this initially. And, of course, if you look at garbage collection, then they really cared about this.

To the end user memory leaks are not important as long as you do not exceed the available memory and the program does not crash. The moment it does, it a problem and it can be a risk. What I agree with, (if you argued that point - but you don't), is that it is more benign and qualitatively different risk than unbounded behavior you may get with an out-of-bounds access or use-after-free.

Do you have a concrete example of C which will "still explode" ?

  #include <stdio.h>
  #include <stdlib.h>

  int shift(int x, int n) { return x << n; }

  int main(int argc, char **argv) {
      printf("%d\n", shift(1, 32));   /* n == width: UB */
      return 0;
  }
This program exhibits UB in Fil-C, and you can see that the optimizer does different things at -O0 (outputs 1) and -O1/-O2/-O3 (outputs 0). Since this creates poison, which Fil-C doesn't remove, you can use it to construct all kinds of weird things.

    static void loop(void) {
        int s = shift(1, 32);
        int n = 0;
        for (int i = 0; i < s + 3; i++)
            n++;
        printf("[loop] iterations=%d (s+3=%d)\n", n, s + 3);
    }
    
    static void sw(void) {
        switch (shift(1, 32)) {
        case 0:  puts("[switch] case 0"); break;
        case 1:  puts("[switch] case 1"); break;
        default: puts("[switch] default"); break;
        }
    }
    
    int main(int argc, char **argv) {
        loop();
        sw();
        return 0;
    }
In Fil-C -O0, this gives 4 iterations of the loop and executes sw(). At any higher optimization level, it turns loop() into an infinite loop and drops sw() from the binary entirely.
Neither of those sound like Undefined Behaviour to me, they're maybe unspecified but they don't sound undefined at all - are you confusing Undefined Behaviour with "I wanted it to do something else" ?

You said "explode" earlier and so I was expecting something a bit more dramatic than "Unsurprisingly Fil-C has unspecified results for some expressions".

You seem to be trying to apply some colloquial definition of undefined behavior. Undefined behavior is a very specific, defined term. The shift(1, 32) call is by definition undefined behavior (See 6.5.7 in the C standards from C99 up).

The behavior of the program itself when undefined behavior is invoked is allowed to be _anything_. I've simply demonstrated here that the compiler is using the fact that there is undefined behavior to perform optimizations that would not be allowed without undefined behavior. Those optimizations are allowed to result in the program doing anything at the compiler's whim.

I guess I've learned all I was going to, Fil-C does exactly what I understood and some crazy people will insist somehow that doesn't count. Good luck to you.
To decide what is defined or not, you need to reference the relevant specification. The specification is not ISO C, but ISO C plus additional guarantees by Fil-C.
I reference C's here because it gives the best idea of what clang/LLVM itself is going to consider UB and the bulk of the optimization semantics (where they haven't been changed by Fil-C), since Fil-C is really a fork with some additional passes and transforms built-in.
Fil-C is a fork of clang that explicitly defines all these things to have bounded behavior, so ignoring exactly this - the whole point of Fil-C - makes no sense.
But what are "all these things"? It does not define away all of the UB that C/LLVM has for sure, nor does it turn all UB into crashes, which I demonstrated above and is the inaccurate description of Fil-C that spawned this. But beyond that, it's my belief in all of this is that leaving some UB behavior while trying to state a global correctness property puts those guarantees at risk.

I spent a few minutes poking just to see if my gut is right here, and already, here's an example of UB being used in an optimization by the compiler that leaves a fil safety check at on -O0 but drops it at higher optimization levels. I find it difficult to believe that all of the complex interactions of every optimization pass in the presence of even this subset of UB are guaranteed not to violate these memory safety promises.

    #include <stdio.h>
    #include <stdlib.h>
    
    __attribute__((noinline)) static void poke(int *p, int k)
    {
        int n = 32 + (k & 15);          /* always >= 32: shifting an int by >= 32 is UB */
        p[(1 << n) * 20] = 0x41414141;  /* on x86 the CPU computes index 40, out of bounds */
    }

    int main(int argc, char **argv)
    {
        int *p = calloc(16, sizeof(int));
        poke(p, argc);
        puts("after poke");
        return 0;
    }
It does not have to fully specify what happens in each case, Fil-C only formulates a bound that guarantees that any memory access is limited to the memory that can legitimately be accessed. That safety checks can be removed, where the optimizer can prove that they are always fulfilled, would be expected. If it is removed and then still allows an out-of-bounds access, then this would be a bug in Fil-C.

I also do not believe that all complex interactions are guaranteed to not violate all safety promises. I also know that this is not true for Rust, so what is your point?

The compilers are mostly doing the right thing (not always). The C standard specifies what affects the outside world, i.e. file I/O and volatile accesses. For concurrent programming, there is also a memory model that specifies what other threads can see.

Here, the issue seems that compilers can reload variables. If this is a bug, then you already have a data race in your program which you can prevent with correct use of locks and/or atomics.

Compiler does what I tell it. Not the other way around.
The compiler does everything that the language spec allows it to get away with. The language spec itself is a communication protocol between users and compilers, and that includes the definition of the abstract machine.
Then the language purpose is vacuous.
Title should be "Your C compiler can undo your security checks".
Any compiler can do it -- LLVM definitely does even for languages that aren't C
A key problem is that compilation operates on an implicit (compiler writers have this in the back of their heads) notion of correctness which is very roughly “preservation of observable behaviors” where “observable” is sequences of system calls and then return value. That is, the final output of a compiler should never add new sequences of observable behaviors.

Security properties on the other hand are very often about the relationship between these sequences. For example we like to say that an external observer/attacker can’t distinguish internal state by the external observations (confidentiality) which requires that two observable traces given different hidden values don’t have different observations from the same starting observable state.

If you’re an LTS nerd you know this difference as trace properties vs hyper properties. Compilers try to preserve the former but not the latter.

Separately there is the problem of “what is observable?” For example, if you include timing in your observable behaviors suddenly the kinds of compiler passes that are able to preserve observations tends to zero rather quickly.

weird f*ing site. apparently doesn't like hn visitors and doesn't want me opening the console. i know when im not wanted
> doesn't want me opening the console

Whoa. That's not even something it should be able to detect. What's the gaping security hole that lets it do that?

there are a wide variety of ways to indirectly detect that someone opened devtools

the most common way is to set a breakpoint, which pause JS execution only if devtools are open. this pause can be detected with a timer.

...how do you set a breakpoint in someone else's devtool console?
On Firefox, while F12 is intercepted you can still open it with Ctrl+Shift+E