What CPUs in common use still have trap representations? Are there other reasons why reading from an indeterminate value can't give an arbitrary but well-defined result?
x86, in the form of signalling NaNs, when the appropriate fpu flag is set. Some languages feel it's better to fail a computation early and loudly than to silently proceed with a cascade of non-signalling nans possibly getting as far as the screen or database. Delphi for one. C will see this if it's in a library linked with code in such a language.
Aren't signaling NaNs defined to trigger a floating point exception, not invoke undefined trap-representation behavior?
EDIT: I found this proposal (http://www.cl.cam.ac.uk/~pes20/cerberus/n2091.html) which has some discussion about existing trap representations (including whether to consider signaling NaNs to be trap representations). The conclusion seems to be that segmented pointers on the Motorola 68k are the only place trap representations are truly necessary -- maybe it's time to just remove them.
Exactly. A compiler cannot guess what a programmer intended, but has to stick to the language semantics.
Also, the article goes on to argue that uninitialized memory is used to seed random generators, which is generally undefined behavior. I don't think this should be a valid use-case.
That business about uninitialized values being used to seed RNGs threw me for a loop as well.
I think it's safe to say that x ^= x is equivalent to x = 0 and optimize it that way, just like compilers recognize common idioms for e.g. swapping, counting set bits, etc and emit the correct intrinsic instructions.
While I fully agree with you, but I will still like C to be as self explanatory as possible. C is the closest language for which one doesn't need to know the standard to understand it, at least the standardised C. I think it should be one of the biggest aim of C to let the language as it is(except the standard library). This change creates something that might not be expected by some. If this is replaced by an compilation error, I am okay with it.
> but I will still like C to be as self explanatory as possible.
C is not self-explanatory at all and without good knowledge of the standard anyone will write broken code that will fall apart sooner or later. C is not a simple language, nor one with simple semantics. Without knowledge of the standards it is essentially not possible to write portable C code.
I was thinking about reading the code line by line. Name any other language in which you can do that, without spending at least a week in that. I don't know what is a simple language according to you, but according to me C seems quite a simple language. Not saying it means that a good quality code is easier to write in C.
I can't, but I also think your premise is not correct ("reading the code line by line" "without spending at least a week in that"). A lot of production C code is inscrutable without a good understanding of the language, and to understand many common constructs (e.g. overflow checks) you just have to know it. That trivial code is trivial to understand... well... that's true in most languages, no?
The standard can't mandate a diagnostic for undefined behaviour, because it's not always possible to determine if behaviour is undefined. (Turing completeness) However, an implementation is allowed to issue any diagnostics it likes, so if you want one for this case, bug your compiler vendor to add one. Or, check the manual - it's possible that your compiler already does issue a diagnostic if you turn the right warnings on.
> The standard can't mandate a diagnostic for undefined behaviour, because it's not always possible to determine if behaviour is undefined.
Yet that argument is stupid if we are talking about compilers issuing warnings for the UB they "detect", because by definition they have detected them. It might not be possible when it does not work exactly like that in some cases, but at least before removing some code, this is desirable.
It might also not be easy given the current internal design of compilers, but then I argue those design should be changed, because it is just too dangerous.
But pretending that compiler developers are currently just ignoring the issue out of mischief and preference for performance above anything else is disingenous. If the problem were so simple, we'd probably already have a dozen free static analyzers that do a good job, and you'd be happy to use them.
The thing is, "detecting" UB in the sense you imply is usually not what happens in the context of said optimizations (that's not to say they won't attempt to do it... ever seen a compiler warn you about use of uninitialized variable?).
What compilers do is they assume the program is correct. And following that assumption, they perform an optimization that is only correct for correct programs. That is in fact very simple to do. They do not try to prove or disprove that program actually invokes UB there -- that is impossible in general, and even in the subset of cases where it is possible it could require deep whole-program (including libraries!) analysis that could take massive computational resources.
Many people here keep trivializing the problem but I don't think they understand the problem at any depth.
And that is why I say you should pave the way, not in a smug "fuck off get off my lawn fix your own problem" sense, but to get people to honestly gain some background in program analysis, read research papers, study existing analyzers, and gain some appreciation for what it takes. It is far, far from trivial. Especially if you can't just take the language and change it to your liking (breaking nearly all existing code) until all the hard stuff is out.
And in saying that, I suggest that it is easier to start with a new language (or, at least some existing language other than C) that was designed from ground up with such analysis & correctness provability in mind.
But honestly what a waste of resources. I would prefer to be able to work on topics that are not problems created by a recent shift of view of how compiler should be designed and optimize, shift that is for the worst IMO.
And yep I know the reason in the current design for why it is not always easy. That's why I added the caveat that maybe they should be changed.
But then you can not just says that all the users criticizing and "trivializing" the problem are in the wrong and do not know what they are talking about. First, most of the discussion I've seen either points at technical justification of why those crazy optim should not be done, be it for safety, possibility of real optimizations by the programmers (and to compare, I'm not impress about the capability of compilers to take poor code and to transform it to somehow less poor code, especially when the language in question is C, that always required mastering); or points to discussions of compiler authors that are borderline insane (when you see them discussion about how technically the standards would allow to not consider uint8_t and char as alias, a
But honestly what a waste of resources. I would prefer to be able to work on topics that are not problems created by a recent shift of view of how compiler should be designed and optimize, shift that is for the worst IMO.
And yep I know the reason in the current design for why it is not always easy. That's why I added the caveat that maybe they should be changed.
But then you can not just says that all the users criticizing and "trivializing" the problem are in the wrong and do not know what they are talking about. First, most of the discussion I've seen either points at technical justification of why those crazy optim should not be done, be it for safety, possibility of real optimizations by the programmers (and to compare, I'm not impress about the capability of compilers to take poor code and to transform it to somehow less poor code, especially when the language in question is C, that always required mastering); or points to discussions of compiler authors who are borderline insane (when you see them discussion about how technically the standards would allow to not consider uint8_t and char as alias, and they ask themselves if they could not "optimize" more thanks to that, you clearly understand they have completely lost it, and that things will end up badly); or points to bugs "exposed" by that class of optims (I would even say introduced, because 1) when your binary have been OK on all your target platforms for a few decades, and a new compiler come and break it for a technicality, it can be argued that regardless of said technicality the problem is mainly with the compiler; and 2) compilers compromise e.g. for technically invalid benchmarks, so they have no credibility when they don't take into account major breakage in major software, they are just acting like spoiled child who just want to do what they want regardless of the consequences).
What is never addressed by C/C++ compiler authors and their fan base is why the "nonportable" part of the standard is not taken into consideration anymore. This was the original spirit of UB. NOT: "this will permit EXTRA optimization in abstract compiler code"; the only "optim" were because of differences between target processors, and leaving things UB could yield a better mapping to the instruction sets (a trivial mapping) and it was considered that the programmer knew about them, and could use them when needed.
Now there is no mapping anymore. We all have to target the common denominator of all the existing target past, present, even dead ones. The result is a very very very poor language. Unsuitable to more and more things.
> If the code really does make no sense, then why not disallow it or capture it with a compiler warning?
You answered your own question:
> Isn't it possible that the programmer knows something that the optimizer is unaware of?
Here's a fun exercise. Find the best static analyzer you can. Run it on some substantial body of C code. Count all the false positives and false negatives (good luck).
Now you should have an idea as to why you can't simply require the compiler to reject the code and issue a diagnostic. But making code like this UB is about as close you can get to "disallowing it."
Alternatively you could try to eliminate the problem entirely by removing the whole concept of uninitialized memory and requiring that all variables are initialized to some value per default. Depending on which camp you're in, this is a step forward or a step back. Some people just see the words UB and think it is Satan, bad bad bad. For them this is a step forward.
But if all variables are supposed to have a definitive starting value, compilers & static analyzers suddenly can't warn you about the cases where it can tell you forgot to initialize something, because all of a sudden such code is totally legit. The analyzer can't tell whether you forgot to make an initialization or whether you're actually intentionally relying on the default value.
That's a big step back as far as I am concerned. You eliminate a problem because "UB is bad!" and create another problem we can't even issue warnings for, without potentially generating loads of false positives.
Now if one wanted to complain that his compiler detects an obvious case of UB and doesn't warn him about it, he should take it up with the compiler developers (or see if he can help himself by turning on the right flags). Alternatively one could invest in a good static analyzer. The standard committee can't really fix this problem without making substantial changes to the language. Instead I'm glad they allow relatively simplistic implementations that don't do deep advanced analysis.
> Now you should have an idea as to why you can't simply require the compiler to reject the code and issue a diagnostic.
It is quite a different beast, but C# produces compile time failures when attempting to read from a potentially uninitialised variable. What is it that makes C different?
Emphasis on potentially. I did not mean to imply that it is technically impossible to "amend" the standard in the C# direction (but complete analysis with no false positives and no false negatives is impossible!).
However, requiring complicated (yet inherently incomplete, in the sense that it must have "unknowns") data flow analysis and then requiring the compiler to reject plausibly correct programs would be a massive departure from the guiding principles of C standardisation.
In particular, and I quote from the C99 rationale:
> Existing code is important
Requiring correct existing programs to be rejected is not compatible with this principle.
> Keep the spirit of C. [..] Some of the facets of the spirit of C can be summarized in phrases like:
> Trust the programmer.
> Don't prevent the programmer from doing what needs to be done.
> Keep the language small and simple.
> Make it fast, even if it is not guaranteed to be portable.
Complicated mandatory analysis that rejects correct programs pretty much violates every single one of those facets: don't trust the programmer, assume his program is wrong if you can't prove it right. Reject correct programs and in doing so, get in the way of the programmer trying to do what needs to be done. Complicated analysis is not making the language smaller and simpler. Dealing with the shortcomings of said analysis may require unnecessary code (initializations) that are nothing but a performance loss.
> Codify existing practice to address evident deficiencies.
The C standards have always had a focus on codifying existing practice. What are the C implementations that have this practice you propose today?
> Minimize incompatibilities with C90 (ISO/IEC 9899:1990). It should be possible for existing C implementations to gradually migrate to future conformance, rather than requiring a replacement of the environment. It should also be possible for the vast majority of existing conforming programs to run unchanged.
Earlier it was stated that existing implementations are not important, in contrast to existing code. However, in light of this paragraph, it is not supposed to be taken to its logical extreme. Definite assignment analysis can require potentially major changes to existing implementations.
> Maintain conceptual simplicity. The Committee prefers an economy of concepts that do the job. Members should identify the issues and prescribe the minimal amount of machinery that will solve the problems. The Committee recognizes the importance of being able to describe and each new concepts in a straightforward and concise manner.
Interpret this how you will. Having spent a lot of time reading the C spec drafts, I would argue that describing the equations for assignment analysis would look rather complicated and out of place. Because C just doesn't have that sort of complicated machinery.
Basically, this would require substantial changes as I said in my previous post. And these changes would not be in line with the spirit of C.
Why do people want to strong-arm C into being something that is not? We use C because of what it is, not becuase of what you want it to be. If you want a new language, with a different spirit and different guiding principles, why don't you go use D or C++ or Rust or whatever, or invent a new language? I don't see a reason to break C to please all the C haters who wouldn't use the language anyway.
"12. Trust the programmer, as a goal, is outdated in respect to the security and safety programming communities. While it should not be totally disregarded as a facet of the spirit of C, the C11 version of the C Standard should take into account that programmers need the ability to check their work."
I think it's a stretch to read programmers need the ability to check their work as don't trust the programmer.
They codified interfaces with bounds checking. They gave us tools. They deprecated a function that cannot be checked and used securely inside a program without also checking what happens outside the program.
That's good, they give the ability, they give tools. But that's not at all distrusting the programmer, least of all in the sense that I spoke of: assume the program is wrong if you can't prove it right.
And because of that, I do not think these updated guidelines are a reversal of spirit or incompatible with what C alway was and still is.
They still trust the programmer in general, and also allow the use of older interfaces that do no bounds checking. They just gave new toys for the programmer who doesn't trust himself. That may help him check the work.
Still, making the compiler do whatever the hell it wants because "hey it's undefined behavior so we have the license to" is just as idiotic. Make the compilation fail and then add a flag to override for those who feel extra smart.
The compiler is not doing "whatever the hell it wants". It chooses an interpretation of the undefined behavior and then optimizes based on that. Of course, if the programmer had something else in mind, they will be surprised about the result; but it's their fault for not being precise enough.
So why make reads of uninitialized values undefined at all? Consider code like this:
uint64_t x; // Will be initialized later
/* ... */
if(check_some_condition(y, z)) x = (uint8_t) f(b,d);
/* ... */
if(check_some_other_condition(foo)) x = (uint8_t) bar;
/* ... */
x &= 0xf;
/* ... */
if(x & 0x800) do_stuff();
Now the compiler has no way of knowing whether x will be initialized or not. But if it has been initialized, then it's value must be in 0..255, so the & 0xf can be limited to the lowest byte. But this means that the test for x & 0x800 will always be false, and stuff will never be done, so the compiler can optimize it out, reducing code size and thus cache pressure.
If these assumptions don't hold, then some later code may get passed a value for x that has the bit in 0x800 set, and expect do_stuff() to have initialized some data structure, which didn't happen, and the code blows up. But the compiler was just working from what it knew, and under the assumption that the programmer wouldn't depend on completely arbitrary values that happened to be in memory, everything it did was perfectly sensible.
That understandable (but still dangerous). However, considering that reading an unsigned char can reasonably ever be an UB is insanely stupid, considering chars have a special place in the standard. There are going to be (new) problems because of that.
"...It chooses an interpretation ... But the compiler was just working from what it knew, and under the assumption ..."
Why should it do that? Just stop compiling and yell at the user, demanding proper code. I'd not let the code you gave as an example pass any review because it is not clear what the author intended.
That was just an example but I'm starting to have horror visions: imagine you have a struct with some gaps because of alignment, and you read it as chars (which you are supposed to can and must do to get the representation) after having properly initialized all its fields. You might hit an UB just by doing that, maybe not according to the standard for obscure reasons (I hope so), but the probability you are going to get that one day from a compiler developed by a moron^W zealot UB code deletionist is non-negligible (they do have misqualified some valid constructs as UB in the past...)
BTW if my example is actually valid, then that rule of reading uninitialized object being UB is some of the hugest shit, because depending of the type maybe reading an uninitialized byte will be UB, or maybe not, depending of how that byte was in a field of a type or just alignement... In practice, this is even more crazy, because it also depends on which parts of the code your compiler have seen, and it capabilities to break^W understand it, so you might as well considered it has a layer of non-determinism above all that.
C/C++ has become too dangerous. Use something safe instead.
The contents of padding are unspecified after a write; reading it is not UB. Essentially: the padding belongs to the member, and after you initialized the member the padding is initialized as well (you just don't get any guarantee of the value).
This is the reason why padding for wire formats has to be made explicit and be explicitly initialized, otherwise the compiler is free to generate code packing bible quotes into the padding.
> The contents of padding are unspecified after a write; reading it is not UB.
Ok so like I said: "depending of the [effective] type maybe reading an uninitialized byte will be UB, or maybe not, depending of how that byte was in a field of a type or just alignement... "
If I understand correctly this is so insane (to treat those that are UB in that context as effectively UB) that I still have no word.
I absolutely do not want to use a compiler made by people who consider that that situation is OK or even just tolerable for potential gains, regardless of the speedup you could obtain (even at 1000%, I just don't want it -- the risks are way too high)
EDIT: I reread and I think I misunderstood. This is not as bad as I thought, but this is extremely complex and I guess compiler bugs will happen (or have already?) in this area. [ Also what is the relationship with extensions? (like pre-standard or even standard explicit extra alignments...) ]
Actually, it's even more ridiculous than that. Reading with a char * is UB. Reading with an unsigned char is not. Why? Because the C90 standard made it impossible to write memcpy in C so the committee added a hack for unsigned char .
A lot of UB is runtime behaviour that would be difficult to statically analyse. UBSan (-fsanitize=undefined -fno-sanitize-recover) does a good job of detecting it at runtime.
A bunch of UB is of course static, and often comes with compiler warnings. Some UB doesn't have compiler warnings, but is found by linters and analysers (such as clang static analyzer).
UB is pretty damn fundamental to C. Not many people realize it though. It's the fundamental difference between C and C++ compared to Python or C# or Java or Rust or whatever.
The author quotes an article referring to using uninitialized memory for additional entropy and that article cites [10].
Let me just say: DON'T do that, ever. Uninitialized memory is rarely random. If you're lucky, it's quasi-constant because your own program always writes something else to it before. Worst case an attacker manages to write something to that memory before you read it, thereby controlling part of your seed. The cited article also recommends to "In short, just don’t use uninitialized memory for more randomness."
Really, exploiting undefined behaviour in crypto code is a bad idea, and you should make sure you compile with flags that at least catch the most common cases, like the one mentioned in the post.
I can relate to that. I guess undefined behavior makes it somewhat harder to write correct software. I've definitely read someone make the same point.
The problem is, you can't always know if (for example) you're reading from an uninitialized variable, unless you run the program with the possibly infinite number of valid inputs, thereby also solving p != np.
In my opinion, undefined behavior is what we have and we had enough time to learn how to deal with it. If at all possible, compilers should be able to detect UB, so I can use compile flags to make UB an error. Changing the language to "define" undefined behavior, might turn it into unexpected behavior and I wouldn't like that.
At least we've learned the lesson with newer languages.
> At least we've learned the lesson with newer languages.
The thing is, we already knew that from older languages as well. C's UB is a consequence of the people involved in ANSI C89 not wanting to fully define the language, like the other languages were doing.
I think the lesson is that "Performance trumps correctness" is the wrong path to follow, specially since not everyone using C is a 10x coder that always makes use of best practices.
I am 100% in agreement with the C standard committee on this one. Treating all use of uninitialized memory as undefined behavior is a reasonable limitation that gives compiler writers a great deal of flexibility to improve performance. If the only loss is the ability to use uninitialized memory as a source of entropy for random number generators, then that is an incredibly low price to pay for the increased performance.
It's not disallowed by compilers, because they're deficient. In this case I'd guess compiler front-end doesn't track which values in an array are initialized, and when optimizing back-end sees UB it's too late to emit an error.
If your program ALSO compiles VALID programs correctly, and somehow those are faster because it replaces INVALID programs with a nop, that sounds great!
Undefined behaviour was always there to make possible compilers to optimize as they like (nowadays usually for speed or to ease porting to specific architectures) - that is the essence of C "portable assembly" mentality and it always was. Am I wrong?
I tolally agree that this is a bad mentality for software developement in general, but at least it is the "authentic C way" so I feel that part of the critics ungrounded...
Nope. The age of the sufficiently "advanced" (actually: retarded?) compiler was only theoretical during most of the lifetime of C, and UB were actually defined mostly because of differences between processors, NOT compilers. Compilers were faster thanks to their backend.
The standard even says: "undefined behavior:
behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements"
However, today, you should act as if the nonportable part of this statement don't exist anymore, and act as if ALL UB are absolutely forbidden and result in the worst non-deterministic consequences, in all cases and regardless of your actual target. Meaning that today on e.g. x86, if you use a mainstream compiler, you ARE limited by some of the limitations of e.g. some obscure outdated DSPs.
And this is likely impossible to check in a non-trivial program.
So just use another language. C for serious purposes is dead. (some projects that started in that language continue to be developed in it, but given the security issues it creates, this will become unacceptable in a not too far future; so don't wait it is too late to switch: lead that movement.)
I don't think this is true, though I think it's a common misconception. Undefined behavior is not meant to be an optimization tool. It's meant to be an anti-pessimization tool. In other words, it's meant to eliminate overhead in cases where the programmer knows it is unnecessary, but which the compiler cannot prove. Eliminating overhead is conceptually very different from actively allowing insane behavior just to increase optimization opportunities.
Uhm, I'm pretty sure this page is wrong, and the compiler cannot optimize this out. An "indeterminate" value is either an unspecified value or a trap representation. Like they said, the value here cannot be a trap representation, so it's just an unspecified value. An unspecified value is merely an unknown value, NOT a dynamically-mutating value, and NOT a trap representation. So XORing an unspecified value with itself must result in zero, not undefined behavior. (UB would occur with a trap representation, which this isn't.)
> Despite the best efforts of developers of rival programming languages
This is a flippant statement. It could be argued C++ falls into this category, but others are so new that they just haven't had the time to grow into the communities that would benefit from them.
> C’s advantages have preserved it as an indispensable systems and applications programming language.
But what about the disadvantages? I love C, I will always have fond memories of using it to do mind-bending things, but given the choice of using safe languages vs. an unsafe one at this point in time is a mistake.
Where there is an option to use a safe language instead of C, that option should always be chosen. It could potentially save lives depending on where it's deployed; it will definitely save money in the long term with fewer bugs.
C is not the only low cost abstraction and peak performance language available out there anymore.
> C is not the only low cost abstraction and peak performance language available out there anymore.
It never was, it just happened to become one thanks to UNIX's adoption by the enterprise market.
Back in the 80's it was just yet another systems programming language fighting for a spotlight, and any junior Assembly developer could write better code than C compilers for 8 and 16 bit systems.
Oh, the pleasure of using Turbo Basic, Turbo Pascal and Modula-2 and not having to deal with the unsafety issues of C.
Maybe the commitee tries to kill the language. Or maybe it tries to divert from the field the people who routinely abuses the language and tries to outsmart the compiler. Those people will now have to try harder, and maybe at some point this additional time lost on debugging and validation, will start showing up in teams' stats. I say -- good thing.
70 comments
[ 2.7 ms ] story [ 154 ms ] threadhttps://news.ycombinator.com/newsguidelines.html
EDIT: I found this proposal (http://www.cl.cam.ac.uk/~pes20/cerberus/n2091.html) which has some discussion about existing trap representations (including whether to consider signaling NaNs to be trap representations). The conclusion seems to be that segmented pointers on the Motorola 68k are the only place trap representations are truly necessary -- maybe it's time to just remove them.
Then why not just write it like that? This is like deliberately shooting yourself in the foot and complaining that your shotgun works.
Also, the article goes on to argue that uninitialized memory is used to seed random generators, which is generally undefined behavior. I don't think this should be a valid use-case.
I think it's safe to say that x ^= x is equivalent to x = 0 and optimize it that way, just like compilers recognize common idioms for e.g. swapping, counting set bits, etc and emit the correct intrinsic instructions.
C is not self-explanatory at all and without good knowledge of the standard anyone will write broken code that will fall apart sooner or later. C is not a simple language, nor one with simple semantics. Without knowledge of the standards it is essentially not possible to write portable C code.
If the code really does make no sense, then why not disallow it or capture it with a compiler warning?
Instead the optimizer is to silently remove the code.
Isn't it possible that the programmer knows something that the optimizer is unaware of?
Yet that argument is stupid if we are talking about compilers issuing warnings for the UB they "detect", because by definition they have detected them. It might not be possible when it does not work exactly like that in some cases, but at least before removing some code, this is desirable.
It might also not be easy given the current internal design of compilers, but then I argue those design should be changed, because it is just too dangerous.
But pretending that compiler developers are currently just ignoring the issue out of mischief and preference for performance above anything else is disingenous. If the problem were so simple, we'd probably already have a dozen free static analyzers that do a good job, and you'd be happy to use them.
The thing is, "detecting" UB in the sense you imply is usually not what happens in the context of said optimizations (that's not to say they won't attempt to do it... ever seen a compiler warn you about use of uninitialized variable?).
What compilers do is they assume the program is correct. And following that assumption, they perform an optimization that is only correct for correct programs. That is in fact very simple to do. They do not try to prove or disprove that program actually invokes UB there -- that is impossible in general, and even in the subset of cases where it is possible it could require deep whole-program (including libraries!) analysis that could take massive computational resources.
Many people here keep trivializing the problem but I don't think they understand the problem at any depth.
And that is why I say you should pave the way, not in a smug "fuck off get off my lawn fix your own problem" sense, but to get people to honestly gain some background in program analysis, read research papers, study existing analyzers, and gain some appreciation for what it takes. It is far, far from trivial. Especially if you can't just take the language and change it to your liking (breaking nearly all existing code) until all the hard stuff is out.
And in saying that, I suggest that it is easier to start with a new language (or, at least some existing language other than C) that was designed from ground up with such analysis & correctness provability in mind.
But honestly what a waste of resources. I would prefer to be able to work on topics that are not problems created by a recent shift of view of how compiler should be designed and optimize, shift that is for the worst IMO.
And yep I know the reason in the current design for why it is not always easy. That's why I added the caveat that maybe they should be changed.
But then you can not just says that all the users criticizing and "trivializing" the problem are in the wrong and do not know what they are talking about. First, most of the discussion I've seen either points at technical justification of why those crazy optim should not be done, be it for safety, possibility of real optimizations by the programmers (and to compare, I'm not impress about the capability of compilers to take poor code and to transform it to somehow less poor code, especially when the language in question is C, that always required mastering); or points to discussions of compiler authors that are borderline insane (when you see them discussion about how technically the standards would allow to not consider uint8_t and char as alias, a
But honestly what a waste of resources. I would prefer to be able to work on topics that are not problems created by a recent shift of view of how compiler should be designed and optimize, shift that is for the worst IMO.
And yep I know the reason in the current design for why it is not always easy. That's why I added the caveat that maybe they should be changed.
But then you can not just says that all the users criticizing and "trivializing" the problem are in the wrong and do not know what they are talking about. First, most of the discussion I've seen either points at technical justification of why those crazy optim should not be done, be it for safety, possibility of real optimizations by the programmers (and to compare, I'm not impress about the capability of compilers to take poor code and to transform it to somehow less poor code, especially when the language in question is C, that always required mastering); or points to discussions of compiler authors who are borderline insane (when you see them discussion about how technically the standards would allow to not consider uint8_t and char as alias, and they ask themselves if they could not "optimize" more thanks to that, you clearly understand they have completely lost it, and that things will end up badly); or points to bugs "exposed" by that class of optims (I would even say introduced, because 1) when your binary have been OK on all your target platforms for a few decades, and a new compiler come and break it for a technicality, it can be argued that regardless of said technicality the problem is mainly with the compiler; and 2) compilers compromise e.g. for technically invalid benchmarks, so they have no credibility when they don't take into account major breakage in major software, they are just acting like spoiled child who just want to do what they want regardless of the consequences).
What is never addressed by C/C++ compiler authors and their fan base is why the "nonportable" part of the standard is not taken into consideration anymore. This was the original spirit of UB. NOT: "this will permit EXTRA optimization in abstract compiler code"; the only "optim" were because of differences between target processors, and leaving things UB could yield a better mapping to the instruction sets (a trivial mapping) and it was considered that the programmer knew about them, and could use them when needed.
Now there is no mapping anymore. We all have to target the common denominator of all the existing target past, present, even dead ones. The result is a very very very poor language. Unsuitable to more and more things.
You answered your own question:
> Isn't it possible that the programmer knows something that the optimizer is unaware of?
Here's a fun exercise. Find the best static analyzer you can. Run it on some substantial body of C code. Count all the false positives and false negatives (good luck).
Now you should have an idea as to why you can't simply require the compiler to reject the code and issue a diagnostic. But making code like this UB is about as close you can get to "disallowing it."
Alternatively you could try to eliminate the problem entirely by removing the whole concept of uninitialized memory and requiring that all variables are initialized to some value per default. Depending on which camp you're in, this is a step forward or a step back. Some people just see the words UB and think it is Satan, bad bad bad. For them this is a step forward.
But if all variables are supposed to have a definitive starting value, compilers & static analyzers suddenly can't warn you about the cases where it can tell you forgot to initialize something, because all of a sudden such code is totally legit. The analyzer can't tell whether you forgot to make an initialization or whether you're actually intentionally relying on the default value.
That's a big step back as far as I am concerned. You eliminate a problem because "UB is bad!" and create another problem we can't even issue warnings for, without potentially generating loads of false positives.
Now if one wanted to complain that his compiler detects an obvious case of UB and doesn't warn him about it, he should take it up with the compiler developers (or see if he can help himself by turning on the right flags). Alternatively one could invest in a good static analyzer. The standard committee can't really fix this problem without making substantial changes to the language. Instead I'm glad they allow relatively simplistic implementations that don't do deep advanced analysis.
It is quite a different beast, but C# produces compile time failures when attempting to read from a potentially uninitialised variable. What is it that makes C different?
However, requiring complicated (yet inherently incomplete, in the sense that it must have "unknowns") data flow analysis and then requiring the compiler to reject plausibly correct programs would be a massive departure from the guiding principles of C standardisation.
In particular, and I quote from the C99 rationale:
> Existing code is important
Requiring correct existing programs to be rejected is not compatible with this principle.
> Keep the spirit of C. [..] Some of the facets of the spirit of C can be summarized in phrases like:
> Trust the programmer.
> Don't prevent the programmer from doing what needs to be done.
> Keep the language small and simple.
> Make it fast, even if it is not guaranteed to be portable.
Complicated mandatory analysis that rejects correct programs pretty much violates every single one of those facets: don't trust the programmer, assume his program is wrong if you can't prove it right. Reject correct programs and in doing so, get in the way of the programmer trying to do what needs to be done. Complicated analysis is not making the language smaller and simpler. Dealing with the shortcomings of said analysis may require unnecessary code (initializations) that are nothing but a performance loss.
> Codify existing practice to address evident deficiencies.
The C standards have always had a focus on codifying existing practice. What are the C implementations that have this practice you propose today?
> Minimize incompatibilities with C90 (ISO/IEC 9899:1990). It should be possible for existing C implementations to gradually migrate to future conformance, rather than requiring a replacement of the environment. It should also be possible for the vast majority of existing conforming programs to run unchanged.
Earlier it was stated that existing implementations are not important, in contrast to existing code. However, in light of this paragraph, it is not supposed to be taken to its logical extreme. Definite assignment analysis can require potentially major changes to existing implementations.
> Maintain conceptual simplicity. The Committee prefers an economy of concepts that do the job. Members should identify the issues and prescribe the minimal amount of machinery that will solve the problems. The Committee recognizes the importance of being able to describe and each new concepts in a straightforward and concise manner.
Interpret this how you will. Having spent a lot of time reading the C spec drafts, I would argue that describing the equations for assignment analysis would look rather complicated and out of place. Because C just doesn't have that sort of complicated machinery.
Basically, this would require substantial changes as I said in my previous post. And these changes would not be in line with the spirit of C.
Why do people want to strong-arm C into being something that is not? We use C because of what it is, not becuase of what you want it to be. If you want a new language, with a different spirit and different guiding principles, why don't you go use D or C++ or Rust or whatever, or invent a new language? I don't see a reason to break C to please all the C haters who wouldn't use the language anyway.
Maybe you should read the C11 principles instead, and I quote:
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2021.htm
"12. Trust the programmer, as a goal, is outdated in respect to the security and safety programming communities. While it should not be totally disregarded as a facet of the spirit of C, the C11 version of the C Standard should take into account that programmers need the ability to check their work."
They codified interfaces with bounds checking. They gave us tools. They deprecated a function that cannot be checked and used securely inside a program without also checking what happens outside the program.
That's good, they give the ability, they give tools. But that's not at all distrusting the programmer, least of all in the sense that I spoke of: assume the program is wrong if you can't prove it right.
And because of that, I do not think these updated guidelines are a reversal of spirit or incompatible with what C alway was and still is.
They still trust the programmer in general, and also allow the use of older interfaces that do no bounds checking. They just gave new toys for the programmer who doesn't trust himself. That may help him check the work.
That said, I feel like C often lacks the expressiveness to properly define intent and allow the compiler to optimise well.
So why make reads of uninitialized values undefined at all? Consider code like this:
Now the compiler has no way of knowing whether x will be initialized or not. But if it has been initialized, then it's value must be in 0..255, so the & 0xf can be limited to the lowest byte. But this means that the test for x & 0x800 will always be false, and stuff will never be done, so the compiler can optimize it out, reducing code size and thus cache pressure.If these assumptions don't hold, then some later code may get passed a value for x that has the bit in 0x800 set, and expect do_stuff() to have initialized some data structure, which didn't happen, and the code blows up. But the compiler was just working from what it knew, and under the assumption that the programmer wouldn't depend on completely arbitrary values that happened to be in memory, everything it did was perfectly sensible.
Why should it do that? Just stop compiling and yell at the user, demanding proper code. I'd not let the code you gave as an example pass any review because it is not clear what the author intended.
BTW if my example is actually valid, then that rule of reading uninitialized object being UB is some of the hugest shit, because depending of the type maybe reading an uninitialized byte will be UB, or maybe not, depending of how that byte was in a field of a type or just alignement... In practice, this is even more crazy, because it also depends on which parts of the code your compiler have seen, and it capabilities to break^W understand it, so you might as well considered it has a layer of non-determinism above all that.
C/C++ has become too dangerous. Use something safe instead.
This is the reason why padding for wire formats has to be made explicit and be explicitly initialized, otherwise the compiler is free to generate code packing bible quotes into the padding.
Ok so like I said: "depending of the [effective] type maybe reading an uninitialized byte will be UB, or maybe not, depending of how that byte was in a field of a type or just alignement... "
If I understand correctly this is so insane (to treat those that are UB in that context as effectively UB) that I still have no word.
I absolutely do not want to use a compiler made by people who consider that that situation is OK or even just tolerable for potential gains, regardless of the speedup you could obtain (even at 1000%, I just don't want it -- the risks are way too high)
EDIT: I reread and I think I misunderstood. This is not as bad as I thought, but this is extremely complex and I guess compiler bugs will happen (or have already?) in this area. [ Also what is the relationship with extensions? (like pre-standard or even standard explicit extra alignments...) ]
https://archive.is/JmzfZ
A bunch of UB is of course static, and often comes with compiler warnings. Some UB doesn't have compiler warnings, but is found by linters and analysers (such as clang static analyzer).
Let me just say: DON'T do that, ever. Uninitialized memory is rarely random. If you're lucky, it's quasi-constant because your own program always writes something else to it before. Worst case an attacker manages to write something to that memory before you read it, thereby controlling part of your seed. The cited article also recommends to "In short, just don’t use uninitialized memory for more randomness."
Really, exploiting undefined behaviour in crypto code is a bad idea, and you should make sure you compile with flags that at least catch the most common cases, like the one mentioned in the post.
[10] http://kqueue.org/blog/2012/06/25/more-randomness-or-less/
https://blog.regehr.org/archives/1054
The problem is, you can't always know if (for example) you're reading from an uninitialized variable, unless you run the program with the possibly infinite number of valid inputs, thereby also solving p != np.
In my opinion, undefined behavior is what we have and we had enough time to learn how to deal with it. If at all possible, compilers should be able to detect UB, so I can use compile flags to make UB an error. Changing the language to "define" undefined behavior, might turn it into unexpected behavior and I wouldn't like that. At least we've learned the lesson with newer languages.
The thing is, we already knew that from older languages as well. C's UB is a consequence of the people involved in ANSI C89 not wanting to fully define the language, like the other languages were doing.
I think the lesson is that "Performance trumps correctness" is the wrong path to follow, specially since not everyone using C is a 10x coder that always makes use of best practices.
It's not disallowed by compilers, because they're deficient. In this case I'd guess compiler front-end doesn't track which values in an array are initialized, and when optimizing back-end sees UB it's too late to emit an error.
School of thought of programmers in the C language family of systems programming languages, "Performance trumps Correctness".
At least the C++ camp, which has quite a few Algol family refugees, there is some effort to remove few UB from the standard, but not all of them.
In resume, pick your side and don't be surprised for what you get in return.
I tolally agree that this is a bad mentality for software developement in general, but at least it is the "authentic C way" so I feel that part of the critics ungrounded...
The standard even says: "undefined behavior: behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements"
However, today, you should act as if the nonportable part of this statement don't exist anymore, and act as if ALL UB are absolutely forbidden and result in the worst non-deterministic consequences, in all cases and regardless of your actual target. Meaning that today on e.g. x86, if you use a mainstream compiler, you ARE limited by some of the limitations of e.g. some obscure outdated DSPs.
And this is likely impossible to check in a non-trivial program.
So just use another language. C for serious purposes is dead. (some projects that started in that language continue to be developed in it, but given the security issues it creates, this will become unacceptable in a not too far future; so don't wait it is too late to switch: lead that movement.)
https://stackoverflow.com/questions/11962457/why-is-using-an...
edit: i recognised that it does not count as the original example is about an array element...
This is a flippant statement. It could be argued C++ falls into this category, but others are so new that they just haven't had the time to grow into the communities that would benefit from them.
> C’s advantages have preserved it as an indispensable systems and applications programming language.
But what about the disadvantages? I love C, I will always have fond memories of using it to do mind-bending things, but given the choice of using safe languages vs. an unsafe one at this point in time is a mistake.
Where there is an option to use a safe language instead of C, that option should always be chosen. It could potentially save lives depending on where it's deployed; it will definitely save money in the long term with fewer bugs.
C is not the only low cost abstraction and peak performance language available out there anymore.
It never was, it just happened to become one thanks to UNIX's adoption by the enterprise market.
Back in the 80's it was just yet another systems programming language fighting for a spotlight, and any junior Assembly developer could write better code than C compilers for 8 and 16 bit systems.
Oh, the pleasure of using Turbo Basic, Turbo Pascal and Modula-2 and not having to deal with the unsafety issues of C.