The problem comes from the assumption that NULL is the same as zero. Even the author makes that assumption, when he gives an example that explicitly uses zero and then talks about it as if it had used NULL. In fact, the two have never been the same. I've worked on machines that represented NULL with a different value/bitpattern, and yes, it smoked out a lot of these bad assumptions. NULL can never point to valid memory, by language definition. An explicit zero can, from a language standpoint, even if the runtime/OS might disagree. ;) Thus, uses like that in offsetof are quite valid.
Casting zero to a pointer yields a null pointer. That doesn't preclude other-valued null pointers, or ensure that a cast the other way will yield zero. That's where the "undefined" part comes from.
The way I like to think about it is that integer zero is the name in the C/C++ language (and runtime) for the actual null pointer, nothing more or less. In particular you can't assume anything about the integer representation of the actual null pointer.
Casting a constant zero to a pointer type yields a null pointer. The conversion is done at compile time, so the compiler knows that a null pointer is the required result.
There is no requirement for an integer-to-pointer conversion to retain the bit pattern of the integer value, any more than an integer-to-floating-point conversion is required to do so.
In most modern implementations, conversions among integer and pointer types of the same size are trivial, simply reinterpreting the bits as a value of a different type.
A conforming C or C++ compiler can use, for example, the bit pattern 0xFFFFFFFF to represent a null pointer. Converting a constant 0 to a pointer type would yield a pointer with the representation 0xFFFFFFFF. Converting a non-constant 0 to a pointer type does not necessarily yield a null pointer; the result is implementation-defined, and may be indeterminate garbage.
The constant 0 has to convert to NULL, but nonconstant zeroes don't, and NULL doesn't have to be 0 when converted back to an integer. This would break code like the example offsetof implementation.
No, the problem comes from the assumption that you can legally fetch the "address" of a structure member via pointer dereference without checking for null so long as you don't actually try to access the resulting address.
struct usb_line6 *line6 = &podhd->line6;
The assumption is that this is OK so long as you don't try to access the contents of line6 without doing some upstream null checking first. This assumption is false. The very act of dereferencing is undefined, regardless of whether you access the resulting address or not.
The assumption fails whether null is implemented as 0, 200, 0x8000000000000000, 0xffffffffffffffff.
Sure, &foo->bar looks like foo + offsetof(struct thingity, bar), but there is no pointer arithmetic involved since C doesn't specify how the member access ( -> or . ) is actually calculated.
It does however specify that [] operator is equivalent to pointer arithmetic. But we are talking ->,. operators and not [].
Most underlying implementations won't actually dereference here (which is why it usually works), but as far as the spec is concerned, you are dereferencing it, which is undefined.
That becomes a problem with modern optimizing compilers, because they make assumptions about your code based on what's allowed according to the spec.
By not having null pointers. If you want to have a pointer that might not point to anything valid, you do that explicitly with an option type. Of course if you really need to use a possibly-invalid pointer for performance you can drop into unsafe mode.
(At least, the above is what I'd expect from a language like Rust)
But when you do need unsafe code (e.g. for defining a structure that interacts with a C API), is behavior undefined? I’m just looking at a bit of my own offsetof calculation and hoping it won’t break once I turn optimizations on.
unsafe extern "stdcall" fn callback(arg0: *const IUnknown) {
let myself: &MyStruct = &*(arg0.offset(-(&((*std::ptr::null::<MyStruct>()).as_event_handler) as isize)) as *const MyStruct);
}
Yes, it generally is. Rust has similar (but not the same) undefined behaviours to C/C++, the compiler just ensures that they can't happen in safe code.
Currently, by having a single implementation, so that there isn't really a "Rust standard" besides what this or that version of the Rust compiler does and what the documentation says, and if there's a discrepancy, it's a documentation or compiler bug.
There might one day be a "Rust standard", but then let's all wish the standardization process leads to something like the Ada Language Reference Manual instead of the mess that is ANSI C.
Rust does not have undefined behaviours in the sense of C. C and C++ are rather unique in the very large amount of undefined behaviour that is in common use. A nice comment pcwalton https://news.ycombinator.com/item?id=7601944 gives a better explanation than I can.
The undefined behaviour in C is in my opinion a bad design that unfortunately poisoned a whole lot of software for minor gains in speed somewhere. A mistake that did not need to have become popular but for chance in history.
The author may be correct, which is of course the best kind of correct. Yet I have written code for more than 25 years that depends on the behavior exemplified in the `offsetof()` macro referenced there. When writing low-level code it's really handy to know the offset of a member in a struct.
GCC advances frequently break code exploiting undefined behavior that had been working in a certain way for years. "Works as you'd expect" is certainly one of the possibilities of undefined behavior, after all, but that's no guarantee the next compiler version will continue to work as you'd expect.
In fact I'm pretty sure a GCC update broke this exact type of code in the Linux kernel, by effectively removing null pointer checks after the optimizer used its knowledge of a pointer use later in a function to assume that the pointer must be valid and non-null (after all, if it had been null, the behavior would be undefined, and the compiler is allowed to assume no undefined behavior happens, which means the earlier null pointer check had been redundant).
I believe that got fixed in GCC to do what you'd expect, but that's just one example of the awesomeness of undefined behavior in C-based languages.
offsetof is part of C89 stddef.h, why would you call the underlying GCC-specific __builtin_offsetof? Just call offsetof that's what it's here for, __builtin_offsetof is the GCC implementation detail.
So have many people. But from my experience, it is not a fun day when you find out the new compiler version you upgraded to 3 months month ago have "miscompiled" your code for the last 2 releases of your codebase, since it contained undefined behavior which worked fine for the past 15 years when using older versions of the compiler.
This article highlights yet more "gotcha" optimizations that, IMHO, do more harm than good. There is no reason that any compiler must miscompile &x->y, where x == NULL. What's undefined in C can be well-defined in POSIX or in the documentation of a specific build environment. It would behoove higher-level standards to make this construct well-defined even if C itself doesn't.
I think POSIX almost always specifies strict additions in the form of function declarations and expected behavior of those functions, and some generic system-wide features.
Your suggested change would amount to a delta on the language spec. That would probably be unprecedented.
That's a reasonable way to do things, but only if it is not a misappropriation of resources and effort. If someone is writing C code, it is generally inadvisable to rely on behavior that is implementation specific, even if your implementation does nice things. That would violate portability and readability, both things that are nearly unanimously considered to be ideals of programming practice.
So really, the "something sane" that you are talking about should be issuing an error or warning.
I don't see the difference between saying "this is valid C99 code" and "this is valid 'paranoid C' code". At least not if your 'paranoid C' compiler is working to a specification.
People would compile "paranoid C" with a normal C compiler (perhaps there is no paranoid C compiler for their platform). If your code is silently broken on some compilers, that's arguably even worse than being silently broken on all compilers. It would be better to a) make a compiler that errors out on any undefined behaviour (so that people would fix it in their code and it would then be fixed everywhere they compile) and/or b) use a new language.
"Paranoid C" would be a new language. It's just far easier to define a new language in terms of an extension to an existing language than making a new one from scratch. No need to throw the baby out with the bathwater. I mean, C++ started as an extension to C in the same way.
The reasoning for having it an extension is mainly so that current code can be trivially ported.
If you permit calling C libraries from "paranoid C", that would give a very confusing debugging experience, where the semantics of identical code was different depending on which file that code happened to reside in.
If you don't... maybe. But do you realistically think "paranoid C" would ever get its library ecosystem up to a level where it matched e.g. OCaml? Do you think embedded vendors would be more likely to support it (the other big use case for C)? If you're going to be switching libraries anyway, OCaml is a much nicer language to work in.
(On a side note. I like OCaml, but don't like the syntax. I wish there was a language out there that was basically OCaml with c-like syntax. Or rather, I wish language syntax in general was less coupled with language semantics. It should be possible, and is, to, for example, automatically convert between Pythonic-style indentation and C-style braces and lisp-esque braces. Define the language in terms of an AST and leave the syntax up to the editor.)
Also, I'd argue that half the time trying to write things in a portable manner in C is drastically less readable. Look at the madness that ensues when you try to do something as simple as a bitwise rotation, for instance.
Being able to assume that a pointer is not null opens up some really interesting optimizations. For example, if you know that a pointer is dereferenceable, then you can speculatively hoist loads of it out of a loop as long as there are no possibly-aliasable pointer stores in between. That's why it's helpful for C compilers to be able to aggressively infer that pointers are not null.
Just wondering: how much do these particular optimizations end up being worth in the end? Is it something noticeable or is it like array bounds checking where its almost always OK to leave the checks in the end?
It's impossible to give a general answer. You could have a tight loop in a performance critical part of your code, where optimising out the NULL check removes (say) 25% of the loop code. Or, the check might be negligible. It depends upon the code.
That's why IMO people saying that the compiler shouldn't bother with these 'excessive' optimisations are wrong. It might be worthless for them and their programs, but might be valuable for others. Don't force your priorities upon other people.
Yes. I'd also add that loop optimizations tend to magnify in importance when you consider that the loop might be vectorized. Especially since the state of scatter/gather SIMD intrinsics is so poor at the moment...
... &p[1].foo ...
for (int i = 0; i < ...; i++) {
bar += p[0].foo;
}
The compiler wants to transform the loop as follows:
auto tmp = p[0].foo;
for (int i = 0; i < ...; i++) {
bar += tmp;
}
Assume that the compiler cannot prove that the loop will execute at least once. Now observe that this transformation is correct if and only if the pointer p is dereferenceable (with the offset of the field foo applied). The optimization would miscompile a perfectly valid program if p were NULL and the loop ran zero times. And because of strict-aliasing rules, the only way that p would not be dereferenceable in this way is if it were NULL or one element beyond the end of an array.
So the compiler wants to prove that p is not NULL upon entry to the loop. And in this program, it can--because of the address computation, the behavior is undefined if p is NULL or one element beyond the end of an array. Therefore, the compiler can safely assume that p is not NULL and perform the loop invariant code motion. In this program, it can only do so thanks to the rule about undefined behavior in address computations.
Note that I'm not aware of any compiler that actually does this, because the C++ standard committee seems to consider this undefined behavior an oversight in the spec (per the discussion linked there). But the OP successfully argues that it could, per a lawyer-y reading of the spec.
Edit: Argh, I missed the case in which p was one element beyond the end of an array. Updated.
I like your detailed example, but wouldn't a slightly kinder but equally optimizing compiler simply add a conditional before the load:
if (max > 0) {
int i = max;
int tmp = p[0].foo;
do {
bar += tmp;
} while (--i);
}
I'd even argue the assembly is slightly better when interpreted this way, since branch prediction will tend execute the load or not as appropriate, rather than potentially taking a unnecessary TLB miss for a load that's never used.
It's not necessarily a "miscompile". C is supposed to be portable to even completely stupid architectures, and on some architectures, just loading an invalid pointer value into a register will cause a trap, even if you don't dereference it. So on those machines, if x is NULL, then an unoptimized compile of ((size_t)(&x->y)) will result in code that crashes. So that's why it's undefined.
(I'm pretty sure POSIX does impose certain non-stupidity requirements on its underlying architecture, so there are things that are undefined in plain C that are guaranteed to work on any POSIX platform. But I don't know if this is one of them.)
As it stands now, C might outlive most, if not everyone in this thread. :-)
Although, I don't mind C. I work in it every day and I've comfortable with it. A colleague of mine (who works primarily in Python) teased me it's because I have developed a Stockholm syndrome.
The standard offsetof macro is sometimes defined as:
#define offsetof(st, m) ((size_t)(&((st *)0)->m))
That expression has undefined behavior, since it dereferences a null pointer. That doesn't mean it's an invalid definition of offsetof for a given implementation.
Saying that it has "undefined behavior" means that the C language standard says nothing about how it behaves. If its behavior for the current compiler happens to satisfy the language requirements for offsetof, then it's a legitimate implementation. Code that implements the standard library is free to do whatever it likes, as long as the resulting behavior is correct.
Part of the problem, I think, is the way the question is phrased. Quoting the linked article:
> The programmers' community divided into two camps. The first claimed with confidence that it wasn't legal while the others were as sure saying that it was.
The word "legal" doesn't even occur in the C standard. Some program constructs violate constraints or syntax rules; any such violation requires a compile-time diagnostic (which may be a non-fatal error). Other constructs have "undefined behavior"; the standard places no restrictions on what compilers can do for such constructs. Only a `#error` directive actually requires a program to be rejected.
Strictly speaking, yes, it is, at least in the abstract machine.
If a unary "&" is applied directly to the result of a unary "" operator, then neither is evaluated. But in this case, the "&" is being applied to "((st )0)->m". "x->y" is syntactic sugar for "(x).y", so this is equivalent to If a unary "&" is applied directly to the result of a unary "" operator, then neither is evaluated. But in this case, the "&" is being applied to "((st )0)->m". "x->y" is syntactic sugar for "(x).y", so this is equivalent to "((st )0)).m)".
A compiler will very likely optimize the expression so it doesn't generate machine code to dereference the null pointer. And in fact using this definition for the offsetof macro depends on that optimization, and on the compiler treating a null pointer as address 0.
> the standard places no restrictions on what compilers can do for such constructs.
But it does list some suggestions on what could happen, one of which is "behaving during translation or program execution in a documented manner characteristic of the environment", which is what most C programmers expect of the language. Violating this implicit assumption is what makes compilers user-hostile; choosing the worst possible behaviour, just because the standard does not define it, is never the right thing to do.
That suggestion is pretty vague; the only way a compiler could violate it is by not documenting its behavior. The behavior itself can still be completely arbitrary.
Before the first ANSI standard, we had the C Reference Manual in the back of K&R1. The definition was merely less rigorous (and that lack of rigor often led to complex nests of "#ifdef"s). If a compiler generated code that evaluated 2 + 2 and yielded a result of 5, we could confidently say that compiler had a bug.
> Saying that it has "undefined behavior" means that the C language standard says nothing about how it behaves. If its behavior for the current compiler happens to satisfy the language requirements for offsetof, then it's a legitimate implementation. Code that implements the standard library is free to do whatever it likes, as long as the resulting behavior is correct.
Undefined is not the same as implementation-defined. Yes, a given implementation is free to do whatever it wants, but code that includes a null pointer dereference is not C code.
I'm sure he's right, but I'm not so sure he actually proved it. The real question is what happens when you deference a null pointer with '->'. He doesn't show us what the spec says on that, meaning there's a gap in his logic, leaving the quoted sentence in bold unsupported. I don't know but would bet that bolded sentence isn't even necessary. The undefinedness probably comes from the silence of the spec on what happens when you dereference a null pointer with '->'.
As he showed, a null pointer cannot point to any object, and since a lvalue that does not designate an object gives undefined behavior, saying '* podhd' would clearly give undefined behavior when 'podhd' is a null pointer.
However, instead of '* podhd' evaluating to a non-existant object, we have 'podhd->line6' asking for the object some particular offset from a non-existant object. It's entirely possible that a sufficiently insane spec might actually define such an operation.
What the spec says on that point is the key. That's where the undefinedness must come from, but he doesn't talk about it or quote from the spec on '->'. He just assumes it, which leaves his original point unproven, even if it is correct.
So why would anybody want to define legal behavior for dereferencing a null pointer with '->'? Well, some people start college funds for children who have yet to be conceived. If non-existant people can have account balances, why can't non-existant objects have properties? It's crazy, but it's not crazy like Intecal, where syntax errors are legal statements that you actually use for productive purposes. http://catb.org/~esr/intercal/ick.htm#Syntax-Error See also the statement "COME FROM".
> As he showed, a null pointer cannot point to any object
From my reading, he didn't even show that:
>> If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function.
So a null pointer could point to an object, it just wouldn't compare equal to any pointer, even itself.
This is ridiculous. The C standard was apparently written in order to allow compilers to introduce subtle security holes into previously-working code, all in the name of a fraction of a percent of performance increase. It’s totally irrational, especially in an epoch where C is no longer the language you’d use to get maximum bit-twiddling performance anyway — because it won’t run on your GPU!
You've absolutely misrepresented the problem, presumably for effect. It's a far more subtle problem; the question is whether compiler writers should not only make their analyses and transformations be safe in the presence of valid code, but also safe in the presence of invalid code.
This is ironically mainly a problem because older compilers acted normally on some undefined code. So more modern optimising compilers will expect defined behaviour and so forth, leading to new bugs reported in old code. These bugs were always present, they just appear only now.
It's not the compiler trying to exploit the undefined behaviour deliberately; it's that the compiler is expecting the behaviour to be defined and behaving accordingly. So if you have
int x = *foo;
if (foo == NULL) {
ERROR();
}
then clearly in valid C, the if statement will never be taken, so it's safe to remove it. This is something you'd want to happen in most cases, it just so happens that in the above case you might not want it to.
In terms of whether the fraction of a percent is worth it, well the people who do most of the development of the compilers is large companies like Google, and they don't do it out of the goodness of their hearts, they do it because the benefit of having high performance compilers is worth the expenditure.
In terms of whether other languages are better, of course they are for many tasks, but C is king of the embedded world, where implementing a full Rust/Java/Node/language of day compiler is too much work. Then for many high performance tasks it's very difficult to beat the sheer performance of C or C++. If a task written in C or C++ runs just 20% faster than Java for Google/Microsoft/etc's 40,000 computer cluster, then that's 7000 computers that you don't need to buy.
For the bit about GPUs, well that's kinda an aside but GPUs are useful for some things and not others and appealing to their use suggests ignorance. People use C for lots of things, and 'high performance bit-twiddling' isn't a standard one. GPUs are terrible at most tasks.
The proposal for a friendly dialect of C is useful, but I suspect that the smarter way of doing it is the [0] undefined behaviour sanitiser, which produces an instrumented binary that errors when it reaches undefined behaviour.
The developer, upon finding a new bug in their code, compiles, runs, gets an error, fixes, rinses, repeats.
> This is ironically mainly a problem because older compilers acted normally on some undefined code.
As they should. C and C code predates all of the standards, so ALL that C code was, at the time, "undefined". There were still working compilers and working code, how could that possibly be??
Your accusations of dishonesty are completely unfounded, and merely serve to demonstrate that you haven’t bothered to read the links I provided, and therefore have a very limited grasp of the situation. Please retract them.
You are framing the problem in terms that make the situation sound almost reasonable: given that this code is invalid, what should the compiler do? In fact, the question I asked is whether particular code should be invalid.
The choices made by the standards committees so far on this question — the one you are trying to divert attention from — have proven catastrophic. This is far from being a “misrepresentation”; it is patently obvious. We have reached a situation where every non-trivial execution of GCC (which is written in C, remember) executes undefined behavior; and where even zlib, which has gone nine years without a security hole, is full of undefined behavior.
> The C standard was apparently written in order to allow compilers to introduce subtle security holes into previously-working code, all in the name of a fraction of a percent of performance increase.
Yes it was. Or at least, that's the constituency C currently has - and understandably C continues to serve it. There needs to be a language for that kind of people, and that language is C.
If you don't want that, stop using C. It's not hard.
Your suggestion is ridiculous. First, it wouldn’t be enough to stop using C; I would also have to stop using software written in C. And C++. Second, there aren’t any reasonable alternatives to C — it’s not the only portable low-level language, but it’s the only popular one, other than C++, which shares the same problems.
No, it wasn't written for that purpose at all. Many of these undefined behaviours exist in the spec because long ago, there was some exotic architecture where doing that thing caused a hardware exception or behaved strangely. For instance, on some systems that are no longer in use, trying to obtain the pointer in this example would cause an access violation and abort the program because pointer calculations are checked in hardware. It was never intended as an excuse for compiler writers to randomly decide to be dicks.
There’s that, and then there’s also long-ago exotic architectures where you weren’t processing untrusted data, you really did care about improving your performance by ½%, and it really was worth the massive amount of programmer time needed to debug the resulting problems. There’s a famous apocryphal quote that writing C for the Cray (?) compiler was like “writing to [the programmer’s] ex-wife’s lawyer”.
Also, there are still Unisys Clearpath/MCP systems in use.
Ah, the wonders of C, where the obvious solution to "we can't guarantee <x> because it may cause problems on weird hardware" is "let's cause problems on mainstream hardware instead".
When you have a bunch of experts arguing on the interpretation of the language spec to know if a construction in the most fundamental piece of software on the computer is valid like a herd of clerics around a vulgate deciding if dead newborns can go to heaven, you know you have a cultural problem.
The post hinges on the faulty assumption that calculating the address of an lvalue "evaluates" that lvalue. This is not so.
Any compiler that generates anything other than an add instruction (or lea or similar) is broken, for rather unsubtle reasons.
Or to put it another way: dereferencing a null pointer may be undefined, but arithmetic on any pointer (including null) is well defined, and the code in question is arithmetic, not dereference.
And, yet another broken blog post will make the rounds on the internet every three years and confuse even more people. It's sad.
The post cites the spec. The C standard clearly seems to state that & must reference "an object", and NULL does not reference an object. A "pointer to any object" is distinguished from a null pointer.
You may be mistaking C for a fancy macro assembler.
It's not-- It's a high level language, even though a fairly low level looking one. The obligation of the language is to behave as the spec says, and there may not be a close mapping between your program and what runs on the machine, though there often is.
An "Object" is a concept defined and used extensively in the language specification; "a region of data storage in the execution environment,".
I don't get it, there is no actual dereferencing going on here, It is just basic addition arithmetic between two values, one of them happens to be 0. Why It is illegal or "The program running well is pure luck"?
The C standard does not specify the representation of null pointers, nor does it allow arithmetics with invalid pointers.
Eg the conversion (void *)0 might result in a value with all bits set and member access will overflow.
Another example would be hardware that raises a signal when encountering an invalid value during address calculations.
In practice, the address calculation will work as expected (ie I'm unaware of machines where it doesn't). However, an aggressively optimizing compiler migh treat an expression like foo->bar as assert(foo != NULL), which might lead to unintended consequences in worst-case scenarios.
I find this entire debate absurd. Pragmatic approach would be to refrain from using construct that are known to be controversial. By controversial I mean "there are reasons to believe that under certain conditions this may be dangerous, even though most people agree on what reasonable behavior is in this case". Don't write code that can bite you. The "struct foo *bar = &baz->qux;" is not your child. Kill it by refactoring it into something that's known to be obviously correct.
There's no performance penalty here. The only difference is that you're on the safe side if you don't dereference before the null check. Whether discussed case is UB or not, compilers I've tried (MSVC 2013, GCC 4.9.something) produce the same code in both cases.
Refactoring for the sake of C pedantry rather than legitimate portability concerns is not a pragmatic approach.
If this code were a part of some abstract "spherical horse in vacuum" project that may potentially build on esoteric platforms using random C compilers, then, yes, it might be worth re-factoring. In the context of Linux kernel project given the toolchain used the code is perfectly fine as is.
The debate is silly though, can't argue with that.
Linux kernel has official support for over 20 different ISAs. Some of the cores supported are pretty esoteric: BlackFin, SuperH, Etrax. Most of the problems with optimizing compilers - around UBs and others - come from architecture-specific optimizations. Problem discussed here is not abstract - people writing kernel code encounter these issues from time to time. I did, my friends did, we will be bitten by UBs in the future.
Code my team is working on at the moment was written for a completely different architecture than it is run on right now. Sudden platform shifts happen, like when tsunami affected Renesas' ability to manufacture chips used in automotive. It IS pragmatic to avoid code that can potentially introduce problems in the future. Perhaps not for every piece of software, but low level one? Definitely.
That's why you should be passing input params by reference and not as a pointer. The compiler should be be used to check that inputs are valid references so you don't have to null check everything.
95 comments
[ 1.9 ms ] story [ 172 ms ] threadC standard 3.2.2.3: "An integral constant expression with the value 0, or such an expression cast to type void * , is called a null pointer constant."
C standard 4.1.5: "The macros are... NULL, which expands to an implementation-defined null pointer constant."
See [1] for more info.
[1]: http://stackoverflow.com/questions/2599207/can-a-conforming-...
There is no requirement for an integer-to-pointer conversion to retain the bit pattern of the integer value, any more than an integer-to-floating-point conversion is required to do so.
In most modern implementations, conversions among integer and pointer types of the same size are trivial, simply reinterpreting the bits as a value of a different type.
A conforming C or C++ compiler can use, for example, the bit pattern 0xFFFFFFFF to represent a null pointer. Converting a constant 0 to a pointer type would yield a pointer with the representation 0xFFFFFFFF. Converting a non-constant 0 to a pointer type does not necessarily yield a null pointer; the result is implementation-defined, and may be indeterminate garbage.
See also http://stackoverflow.com/questions/2597142/when-was-the-null...
The assumption fails whether null is implemented as 0, 200, 0x8000000000000000, 0xffffffffffffffff.
-> and . are not arithmetic operands.
If you look at the definition of the -> and [], you might learn something!
[] operator doesn't have any place in this debate.
What @jwatte tried to say is that `->` works like this:
The `[]` does a similar thing:It does however specify that [] operator is equivalent to pointer arithmetic. But we are talking ->,. operators and not [].
That becomes a problem with modern optimizing compilers, because they make assumptions about your code based on what's allowed according to the spec.
(At least, the above is what I'd expect from a language like Rust)
There might one day be a "Rust standard", but then let's all wish the standardization process leads to something like the Ada Language Reference Manual instead of the mess that is ANSI C.
The undefined behaviour in C is in my opinion a bad design that unfortunately poisoned a whole lot of software for minor gains in speed somewhere. A mistake that did not need to have become popular but for chance in history.
In fact I'm pretty sure a GCC update broke this exact type of code in the Linux kernel, by effectively removing null pointer checks after the optimizer used its knowledge of a pointer use later in a function to assume that the pointer must be valid and non-null (after all, if it had been null, the behavior would be undefined, and the compiler is allowed to assume no undefined behavior happens, which means the earlier null pointer check had been redundant).
I believe that got fixed in GCC to do what you'd expect, but that's just one example of the awesomeness of undefined behavior in C-based languages.
GCC has __builtin_offsetof: https://gcc.gnu.org/onlinedocs/gcc/Offsetof.html
Your suggested change would amount to a delta on the language spec. That would probably be unprecedented.
All it's doing is saying that "my compiler uses the freedom to do anything that undefined behavior grants to do something sane".
By definition, anything that would be broken by such a change was already undefined behavior.
So really, the "something sane" that you are talking about should be issuing an error or warning.
The reasoning for having it an extension is mainly so that current code can be trivially ported.
If you don't... maybe. But do you realistically think "paranoid C" would ever get its library ecosystem up to a level where it matched e.g. OCaml? Do you think embedded vendors would be more likely to support it (the other big use case for C)? If you're going to be switching libraries anyway, OCaml is a much nicer language to work in.
(On a side note. I like OCaml, but don't like the syntax. I wish there was a language out there that was basically OCaml with c-like syntax. Or rather, I wish language syntax in general was less coupled with language semantics. It should be possible, and is, to, for example, automatically convert between Pythonic-style indentation and C-style braces and lisp-esque braces. Define the language in terms of an AST and leave the syntax up to the editor.)
[1] http://stackoverflow.com/questions/12887700/can-code-that-is...
That's why IMO people saying that the compiler shouldn't bother with these 'excessive' optimisations are wrong. It might be worthless for them and their programs, but might be valuable for others. Don't force your priorities upon other people.
So the compiler wants to prove that p is not NULL upon entry to the loop. And in this program, it can--because of the address computation, the behavior is undefined if p is NULL or one element beyond the end of an array. Therefore, the compiler can safely assume that p is not NULL and perform the loop invariant code motion. In this program, it can only do so thanks to the rule about undefined behavior in address computations.
Note that I'm not aware of any compiler that actually does this, because the C++ standard committee seems to consider this undefined behavior an oversight in the spec (per the discussion linked there). But the OP successfully argues that it could, per a lawyer-y reading of the spec.
Edit: Argh, I missed the case in which p was one element beyond the end of an array. Updated.
(I'm pretty sure POSIX does impose certain non-stupidity requirements on its underlying architecture, so there are things that are undefined in plain C that are guaranteed to work on any POSIX platform. But I don't know if this is one of them.)
This is another reason I look forward to the day when C is a legacy language.
Although, I don't mind C. I work in it every day and I've comfortable with it. A colleague of mine (who works primarily in Python) teased me it's because I have developed a Stockholm syndrome.
However it doesn't help that C++ allows for a C subset.
We are several decades away from achieving that goal.
Saying that it has "undefined behavior" means that the C language standard says nothing about how it behaves. If its behavior for the current compiler happens to satisfy the language requirements for offsetof, then it's a legitimate implementation. Code that implements the standard library is free to do whatever it likes, as long as the resulting behavior is correct.
Part of the problem, I think, is the way the question is phrased. Quoting the linked article:
> The programmers' community divided into two camps. The first claimed with confidence that it wasn't legal while the others were as sure saying that it was.
The word "legal" doesn't even occur in the C standard. Some program constructs violate constraints or syntax rules; any such violation requires a compile-time diagnostic (which may be a non-fatal error). Other constructs have "undefined behavior"; the standard places no restrictions on what compilers can do for such constructs. Only a `#error` directive actually requires a program to be rejected.
If a unary "&" is applied directly to the result of a unary "" operator, then neither is evaluated. But in this case, the "&" is being applied to "((st )0)->m". "x->y" is syntactic sugar for "(x).y", so this is equivalent to If a unary "&" is applied directly to the result of a unary "" operator, then neither is evaluated. But in this case, the "&" is being applied to "((st )0)->m". "x->y" is syntactic sugar for "(x).y", so this is equivalent to "((st )0)).m)".
A compiler will very likely optimize the expression so it doesn't generate machine code to dereference the null pointer. And in fact using this definition for the offsetof macro depends on that optimization, and on the compiler treating a null pointer as address 0.
But it does list some suggestions on what could happen, one of which is "behaving during translation or program execution in a documented manner characteristic of the environment", which is what most C programmers expect of the language. Violating this implicit assumption is what makes compilers user-hostile; choosing the worst possible behaviour, just because the standard does not define it, is never the right thing to do.
Related reading:
http://blog.metaobject.com/2014/04/cc-osmartass.html
http://thetrendythings.com/read/7057
http://blog.regehr.org/archives/1180
Undefined is not the same as implementation-defined. Yes, a given implementation is free to do whatever it wants, but code that includes a null pointer dereference is not C code.
As he showed, a null pointer cannot point to any object, and since a lvalue that does not designate an object gives undefined behavior, saying '* podhd' would clearly give undefined behavior when 'podhd' is a null pointer.
However, instead of '* podhd' evaluating to a non-existant object, we have 'podhd->line6' asking for the object some particular offset from a non-existant object. It's entirely possible that a sufficiently insane spec might actually define such an operation.
What the spec says on that point is the key. That's where the undefinedness must come from, but he doesn't talk about it or quote from the spec on '->'. He just assumes it, which leaves his original point unproven, even if it is correct.
So why would anybody want to define legal behavior for dereferencing a null pointer with '->'? Well, some people start college funds for children who have yet to be conceived. If non-existant people can have account balances, why can't non-existant objects have properties? It's crazy, but it's not crazy like Intecal, where syntax errors are legal statements that you actually use for productive purposes. http://catb.org/~esr/intercal/ick.htm#Syntax-Error See also the statement "COME FROM".
From my reading, he didn't even show that:
>> If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function.
So a null pointer could point to an object, it just wouldn't compare equal to any pointer, even itself.
See http://blog.regehr.org/archives/761 for a sarcastic take on the problem, http://blog.regehr.org/archives/880 for a more serious description of the problem, and http://blog.regehr.org/archives/1180 for the "Proposal for a Friendly Dialect of C" that proposes to actually solve it.
This is ironically mainly a problem because older compilers acted normally on some undefined code. So more modern optimising compilers will expect defined behaviour and so forth, leading to new bugs reported in old code. These bugs were always present, they just appear only now.
It's not the compiler trying to exploit the undefined behaviour deliberately; it's that the compiler is expecting the behaviour to be defined and behaving accordingly. So if you have
then clearly in valid C, the if statement will never be taken, so it's safe to remove it. This is something you'd want to happen in most cases, it just so happens that in the above case you might not want it to.In terms of whether the fraction of a percent is worth it, well the people who do most of the development of the compilers is large companies like Google, and they don't do it out of the goodness of their hearts, they do it because the benefit of having high performance compilers is worth the expenditure.
In terms of whether other languages are better, of course they are for many tasks, but C is king of the embedded world, where implementing a full Rust/Java/Node/language of day compiler is too much work. Then for many high performance tasks it's very difficult to beat the sheer performance of C or C++. If a task written in C or C++ runs just 20% faster than Java for Google/Microsoft/etc's 40,000 computer cluster, then that's 7000 computers that you don't need to buy.
For the bit about GPUs, well that's kinda an aside but GPUs are useful for some things and not others and appealing to their use suggests ignorance. People use C for lots of things, and 'high performance bit-twiddling' isn't a standard one. GPUs are terrible at most tasks.
The proposal for a friendly dialect of C is useful, but I suspect that the smarter way of doing it is the [0] undefined behaviour sanitiser, which produces an instrumented binary that errors when it reaches undefined behaviour.
The developer, upon finding a new bug in their code, compiles, runs, gets an error, fixes, rinses, repeats.
[0] - https://developerblog.redhat.com/2014/10/16/gcc-undefined-be...
As they should. C and C code predates all of the standards, so ALL that C code was, at the time, "undefined". There were still working compilers and working code, how could that possibly be??
You are framing the problem in terms that make the situation sound almost reasonable: given that this code is invalid, what should the compiler do? In fact, the question I asked is whether particular code should be invalid.
The choices made by the standards committees so far on this question — the one you are trying to divert attention from — have proven catastrophic. This is far from being a “misrepresentation”; it is patently obvious. We have reached a situation where every non-trivial execution of GCC (which is written in C, remember) executes undefined behavior; and where even zlib, which has gone nine years without a security hole, is full of undefined behavior.
Yes it was. Or at least, that's the constituency C currently has - and understandably C continues to serve it. There needs to be a language for that kind of people, and that language is C.
If you don't want that, stop using C. It's not hard.
What are my options if I want a simple, easy to understand language that is close to the execution model of current machines?
Also, there are still Unisys Clearpath/MCP systems in use.
Any compiler that generates anything other than an add instruction (or lea or similar) is broken, for rather unsubtle reasons.
Or to put it another way: dereferencing a null pointer may be undefined, but arithmetic on any pointer (including null) is well defined, and the code in question is arithmetic, not dereference.
And, yet another broken blog post will make the rounds on the internet every three years and confuse even more people. It's sad.
Do you have links stating otherwise?
It's not-- It's a high level language, even though a fairly low level looking one. The obligation of the language is to behave as the spec says, and there may not be a close mapping between your program and what runs on the machine, though there often is.
An "Object" is a concept defined and used extensively in the language specification; "a region of data storage in the execution environment,".
Eg the conversion (void *)0 might result in a value with all bits set and member access will overflow.
Another example would be hardware that raises a signal when encountering an invalid value during address calculations.
In practice, the address calculation will work as expected (ie I'm unaware of machines where it doesn't). However, an aggressively optimizing compiler migh treat an expression like foo->bar as assert(foo != NULL), which might lead to unintended consequences in worst-case scenarios.
In fact most modern optimising compilers do exactly that, and the article does refer to such a behaviour having had security implications in the past.
If this code were a part of some abstract "spherical horse in vacuum" project that may potentially build on esoteric platforms using random C compilers, then, yes, it might be worth re-factoring. In the context of Linux kernel project given the toolchain used the code is perfectly fine as is.
The debate is silly though, can't argue with that.
Code my team is working on at the moment was written for a completely different architecture than it is run on right now. Sudden platform shifts happen, like when tsunami affected Renesas' ability to manufacture chips used in automotive. It IS pragmatic to avoid code that can potentially introduce problems in the future. Perhaps not for every piece of software, but low level one? Definitely.