I've been a professional C programmer for nearly 40 years and didn't do especially well (I got about 2/3rds right). In my view it doesn't really matter. Enable maximum warnings and fix everything the compiler warns about. I also use Coverity once in a while, and every test is run with and without valgrind. If you do that you'll be fine.
Yeah, in practice these sort of gotchas ought to trigger warnings. And for a C/C++ developer, warnings should never be allowed to persist in a code base.
When you write C programs, you are coding on the C abstract machine as specified by the language standard. If your code triggers signed integer overflow, then the C abstract machine says that it's undefined behavior, and the compiler and runtime can make your code behave however it wishes. It doesn't matter that the underlying concrete machine (POSIX, x86 ISA, etc.) has wraparound signed overflow, because your code has to interact with both the abstract machine and the concrete machine at the same time. See: https://www.youtube.com/watch?v=ZAji7PkXaKY
> When you write C programs, you are coding on the C abstract machine as specified by the language standard.
Are you really, though? I would argue that it's a matter of perspective and/or semantics.
The Linux kernel is built with -fwrapv and with -fno-strict-aliasing, and uses idioms that depend on it directly. We can surmise from that that the kernel must be:
1. Exhibiting undefined behavior (according to a literal interpretation of the standard)
OR:
2. Not written in C.
Either way, it's quite reasonable to wonder just how much practical applicability your statement really has in any given situation -- since you didn't have any caveats. It's not as if the kernel is some esoteric, obscure case; it's arguably the single most important C codebase in the world. Plus there are plenty of other big C codebases that take the same approach besides Linux.
Lots of compiler people seem to take the same hard line on the issue -- "the C abstract machine" and whatnot. It always surprises me, because it seems to presuppose that the only thing that matters is what the ISO standard says. The actual experience of people working on large C codebases doesn't seem to even get acknowledged. Nor does the fact that the committee and the people that work on compilers have significant overlap.
I'm not claiming that "low-level C hackers are right and the compiler people are wrong". I'm merely pointing out that there is a vast cultural chasm that just doesn't seem to be acknowledged.
One of these days, the compiler will do something surprising to one of your expressions involving signed integer overflow, like converting x < x + 1 to true. Or it'll delete a whole loop because it noticed that your code is guaranteed to trigger an out-of-bounds array read, e.g.: https://devblogs.microsoft.com/oldnewthing/?p=633
> If you do that you'll be fine.
I would not trust code written based on the methodology you described. However, if you also add -fsanitize=undefined,address (UBSan and ASan) and pass those tests, then I would trust your code.
Almost all of these examples would be sloppy programming in real code. The “int”, “short”, “long”, “long long” and “char” types are generic labels which shouldn’t be used where size and signedness matters. For a guarantee of size and signedness one should use the “[u]intXX_t” types. For sizes and pointer arithmetic - size_t. For subtracting pointers - ptrdiff_t. You simply wouldn’t have most of these issues if you stick to the correct types.
Absolutely. int is a "code smell" for us, except in some well-defined cases (storing file descriptors for example). If someone is using int to iterate a loop, then the code is more usually wrong than right, they should be using size_t.
I don't think a for loop using an int is bad or even "more wrong than right". If anything int is much better than using size_t.
Using an integer, in the 1000s of for loops I've written, none get even remotely close to the billions - it is optimizing for a 1 in a million case, and if I know something can run into the billions of iterations I'm going to pay more attention to anyway. I've seen 0 occurrences of bugs relating to this kind of overflow.
Using a size_t, it is effectively an unsigned integer that risks underflowing which can easily cause bugs like infinite loops if decrementing or other bugs if doing any index arithmetic. I've seen many occurrences of these kind of bugs.
On Linux/x86-64 int is 31 bits, so you've probably introduced "1000s" of security bugs where the attacker only needs the persistence to add 2 billion items to a network input, local file or similar, to generate a negative pointer of their choosing.
Any such code submitted to one of our projects would be rejected or fixed to use the proper type.
I've not introduced a security bug in every for loop I've written. What I've written shouldn't be controversial, just take a look at Googles style guide:
"We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t or uint64_t.
For integers we know can be "big", use int64_t.
You should not use the unsigned integer types such as uint32_t, unless there is a valid reason such as representing a bit pattern rather than a number, or you need defined overflow modulo 2^N. In particular, do not use unsigned types to say a number will never be negative. Instead, use assertions for this.
If your code is a container that returns a size, be sure to use a type that will accommodate any possible usage of your container. When in doubt, use a larger type rather than a smaller type.
Use care when converting integer types. Integer conversions and promotions can cause undefined behavior, leading to security bugs and other problems."
Unfortunately, even if you’re using uint16_t (which, remember, the C standard does not guarantee exists) on a platform with (say) 32-bit integers, you’re still actually computing in signed int due to promotion.
The true types (int, unsigned etc) are correct for local variables, since they describe registers. The sized aliases are correct for struct fields and for arrays in memory.
You should prefer signed types for computing (if not storing) sizes and for pointer arithmetic, since they are more forgiving with underflow.
Mathematically speaking, the mod is usually taken to be positive. Any reasonable definition would involve taking the quotient of Z over the congruence relation, so I can see both -3 and 2 being reasonable conventions to represent partitions.
OTOH, the largest "wat" of C-like languages is the following:
Mathematically speaking if you ask for the modulus of -5 you may as well get a negative number, and I think people may validly have their own interpretations of what's "correct" at this point.
You may as well use {white, blue, black, red, green} to represent congruences mod 5, mathematically speaking it's not wrong, as long as they respect the axioms of a field.
What's "wat" (in the same sense as the js "wat" talk) is that the answer changes if the argument becomes negative. By all reasonable definitions of modular arithmetic, -8 and 7 are in the same class mod 5. Why is then:
> (-8 % 5) == (7 % 5)
false
in C-like languages? I'm pretty sure it's perfectly consistent with all the relevant language standards, but it's a "wat" nonetheless.
> What's "wat" (in the same sense as the js "wat" talk) is that the answer changes if the argument becomes negative.
It's the same in Python though? you just prefer the way it behaves (though to be fair it's generally more useful and less troublesome): Python's remainder follows the sign of the divisor (because it uses floored division), while C's follows the sign of the dividend (because it uses truncated division).
Strictly speaking, neither is an euclidian modulo.
It's not symmetric. I believe most people with strong mathematical background (like Guido:)) expect lambda x: x % m to be some "computer approximation" to the standard mapping from Z to Z/mZ, while not having deep expectations about lambda m: x % m.
The really useful fact of C's remainder operator is that:
x == x / y * y + x % y, provided nothing overflowed. This is usually what you want when doing accurate calculations with integer division.
The modulo operation together with floor division also has this property. And unlike C's operators, it also has the useful property that (x + y) % y == x % y.
That's an inherent property of both the modulus interpretation and the remainder interpretation.
Haskell does it right. Instead of having operators like %, it gives the programmer choice to choose either the modulus or the remainder. Using GP's example:
Fortran supports both of these, with `mod` as the C-like truncated modulo and `modulo` as the floored modulo. Having both is convenient, but you do get errors from people who don't realize the difference
Undefined behavior is probably the worst way we can imagine to define constraints usable by optimizers. Sad that major languages and implementations went this way.
Yeah, I think a lot of C's `undefined behaviour` semantics around assignments to ints should be reconsidered and changed to `unspecified` or `implementation defined` behaviour, and compilers can just do "whatever the hardware does". If that includes traps on one arch, fine, let it trap.
I think `undefined behaviour` still has its place in C - dereferencing a freed pointer comes to mind as an obvious example - but I think a good proportion of the really unintuitive UB conditions could be made saner without sacrificing portability or optimisation opportunities.
I don't think so. With "unspecified" and "implementation-defiend" behaviours, the implementation has to pick a behaviour and be consistent with it. The difference is whether they have to document that behaviour or not.
If the hardware traps on bogus pointers, then reading a bogus pointer may trap. But if you read a recently-freed pointer, it may still be valid according to the hardware (e.g. will have valid PTEs into the processes address space) so won't trap. Therefore you won't be able to guarantee any particular behaviour on an invalid read, so I don't think you'd be able to get away with "unspecified" or "implementation defined" behaviour on most hardware.
AFAIR reading uninitialized int, for example, is "unspecified" (any value could be there). If we consider adding "implementation defined with possible trap" for overflow, we might as well add "unspecified with possible trap" for reading an invalid pointer (any value could be there, or it could trap, but no nasal demons).
Using unitialized values is UB. Going back to naive pointers is a lost cause because compilers have started to do crazy optims (not even currently allowed by the standards...) like origin analysis. You can't steal that toy from the people implementing optims.
"Implementation defined" is worse in a lot of ways. Now the compiler has way less authority to tell you to stop! And we still have the problem of the application probably not doing what you want.
Current compilers don't "tell you to stop". They silently transform your program into complete garbage without even causality restraints. The sanitizers can help but they are merely dynamic when the dangerous tranformations are static in the first place.
It is true that the language has failed to provide tools that help developers prevent UB and that this is a very bad thing for the ecosystem. It can change, though.
In practice, compilers aren't actually adversarial. A lot of the discussion around UB is catastrophizing and talks about how the compiler will order you pizza or delete your disk. Some problems are real and I know some people whose graduate school work was very specifically on the problems that this causes for security-related checks but compilers really really do not transform your program into "complete garbage." They transform your program into a program with a bug, which is a true statement about that program.
I'm reminded of the apocryphal story about being asked how a computer knows how to do the right thing if given the wrong inputs. This feels similar.
> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
I do agree that the story around UB in C and C++ sucks. But UB doesn't exist because the compiler engineers want to be able to stick in optimizations. Once they are in the spec it makes sense to follow the rules but most UB comes from a desire for portability and to not privilege one platform over another.
And further, defining a lot of UB won't actually improve things. Imagine we define signed integer overflowing behavior. Hooray. Now your program just has a different bug. If you've accidentally got signed integer overflow in your application then "did some weird things because the compiler assumed it would never overflow" is going to cause exactly the same amount of havoc as "integer overflowed and now your algorithm is almost certainly wrong."
> And further, defining a lot of UB won't actually improve things. Imagine we define signed integer overflowing behavior. Hooray. Now your program just has a different bug.
So yeah, defining signed integer overflow isn't going to fix bugs in the vast majority of existing programs.
That being said, enforcing signed overflow wraparound at least makes debugging easier because it's reproducible. This is how it is in Java land - int overflow wraps around, and integer type widths are fixed, so if you trigger an overflow while testing then it is reliably reproducible across all conforming Java compiler and VM versions and platforms.
It also makes tools less able to loudly yell at you for having it in your application. Yes, you can turn on optional warnings if you want but if you've got one or two intended uses for this behavior then now you need suppressions and all sorts of mess.
I thought I did well until the bit-shifting questions. Who in their right mind designs a language where shifting the bits of a u16 silently converts into an i32? Doubly so since that fact alone directly causes UB — keeping the u16 would have been perfectly fine.
(Edit, since some respondents seem to miss this, explanations about efficiency or ISAs might justify promoting to u32 (though even that's debatable), but not i32. A design that auto-promotes an unsigned type, where every operation is nicely defined and total, into a signed type, where you run into all kinds of undefined behavior on overflow, is simply crazy.)
Because everything smaller than an int is usually promoted to int. int is the 'word' in C that things are calculated in. Even character constants are ints, all enum constants are ints, the default type was int when default types were still a thing.
In the last statement x is promoted to an int, and then when the logical NOT occurs every bit is set to 1, including the high bit. When it's converted to a uint64_t for the AND, the high bits are also set to 1. So the result is that the final statement clears only bit 2 in *reg.
If it promoted to unsigned int, then it would also clear bits 32-63.
But of course when you write that chances are somebody will point out that you're running in interruptible context sometimes in this function, so that's actually introducing a race condition. Why didn't they say so when you wrote it your way? Because that looked like a single operation and so it wasn't obvious it might get interrupted.
The unsigned preserving rules greatly increase the number of situations where unsigned int confronts signed int to yield a questionably signed result, whereas the value preserving rules minimize such confrontations. Thus, the value preserving rules were considered to be safer for the novice, or unwary, programmer. After much discussion, the Committee decided in favor of value preserving rules, despite the fact that the UNIX C compilers had evolved in the direction of unsigned preserving.
Yes. The model C has is that the CPU has an ALU that that operates as (word,word)->word with int being the smallest word size. This explains many of C's conversion rules: to operate on a single integer, it first has to be promoted to a word size; to operate on two integers, they first have to be converted to a common word size, etc.
And that makes writing a correct and portable C futile. Yes, defined-size types are a thing and I almost exclusively use them, but since the size of `int` itself is unknown and integer promotion depends on that size, the meaning of the code using only defined-size types can still vary across platforms. intN_t etc. are only typedefs and do not form their own type hierarchy, which in my opinion is a huge mistake.
If you have a 32-bit architecture, you don't necessarily have 8-bit and 16-bit hardware operations outside of memory operations (load/store).
Now, I don't find this reasoning persuasive--it's not that hard to emulate an 8-bit or 16-bit operation--and judging from the history of post-C languages, most other language designers are equally unmoved by this reasoning, but I can see someone in their right mind designing a language that acts like this. Especially if the first architecture they're developing on is precisely such on architecture (PDP11 doesn't have byte-sized add/sub/mul/div).
I think the prevailing philosophy for C (and later C++) was that code should map closely to the hardware and not expand to complicated "microcode". A left shift in Cshould just be a left shift in the underlying ISA, within reason. That's where most of the undefined behaviours come from. Having to add masking and other operations to emulate a 16bit shift on a 32bit architecture feels un-C-like, for better or worse.
IMO the real issue is not so much the fact that all shifts of any type < int is treated as if it were an int, it's that the language doesn't force you to acknowledge that in the code. If you got a compilation error when trying to shift a short and had to explicitly promote to int in order to make it through, at the very least it can't lead to an oversight from a careless programmer.
C is trying to be clever but only goes half way, resulting in the worst of both worlds IMO.
The ISA might force someone to extend a value to 32 bits (debatable, but let’s go with it). It never forces you to treat an unsigned int as signed. It also doesn’t require inserting UB into the process.
I agree, the whole signed vs. unsigned generally feels like an afterthought in C (probably because, to a certain extent, it was). `char`'s sign being implementation-defined is a pretty wild design choice that wasted many hours of my life while porting code between ARM and x86.
UBs are not required, but you need them if you want C to behave as a macro-assembler as well as allowing for aggressive optimizations. For instance `a << b` if b is greater than a's width is genuinely UB if you write portable code, different CPUs will do different things in this situation. Defining the behaviour means that the compiler would have to insert additional opcodes to make the behaviour identical on all platforms.
You may argue that it's still better than having UB but that's just not C's design philosophy, for better or worse.
That’s true, I’d add something more. Your reasoning about hardware differences would only justify implementation-defined behavior, not undefined behavior. The distinction is important here: undefined behavior is when the compiler can make surprising optimizations in other parts of the code assuming something doesn’t happen.
It's worse than that. `char`'s signedness being implementation defined is one thing, but then having the standard library provide a function called `getchar()` that returns not a `char` but an `unsigned char` cast to an `int` is diabolical.
> For instance `a << b` if b is greater than a's width is genuinely UB if you write portable code, different CPUs will do different things in this situation. Defining the behaviour means that the compiler would have to insert additional opcodes to make the behaviour identical on all platforms.
Your seem to be mixing up implementation defined behavior and undefined behavior. It would have been perfectly reasonable to make this choice if signed integer ovwrflow were implementation-defined, but it is unfortunately not - it is undefined behavior instead. This means that a program containing this instruction is not valid C and may "legally" have any effect whatsoever.
Third question is wrong. It's implementation defined, because unsigned short might be the same rank as unsigned int in some implementations, in which case it remains unsigned when promotion occurs.
He does say on the first page: All other things being equal, assume GCC/LLVM x86/x64 implementation-defined behaviors. Are there any normal, modern machines where short and int are the same size? I think the last machine where that was true was the 8086.
On x86 systems like 8086 short and int were the same size. And that appears to be a footnote they've added after being called out for being wrong. The question gives "implementation defined" as an option and in other questions seems to specify the ABI, and in some assumes it without saying again. Very inconsistent, they should fix their quiz really.
The last x86 processor I know of to have these sizes is 80286.
Avr microcontrollers still have 16bit ints, probably 8051 and pic too, but I don't use those. Lots of people do though. TI dsp uses 48 bit long, so don't count on int and long being the same either.
That's still a bad question: the behavior is implementation-defined according to the standard, so having "implementation-defined" as a wrong option is ambiguous and confusing.
The question would be fine (given that note) if "impementation-defined" was not an option, like for example the question about "SCHAR_MAX == CHAR_MAX".
> I think the last machine where that was true was the 8086.
The last machine with a word size of 16 bits? The 286 was as well. There were others like the WDC 65816 (the Apple IIgs and SNES CPU).
It just so happens that there simply far fewer 16 bit CPUs then there were 8 or 32 (or “32/16” like the 68k). Also 8 bit CPUs are simply a poor fit for C by their nature and the assumptions C makes. But the numerous ones still relevant today will use 16 bit ints.
The use of Real or V86 mode on the x86 went on for many years after the demise of the 8086. I think it is a somewhat of a joke at this point that they’re teaching Turbo C in some developing countries.
They may have the same width, but not the same rank.
From C99 §6.3.1.1:
> — The rank of long long int shall be greater than the rank of long int, which
shall be greater than the rank of int, which shall be greater than the rank of short
int, […]
> — The rank of any unsigned integer type shall equal the rank of the corresponding
signed integer type, if any.
That's true, but it doesn't matter to the point being made.
According to the standard[1], if short and int have the same size (even if not the same rank) both numbers are converted to unsigned int (that is, the unsigned integer type corresponding to int) because int can't represent all values of unsigned short.
The usual arithmetic conversions never "promote" an unsigned to signed if doing so would change the value.
Neat quiz. Reminds me that the absolute value of INT_MIN in C (and many other languages) is undefined, but will generally still return a negative value. This is a "gotcha" that a lot of people are unaware of.
A consequence of most, if not all, CPUs today using two's complement integers.
I think one's complement is more sensible since it doesn't have this problem, but it loses out because it requires a more complex ISA and implementation.
It's annoying that negation, ABS, and division can overflow with two's complement. But how I look at it: lots of operations can already overflow, just a fact of signed integers, and you need to guard against that overflow in portable code already. It doesn't seem to be fundamentally worse that those extra operations can overflow.
> The name "ones' complement" (note this is possessive of the plural "ones", not of a singular "one") refers to the fact that such an inverted value, if added to the original, would always produce an 'all ones' number
The radix complement in base two. They are closely related but not identical - the two's complement representation of a negative number is the ones' complement representation + 1 (ignoring the final carry). In ones' complment there are two representations for 0 (all 0s or all 1s), but every number has a positive and negative representation. In two's complement, there is a single 0, but there is also negative number that doesn't have a corresponding positive representation (INT_MIN).
In fact, for every numerical base there is an equivalent. For example, in base 3 you can represent a number in twos' complement notation (each trit X is replaced with 2-X, so -3 is represented as 212 on three trits, with +3 being 010) is or in three's complement by adding 1 (212+1 = 220). For base 10, you can do nines' complement (-3 on 3 digits is 996) or ten's complement (996+1 = 997).
The first rule of C good programming "Treat warnings as errors". Compile with all warnings enabled, selectively ignore only warnings you know for sure are not important for program semantics.
Simple MISRA C with all warnings on is already close to language with strict type checking. C compilers give you a choice, use it. If you are programming for money, good static analyzer makes it possible to write safety critical code.
I do agree about compiler warnings, but MISRA C imposes a lot of unnecessary rules, a lot of unhelpful rules on how you write expressions, and tries to also act like C's type system works a different way than it does. In practice I have found it to actually create bugs. Read the MISRA rules and appendices on their effective type model and the C standard, they have gaps where MISRA actually forces you to write code that looks correct and doesn't work with C's model. I strongly recommend against using MISRA, even in automotive or aviation code (although it may unfortunately be a requirement on any such project).
"Although the first edition of K&R described most of the rules that brought C's type structure to its present form, many programs written in the older, more relaxed style persisted, and so did compilers that tolerated it. To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions."
What is the history of "undefined behavior" [in C compilers and the standard] in general? I suppose originally it was supposed to guide compiler engineers, but we all know that backfired, as many compilers try to exploit undefined behavior to optimize code, but that can be problematic in security sensitive code (e.g. if uninitialized memory is optimized away) - there have been discussions between security engineers, kernel developers and GCC hackers about how to implement/interpret the standard.
Would it be possible to have a standard, where undefined behavior is just a compile error? What would we lose - apart from legacy compatibility?
> Would it be possible to have a standard, where undefined behavior is just a compile error?
No [if you're aiming for something in the same vein as C]. Undefined behavior is ultimately an inherently dynamic property--certain values could make a statement execute undefined behavior, and consequently, virtually every statement could potentially cause undefined behavior. Note that this remains true even in languages like Rust: Rust has loads of undefined behavior, but you do have to wrap code in unsafe blocks to potentially cause undefined behavior.
> What would we lose - apart from legacy compatibility?
In particular, it is clear at this point that if you want to permit converting integers to pointers, you will either have to live with undefined behavior (via pointer provenance) or forgo basically all optimization whatsoever.
C sucked on 8 and 16 bit home computers, to be fair, all high level systems programming languages had their own set of issues regarding optimal code generation, thus Assembly was the name of the name for ultimate performance.
UB started as means to not kick out computer architectures that would otherwise not be able to be targeted by fully compliant ISO C compilers.
Given that C prefers to be a kind of portable macro assembler than care about security, it was only a matter of time until those escape hatches started to be taken advantage for optimizations.
Same applies to other languages, however since their communities tend to prefer security before ultimate performance, some optimization paths are not considered as that would hurt their safety goals.
In what concerns C, C++ and Objective-C, dropping UB optimizations would mean going back to the 1990's in terms of code quality.
Unfortunately some of this stuff just can't be detected statically. So while warnings are an excellent starting point, I recommend also building and testing with UBSan+ASan enabled.
That is the correct answer to the questions in this quiz!
Most professionals don't worry about these effects, instead choosing to cast and use parenthesis directly. Those instruct the compiler instead of relying upon warnings from the compiler.
I think that is expected (it’s what happens for x86 as well) what’s surprising about the parent is that long is apparently 32 bit on windows x64. Usually long should be 64 bit on a machine with 64 bit words
Surprised I got all the questions right because I tend to code around those limits, sticking to defining always safe types for the problem at hand, in all the cases I'm not sure about ranges, possibile overflows and so forth. What I mean is that if you have to remember the rules in a piece of code you are writing, it is better to rewrite the code with new types that are obviously safe.
Admittedly my C is (very) rusty, but I was surprised at how few of the answers I knew, so your comment made me feel even worse. But then I saw your nickname. Yeah. :) (redis rocks btw)
Completely agree with you, and not just in C. Using esoteric features of any language is equivalent to putting landmines in front of your teammates - bad idea.
> Using esoteric features of any language is equivalent to putting landmines in front of your teammates
Also, outside of production code, using esoteric features is a good way to get familiar with every corner of that language - which is useful so that you can diffuse those landmines when they are accidentally created. i.e know them to avoid them.
The shorts are not promoted to ints (if you even declared the result as that type) until after the multiplication. The result will first be put into a short and then promoted to int. It's basically asking for an overflow error. Given that 256^2 is 65536 you don't have to be multiplying large numbers before hitting that overflow.
I don't follow, assuming short is half the size of int it would only overflow if most significant bits of both values are 1. Where did that 256^2 come from? It wouldn't overflow if a and b were 256
I missed that the result would be promoted to signed int, which only most significant bits are set
All operands are always promoted to int (or unsigned int, long, unsigned long, etc.) before any operation. The footgun in this example is that even though you're using unsigned integers and would expect guaranteed wrapping semantics, the promotion to signed int makes it UB for USHORT_MAX*USHORT_MAX.
(B), assuming int is 32-bit and short is 16-bit. The multiplication promotes both operands and result to signed int, right? So if both x and y are uint16_max, the result overflows signed int and is UB, I think.
But if int were larger (eg 64 bit) and short remained 16 bits, there’s no overflow and the answer is (A). I think.
If short = 16 bits and int = 16 bits, then x and y will be promoted to unsigned int. Unsigned multiplication has wraparound behavior, so x*y will be defined for all values.
If short = 16 bits and int = 32 bits (or heck even 17 bits), then x and y will be promoted to signed int. Signed multiplication overflow is undefined behavior, so x*y will be undefined for some values when x*y is too large. In particular, 0xFFFF * 0xFFFF = 0xFFFE_0001, which is larger than INT_MAX = 0x7FFF_FFFF.
If short = 16 bits and int = 64 bits (or even 33 bits), then x and y will be promoted to signed int. The range of x*y will always fit int, so no overflow occurs, and the expression is defined for all input values.
One of my friends likes to scold me about using a programming language (Python) that doesn't enforce type declarations. I gently remind him that his language (C) has eight different types of integers.
For char, you have 3: signed char, unsigned char and char. It's not specified if char without keyword is signed or unsigned.
You have integer types such as size_t, ssize_t and ptrdiff_t. They may, under the hood, match one of the other standard int types, however this differs per platform, so you can't e.g. just easily print size_t using the standard printf formatters, you really have to treat is as its own type. Also wchar_t and such of course.
Then you have all the integers in stdint.h and inttypes.h. Same here applies as for size_t. At least you know how many bits you get from several of them, unlike from something like "long".
Then your compiler may also provide additional types such as __int128 and __uint128_t.
I taught first-semester C for several years, and still got a couple of these wrong - although, to be honest, the way we taught the class at my alma mater we steered well clear of these situations. We told students that C perform type promotions and automatic conversions, but either had them use a uniform type to begin with - int typically - or told them to convert explicitly and avoid
Still, the most important suggestion I would make here is: Always compile with warnings enabled, in particular:
* -Wstrict-overflow=1 and maybe even -Wstrict-overflow=3
* -Wsign-compare
* -Wsign-conversion
* -Wfloat-conversion
* -Wconversion
* -Wshift-negative-value
(or just `-Wall -Wextra`)
And maybe also:
* -fsanitize=signed-integer-overflow
* -fsanitize=float-cast-overflow
(These are GCC flags, should work for clang as well.)
> What does the expression SCHAR_MAX == CHAR_MAX evaluate to?
> Sorry about that -- I didn't give you enough information to answer this one. The signedness of the char type is implementation-defined
...why not have an "implementation-defined" answer button then, because that's what people should know (instead of knowing all ABIs) and what the question is about anyway?
> If these operators were right-associative, the expression [x - 1 + 1] would be defined for all values of x.
That's just wrong, no? If + and - were right-associative, it would be parsed as x - (1 + 1), which is decidedly not "defined for all values of x".
I raised an eyebrow on that as well, in some questions you have implentation defined for an answer so I assumed the author just wrongly assumed undefined covers that
Even the compiler assumption isn't enough. There is also an implicit assumption in the answers that x86-64 is using LP64 when some platforms will use LLP64 (Windows) or ILP64 (Cray).
The answer for question about SCHAR_MAX == CHAR_MAX is just wrong there. Of course it is "undefined", or actually, it is configurable with "-funsigned-char" flag (for GCC/Clang/ICC).
I did fairly well in the test (I didn't remember that you couldn't shift into the sign bit), but it really helps highlight how stupid the C promotion rules are. In C as soon as I have to do arithmetics with anything but unsigned ints all sorts of alarm bells start going off and I end up casting and recasting everything to make sure it does what I think it does, coupled with heavy testing. Now add floats and doubles into the mix and all bets are off.
Rust not having a default "int" type and forcing you to explicitly cast everything is such an improvement. Truly a poster child for "less is more". Yeah it makes the code more verbose, but at least I don't have to worry about "lmao you have a signed int overflow in there you absolute nincompoop, this is going to break with -O3 when GCC 16 releases 8 years from now, but only on ARM32 when compiled in Thumb mode!"
Still, I would rather not have Rust's 'as' cast. I should like to see all, or at least most uses of 'as' deprecated in a later Rust edition.
Rust's 'as' will silently throw away data to achieve what you asked. Sometimes you wanted that, sometimes you didn't realise, and requiring into() and try_into() instead helps fix that.
For example suppose I have a variable named beans, I forgot what type it is, but it's got the count of beans in it, there should definitely be a non-negative value because we were counting beans, and it shouldn't be more than a few thousand, so we're going to put that in a u16 variable named enough. let enough = beans as u16;
This works just fine, even though beans is i64 (ie a signed 64-bit integer). Until one day beans is -1 due to an arithmetic error elsewhere, and now enough is 65535, which is pretty surprising. It's not undefined but it is surprising.
If we instead write let enough = beans.into(); we get a compiler error, the compiler can't see any way to turn i64 into u16 without risk of loss, so this won't work. We can write beans.try_into().unwrap() and get a panic if actually beans was out of range, or we can actually write the try_into() handler properly if we realise, seeing it can fail, what we're actually dealing with here.
If I can get away with using .into() over as, I will. However, there are so many cases where you can't, and .try_into().unwrap() is just way too unwieldy. In particular, there's no way for me to go "I know I'm only ever going to run on 64-bit machines where sizeof(usize) == sizeof(u64), so usize should implement From<u64>" (and even more annoying, I might be carefully making sure this works on 32-bit and 64-bit machines and get screwed because Rust thinks it could be portable to a 16-bit usize despite the fact I'm allocating hundreds of MB of memory).
And of course there are times I do want to bitcast negative values to large positive unsigned values or vice versa, without error. So while I do understand why maybe you shouldn't use as, at the end of the day, it just ends up being easier to use it than not use it.
I always carry with me a tiu() alias method for exactly that reason (in a TryIntoUnwrap trait implemented for every pair of types which implements TryInto).
A flip side is that some safe conversions also produce compiler errors. On a 64-bit system, usize and u64 are the same width, but they are not convertible: neither is Into the other, so you cannot lean on the compiler in the way you describe.
You might use try_into(), but then you risk panicking on a 32-bit system, instead of getting a compile-time error.
I feel you. I'd really like a way for "as" to be transformed into "try_into().unwrap()", say in a later edition. Or even just "into()" which would throw a nice compile-time error - This is the Rust Way.
The trouble with explicit casting is if the code is refactored to change the underlying integer type, the explicit casts may silently truncate the integer, introducing bugs.
D follows the C integral promotion rules, with a couple crucial modifications:
1. No implicit conversions are done that throw away information - those will require an explicit cast. For example:
int i = 1999;
char c = i; // not allowed
char d = cast(char)i; // explicit cast
2. The compiler keeps track of the range of values an expression could have, and allows narrowing conversions when they can be proven to not lose information. For example:
int i = 1999;
char c = i & 0xFF; // allowed
The idea is to safely avoid needing casts, in order to avoid the bugs that silently creep in with refactoring.
Continuing with the notion that casts should be avoided where practical is the cast expression has its own keyword. This makes casting greppable, so the code review can find them. C casts require a C parser with lookahead to find.
One other difference: D's integer types have fixed sizes. A char is 8 bits, a short is 16, an int is 32, and a long is 64. This is based on my experience that a vast amount of C programming time is spent trying to account for the implementation-defined sizes of the integer types. As a result, D code out of the box tends to be far more portable than C.
D also defines integer math as 2's complement arithmetic. All that 1's complement stuff belongs in the dustbin of history.
Yeah for sure, having more expressive casts and differentiating between "upcasts" and "downcasts" is definitely better. My point that even "lossy" casts are better than C's weird arcane implicit promotion rules by quite a margin IMO.
Rust's current handling of this issue is by no mean perfect, although it's been steadily improving and I definitely don't use `as` as much as I used to.
>D also defines integer math as 2's complement arithmetic.
I think modern C standards does so as well, but I think than signed overflow is still UB, so it's mostly about defining signed-to-unsigned conversions. There are flags on many compilers to tell them to assume wrapping arithmetic but obviously that's not standard...
> The trouble with explicit casting is if the code is refactored to change the underlying integer type, the explicit casts may silently truncate the integer, introducing bugs.
That depends on how the casting is provided. For the C-style casting, or Rust's `as` casting, yes that is a problem. However, another way casting could be provided is through conversion functions that are only infallible if information isn't lost. For example, let's say we have the functions `to_u16` and `to_i16`. For an `i8` the first function could return `Option<u16>`, the second `i16`, while for a `u8` they would return `u16` and `i16`. That way, any change to the types that could cause it to now silently truncate would instead cause a compiler error because of the type mismatch.
Rust almost gets there with its `Into` and `TryInto` traits which do provide that functionality, but trying to use them in an expression causes type inference to fail, which just makes them a pain in the ass to use.
And, notably in this context, Rust's traits are auto-implemented in a chain, so if From<Foo> for Bar, then Into<Bar> for Foo, and thus TryFrom<Foo> for Bar, and thus in turn TryInto<Bar> for Foo.
This means that if first_thing used to have a non-overlapping value space so that converting it to other_thing might fail, so you wrote
... if you later refactor and now first_thing is a subset of other_thing so that the conversion can never fail, the previous code still works fine, the try_into() call just never fails. In fact, the compiler even knows it can't fail, because its error type is now Infallible, a sum type with nothing in it, so the compiler can see this never happens, and optimise accordingly.
char a,b,c;
a = b + c; // error
a = cast(char)(b + c); // ok
produces a truncation error, and requires a cast. But char types have a very small range, and overflow may be unexpected. So D makes the right choice here to promote to int, and require a cast.
Arguably Rust does better for this particular example because, for debug builds and optionally release builds, overflows trigger an error. If you explicitly want wrapping arithmetic in Rust you should use b.wrapping_add(c) in this case, otherwise you'll get a runtime error.
So for instance the following code generates an error (I add to add the function call indirection otherwise rustc would even refuse to compile the straightforward `200u8 + 100u8`):
fn main() {
println!("{}", add(200, 100));
}
fn add(a: u8, b: u8) -> u8 {
a + b
}
// thread 'main' panicked at 'attempt to add with overflow', test.rs:6:5
// note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
IMO that's a more robust and generic solution than hoping that `int` promotion will prevent the overflow. The drawback is that it incurs a performance hit and is therefore only enabled in debug builds by default.
> The drawback is that it incurs a performance hit and is therefore only enabled in debug builds by default.
D has core library functions to check for integer overflow. In my not-so-humble opinion, a targeted solution like this is better than a global switch to turn it on and off. Adding is deeply embedded into computation, and very very few are at any risk of overflowing.
Rust also has checked operations (i.e., checked_add/sub/mul/etc. that return None when there's an over/underflow). You use wrapped operations when that's what's supposed to happen, checked operations when that's what's supposed to happen, debug builds to assert check all operations, and release builds to skip checks anywhere they're not explicit.
I actually dislike this solution, because it relies on tests, while this could easily be a compile-time error.
Not sure I like that D promotes implicitly and then complains about the result of the addition not being assignable to a `char`.
But in theory, every integer could just have the maximum range of values encoded in its type, and for every arithmetic operation the compiler could check if the maximum range of the integer type is exceeded.
The last 1's complement machine I encountered was, never. Not in nearly 50 years of programming. I think those machines left for the gray havens before I was born.
C should also make char unsigned (D does). Optionally signed chars are an abomination.
>The last 1's complement machine I encountered was, never. Not in nearly 50 years of programming. I think those machines left for the gray havens before I was born.
Unisys OS 2200 uses one's complement[1] and Unisys MCP uses signed magnitude[2]. Both are still around.
[2] - https://public.support.unisys.com/aseries/docs/ClearPath-MCP... (page 304) - "ClearPath MCP C uses a signed-magnitude representation for integers
instead of two’s-complement representation. Furthermore, ClearPath MCP C integers use
only 40 of the 48 bits in the word: a separate sign bit and the low order 39 bits for the
absolute value."
IMO everyone gets it wrong. In a CPU instruction set, signed and unsigned are operations, not values, so why not extend that to low level languages? An int32 is neither signed nor unsigned. When you compare them, you choose signed or unsigned comparison.
Rust gets points in my book for being more explicit and avoiding arbitrary promotion rules. Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
> When you compare them, you choose signed or unsigned comparison.
True. I've done a lot of assembler programming. Getting the right jxx instruction after a compare was a prolific source of bugs. I've had a lot fewer bugs with signed types :-)
> Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
I learned C in the late 1970s (although most of my programming back then was in FORTRAN, Assembly, and Pascal). Around the same time I was using a ones-complement system, a CDC 6000 series machine. This was an important machine at the time. The CDC 6600 was the first “super computer” and was three times faster that the preceding fastest computer.
The peculiar vagueness of the early C standards with respect to integers was probably due to the variety of hardware we used in the 1970s. For example:
The CDC 6000/7000s had 60-bit words, ones compliment arithmetic, 6 bit chars (but machine addresses were only of the 60 bit words).
The peripheral processors of the CDC machines (responsible for I/O) had 12-bit words as I recall.
IBMs very popular 360 mainframes supported several native hardware number formats: twos complement, unsigned, zoned decimal (one decimal digit per 8-bit byte), and packed decimal (two digits per byte).
I believe C was first developed on the DEC PDP-11. This was a very popular computer with 16 bit words and 8 bit bytes. It was twos complement, but mixed endianess (big endian for the 16 bit words making up a 32-bit long and little endian for the bytes in the 16-bit words).
Other DEC computers of the 1960s used 12 bit program counters and accumulators; I did some assembly language programming on the PDP-6 (it cost $300,000 back then and 24 were sold).
I seem to recall using a system with 18 bit registers and 9 bit bytes, but I can’t remember the details.
One system that I did assembly language programming on for a year was the TI 960. This machine was notable because it had two separate 64KB address spaces, one for data and one for instruction, known as a Harvard architecture instead of Von Neumann architecture. The advantage of the Harvard architecture was that a program couldn’t inadvertently modify the code it was running, a frequent and difficult to untangle occurrence when programming assembly. The Von Neumann architecture became dominant because it didn’t chop the available memory in two and partly because self modifying code was believed to be sometimes useful (yikes).
D doesn't have any one big feature. It's an accumulation of little things, like a nice desk vs one made out of plywood. :-) People who've used it for a while discover that they're more productive and their code has fewer bugs.
Sort of like putting a nut on a grade 8 bolt rather than a hardware store bolt.
In the examples from the quiz the most tricky promotions IMO are that unsigned shorts are promoted to signed int instead of unsigned int, as signed and unsigned have very different behaviors.
Back before the 1989 C Standard, there was a huge debate about whether to do "value preserving" versus "sign preserving" integer promotions. About half the C compilers did the former, half the latter. Compromise wasn't possible. Value preserving won.
And hence unsigned shorts are promoted to signed ints.
243 comments
[ 0.21 ms ] story [ 208 ms ] threadAre you really, though? I would argue that it's a matter of perspective and/or semantics.
The Linux kernel is built with -fwrapv and with -fno-strict-aliasing, and uses idioms that depend on it directly. We can surmise from that that the kernel must be:
1. Exhibiting undefined behavior (according to a literal interpretation of the standard)
OR:
2. Not written in C.
Either way, it's quite reasonable to wonder just how much practical applicability your statement really has in any given situation -- since you didn't have any caveats. It's not as if the kernel is some esoteric, obscure case; it's arguably the single most important C codebase in the world. Plus there are plenty of other big C codebases that take the same approach besides Linux.
Lots of compiler people seem to take the same hard line on the issue -- "the C abstract machine" and whatnot. It always surprises me, because it seems to presuppose that the only thing that matters is what the ISO standard says. The actual experience of people working on large C codebases doesn't seem to even get acknowledged. Nor does the fact that the committee and the people that work on compilers have significant overlap.
I'm not claiming that "low-level C hackers are right and the compiler people are wrong". I'm merely pointing out that there is a vast cultural chasm that just doesn't seem to be acknowledged.
One of these days, the compiler will do something surprising to one of your expressions involving signed integer overflow, like converting x < x + 1 to true. Or it'll delete a whole loop because it noticed that your code is guaranteed to trigger an out-of-bounds array read, e.g.: https://devblogs.microsoft.com/oldnewthing/?p=633
> If you do that you'll be fine.
I would not trust code written based on the methodology you described. However, if you also add -fsanitize=undefined,address (UBSan and ASan) and pass those tests, then I would trust your code.
Using an integer, in the 1000s of for loops I've written, none get even remotely close to the billions - it is optimizing for a 1 in a million case, and if I know something can run into the billions of iterations I'm going to pay more attention to anyway. I've seen 0 occurrences of bugs relating to this kind of overflow.
Using a size_t, it is effectively an unsigned integer that risks underflowing which can easily cause bugs like infinite loops if decrementing or other bugs if doing any index arithmetic. I've seen many occurrences of these kind of bugs.
Any such code submitted to one of our projects would be rejected or fixed to use the proper type.
I've not introduced a security bug in every for loop I've written. What I've written shouldn't be controversial, just take a look at Googles style guide:
"We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t or uint64_t.
For integers we know can be "big", use int64_t.
You should not use the unsigned integer types such as uint32_t, unless there is a valid reason such as representing a bit pattern rather than a number, or you need defined overflow modulo 2^N. In particular, do not use unsigned types to say a number will never be negative. Instead, use assertions for this.
If your code is a container that returns a size, be sure to use a type that will accommodate any possible usage of your container. When in doubt, use a larger type rather than a smaller type.
Use care when converting integer types. Integer conversions and promotions can cause undefined behavior, leading to security bugs and other problems."
You should prefer signed types for computing (if not storing) sizes and for pointer arithmetic, since they are more forgiving with underflow.
OTOH, the largest "wat" of C-like languages is the following:
whyPython here is once again correct:
You may as well use {white, blue, black, red, green} to represent congruences mod 5, mathematically speaking it's not wrong, as long as they respect the axioms of a field.
What's "wat" (in the same sense as the js "wat" talk) is that the answer changes if the argument becomes negative. By all reasonable definitions of modular arithmetic, -8 and 7 are in the same class mod 5. Why is then:
in C-like languages? I'm pretty sure it's perfectly consistent with all the relevant language standards, but it's a "wat" nonetheless.It's the same in Python though? you just prefer the way it behaves (though to be fair it's generally more useful and less troublesome): Python's remainder follows the sign of the divisor (because it uses floored division), while C's follows the sign of the dividend (because it uses truncated division).
Strictly speaking, neither is an euclidian modulo.
Haskell does it right. Instead of having operators like %, it gives the programmer choice to choose either the modulus or the remainder. Using GP's example:
And we have both (x `quot` y)*y + (x `rem` y) == x and (x `div` y)*y + (x `mod` y) == xI think `undefined behaviour` still has its place in C - dereferencing a freed pointer comes to mind as an obvious example - but I think a good proportion of the really unintuitive UB conditions could be made saner without sacrificing portability or optimisation opportunities.
If the hardware traps on bogus pointers, then reading a bogus pointer may trap. But if you read a recently-freed pointer, it may still be valid according to the hardware (e.g. will have valid PTEs into the processes address space) so won't trap. Therefore you won't be able to guarantee any particular behaviour on an invalid read, so I don't think you'd be able to get away with "unspecified" or "implementation defined" behaviour on most hardware.
In practice, compilers aren't actually adversarial. A lot of the discussion around UB is catastrophizing and talks about how the compiler will order you pizza or delete your disk. Some problems are real and I know some people whose graduate school work was very specifically on the problems that this causes for security-related checks but compilers really really do not transform your program into "complete garbage." They transform your program into a program with a bug, which is a true statement about that program.
I'm reminded of the apocryphal story about being asked how a computer knows how to do the right thing if given the wrong inputs. This feels similar.
> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
-- Charles Babbage
https://en.wikiquote.org/wiki/Charles_Babbage#Passages_from_...
And further, defining a lot of UB won't actually improve things. Imagine we define signed integer overflowing behavior. Hooray. Now your program just has a different bug. If you've accidentally got signed integer overflow in your application then "did some weird things because the compiler assumed it would never overflow" is going to cause exactly the same amount of havoc as "integer overflowed and now your algorithm is almost certainly wrong."
This is exactly what JF Bastien argues in his hourlong talk: https://youtu.be/JhUxIVf1qok?t=2284
So yeah, defining signed integer overflow isn't going to fix bugs in the vast majority of existing programs.
That being said, enforcing signed overflow wraparound at least makes debugging easier because it's reproducible. This is how it is in Java land - int overflow wraps around, and integer type widths are fixed, so if you trigger an overflow while testing then it is reliably reproducible across all conforming Java compiler and VM versions and platforms.
Personally, I love C but I don’t use it in anything serious. Probably for the best given I apparently cannot remember how C ints work.
(Edit, since some respondents seem to miss this, explanations about efficiency or ISAs might justify promoting to u32 (though even that's debatable), but not i32. A design that auto-promotes an unsigned type, where every operation is nicely defined and total, into a signed type, where you run into all kinds of undefined behavior on overflow, is simply crazy.)
Here's a fun standardization problem I came across recently (nothing to do with C): http://mywiki.wooledge.org/BashFAQ/105
If it promoted to unsigned int, then it would also clear bits 32-63.
Now, I don't find this reasoning persuasive--it's not that hard to emulate an 8-bit or 16-bit operation--and judging from the history of post-C languages, most other language designers are equally unmoved by this reasoning, but I can see someone in their right mind designing a language that acts like this. Especially if the first architecture they're developing on is precisely such on architecture (PDP11 doesn't have byte-sized add/sub/mul/div).
IMO the real issue is not so much the fact that all shifts of any type < int is treated as if it were an int, it's that the language doesn't force you to acknowledge that in the code. If you got a compilation error when trying to shift a short and had to explicitly promote to int in order to make it through, at the very least it can't lead to an oversight from a careless programmer.
C is trying to be clever but only goes half way, resulting in the worst of both worlds IMO.
UBs are not required, but you need them if you want C to behave as a macro-assembler as well as allowing for aggressive optimizations. For instance `a << b` if b is greater than a's width is genuinely UB if you write portable code, different CPUs will do different things in this situation. Defining the behaviour means that the compiler would have to insert additional opcodes to make the behaviour identical on all platforms.
You may argue that it's still better than having UB but that's just not C's design philosophy, for better or worse.
Your seem to be mixing up implementation defined behavior and undefined behavior. It would have been perfectly reasonable to make this choice if signed integer ovwrflow were implementation-defined, but it is unfortunately not - it is undefined behavior instead. This means that a program containing this instruction is not valid C and may "legally" have any effect whatsoever.
Yeah, hence why I asked this question years ago: https://stackoverflow.com/questions/39964651/is-masking-befo...
Yes, it'd be better if you had to explicitly cast to int or unsigned to perform arithmetic, but that ship has sailed.
The last x86 processor I know of to have these sizes is 80286.
The question would be fine (given that note) if "impementation-defined" was not an option, like for example the question about "SCHAR_MAX == CHAR_MAX".
The last machine with a word size of 16 bits? The 286 was as well. There were others like the WDC 65816 (the Apple IIgs and SNES CPU).
It just so happens that there simply far fewer 16 bit CPUs then there were 8 or 32 (or “32/16” like the 68k). Also 8 bit CPUs are simply a poor fit for C by their nature and the assumptions C makes. But the numerous ones still relevant today will use 16 bit ints.
The use of Real or V86 mode on the x86 went on for many years after the demise of the 8086. I think it is a somewhat of a joke at this point that they’re teaching Turbo C in some developing countries.
From C99 §6.3.1.1:
> — The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, […]
> — The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any.
According to the standard[1], if short and int have the same size (even if not the same rank) both numbers are converted to unsigned int (that is, the unsigned integer type corresponding to int) because int can't represent all values of unsigned short.
The usual arithmetic conversions never "promote" an unsigned to signed if doing so would change the value.
[1] http://port70.net/~nsz/c/c99/n1256.html#6.3.1.8
> abs(-2147483648) = -2147483648
I think one's complement is more sensible since it doesn't have this problem, but it loses out because it requires a more complex ISA and implementation.
> The name "ones' complement" (note this is possessive of the plural "ones", not of a singular "one") refers to the fact that such an inverted value, if added to the original, would always produce an 'all ones' number
In fact, for every numerical base there is an equivalent. For example, in base 3 you can represent a number in twos' complement notation (each trit X is replaced with 2-X, so -3 is represented as 212 on three trits, with +3 being 010) is or in three's complement by adding 1 (212+1 = 220). For base 10, you can do nines' complement (-3 on 3 digits is 996) or ten's complement (996+1 = 997).
Simple MISRA C with all warnings on is already close to language with strict type checking. C compilers give you a choice, use it. If you are programming for money, good static analyzer makes it possible to write safety critical code.
"Although the first edition of K&R described most of the rules that brought C's type structure to its present form, many programs written in the older, more relaxed style persisted, and so did compilers that tolerated it. To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions."
Dennis M. Ritchie -- https://www.bell-labs.com/usr/dmr/www/chist.html
Unfortunately too many think they know better than the language authors themselves.
Would it be possible to have a standard, where undefined behavior is just a compile error? What would we lose - apart from legacy compatibility?
No [if you're aiming for something in the same vein as C]. Undefined behavior is ultimately an inherently dynamic property--certain values could make a statement execute undefined behavior, and consequently, virtually every statement could potentially cause undefined behavior. Note that this remains true even in languages like Rust: Rust has loads of undefined behavior, but you do have to wrap code in unsafe blocks to potentially cause undefined behavior.
> What would we lose - apart from legacy compatibility?
In particular, it is clear at this point that if you want to permit converting integers to pointers, you will either have to live with undefined behavior (via pointer provenance) or forgo basically all optimization whatsoever.
UB started as means to not kick out computer architectures that would otherwise not be able to be targeted by fully compliant ISO C compilers.
Given that C prefers to be a kind of portable macro assembler than care about security, it was only a matter of time until those escape hatches started to be taken advantage for optimizations.
Same applies to other languages, however since their communities tend to prefer security before ultimate performance, some optimization paths are not considered as that would hurt their safety goals.
In what concerns C, C++ and Objective-C, dropping UB optimizations would mean going back to the 1990's in terms of code quality.
Unless your program can prove, that the function is only ever called with arguments that don't hit UB.
(This is the case in e.g. C#.)
Since most languages use the same IEEE-754, you can hate them all.
Most professionals don't worry about these effects, instead choosing to cast and use parenthesis directly. Those instruct the compiler instead of relying upon warnings from the compiler.
This is relevant to the question asking if -1L > 1U.
Completely agree with you, and not just in C. Using esoteric features of any language is equivalent to putting landmines in front of your teammates - bad idea.
Also, outside of production code, using esoteric features is a good way to get familiar with every corner of that language - which is useful so that you can diffuse those landmines when they are accidentally created. i.e know them to avoid them.
x and y are unsigned short. The expression `x*y` has defined behavior for...
a) all values of x and y.
b) some values of x and y.
c) no values of x and y.
I missed that the result would be promoted to signed int, which only most significant bits are set
But if int were larger (eg 64 bit) and short remained 16 bits, there’s no overflow and the answer is (A). I think.
If short = 16 bits and int = 32 bits (or heck even 17 bits), then x and y will be promoted to signed int. Signed multiplication overflow is undefined behavior, so x*y will be undefined for some values when x*y is too large. In particular, 0xFFFF * 0xFFFF = 0xFFFE_0001, which is larger than INT_MAX = 0x7FFF_FFFF.
If short = 16 bits and int = 64 bits (or even 33 bits), then x and y will be promoted to signed int. The range of x*y will always fit int, so no overflow occurs, and the expression is defined for all input values.
Isn't C fun?
For char, you have 3: signed char, unsigned char and char. It's not specified if char without keyword is signed or unsigned.
You have integer types such as size_t, ssize_t and ptrdiff_t. They may, under the hood, match one of the other standard int types, however this differs per platform, so you can't e.g. just easily print size_t using the standard printf formatters, you really have to treat is as its own type. Also wchar_t and such of course.
Then you have all the integers in stdint.h and inttypes.h. Same here applies as for size_t. At least you know how many bits you get from several of them, unlike from something like "long".
Then your compiler may also provide additional types such as __int128 and __uint128_t.
This has been fixed in C99. For size_t, it's "%zu", for ptrdiff_t it's "%td", for ssize_t it's "&zd" and for wchar_t, it's "&lc".
Still, the most important suggestion I would make here is: Always compile with warnings enabled, in particular:
* -Wstrict-overflow=1 and maybe even -Wstrict-overflow=3
* -Wsign-compare
* -Wsign-conversion
* -Wfloat-conversion
* -Wconversion
* -Wshift-negative-value
(or just `-Wall -Wextra`)
And maybe also:
* -fsanitize=signed-integer-overflow
* -fsanitize=float-cast-overflow
(These are GCC flags, should work for clang as well.)
> Sorry about that -- I didn't give you enough information to answer this one. The signedness of the char type is implementation-defined
...why not have an "implementation-defined" answer button then, because that's what people should know (instead of knowing all ABIs) and what the question is about anyway?
> If these operators were right-associative, the expression [x - 1 + 1] would be defined for all values of x.
That's just wrong, no? If + and - were right-associative, it would be parsed as x - (1 + 1), which is decidedly not "defined for all values of x".
I.e. C standard enforces undefined very sparingly iirc, and most of the corner cases are implementation defined.
I may also be misremembering things: it's been 20 years since I've carefully read C99 (draft, really) for fun :))
https://godbolt.org/z/PTEoW91v5
Rust not having a default "int" type and forcing you to explicitly cast everything is such an improvement. Truly a poster child for "less is more". Yeah it makes the code more verbose, but at least I don't have to worry about "lmao you have a signed int overflow in there you absolute nincompoop, this is going to break with -O3 when GCC 16 releases 8 years from now, but only on ARM32 when compiled in Thumb mode!"
Rust's 'as' will silently throw away data to achieve what you asked. Sometimes you wanted that, sometimes you didn't realise, and requiring into() and try_into() instead helps fix that.
For example suppose I have a variable named beans, I forgot what type it is, but it's got the count of beans in it, there should definitely be a non-negative value because we were counting beans, and it shouldn't be more than a few thousand, so we're going to put that in a u16 variable named enough. let enough = beans as u16;
This works just fine, even though beans is i64 (ie a signed 64-bit integer). Until one day beans is -1 due to an arithmetic error elsewhere, and now enough is 65535, which is pretty surprising. It's not undefined but it is surprising.
If we instead write let enough = beans.into(); we get a compiler error, the compiler can't see any way to turn i64 into u16 without risk of loss, so this won't work. We can write beans.try_into().unwrap() and get a panic if actually beans was out of range, or we can actually write the try_into() handler properly if we realise, seeing it can fail, what we're actually dealing with here.
And of course there are times I do want to bitcast negative values to large positive unsigned values or vice versa, without error. So while I do understand why maybe you shouldn't use as, at the end of the day, it just ends up being easier to use it than not use it.
I always carry with me a tiu() alias method for exactly that reason (in a TryIntoUnwrap trait implemented for every pair of types which implements TryInto).
You might use try_into(), but then you risk panicking on a 32-bit system, instead of getting a compile-time error.
D follows the C integral promotion rules, with a couple crucial modifications:
1. No implicit conversions are done that throw away information - those will require an explicit cast. For example:
2. The compiler keeps track of the range of values an expression could have, and allows narrowing conversions when they can be proven to not lose information. For example: The idea is to safely avoid needing casts, in order to avoid the bugs that silently creep in with refactoring.Continuing with the notion that casts should be avoided where practical is the cast expression has its own keyword. This makes casting greppable, so the code review can find them. C casts require a C parser with lookahead to find.
One other difference: D's integer types have fixed sizes. A char is 8 bits, a short is 16, an int is 32, and a long is 64. This is based on my experience that a vast amount of C programming time is spent trying to account for the implementation-defined sizes of the integer types. As a result, D code out of the box tends to be far more portable than C.
D also defines integer math as 2's complement arithmetic. All that 1's complement stuff belongs in the dustbin of history.
Rust's current handling of this issue is by no mean perfect, although it's been steadily improving and I definitely don't use `as` as much as I used to.
>D also defines integer math as 2's complement arithmetic.
I think modern C standards does so as well, but I think than signed overflow is still UB, so it's mostly about defining signed-to-unsigned conversions. There are flags on many compilers to tell them to assume wrapping arithmetic but obviously that's not standard...
That depends on how the casting is provided. For the C-style casting, or Rust's `as` casting, yes that is a problem. However, another way casting could be provided is through conversion functions that are only infallible if information isn't lost. For example, let's say we have the functions `to_u16` and `to_i16`. For an `i8` the first function could return `Option<u16>`, the second `i16`, while for a `u8` they would return `u16` and `i16`. That way, any change to the types that could cause it to now silently truncate would instead cause a compiler error because of the type mismatch.
Rust almost gets there with its `Into` and `TryInto` traits which do provide that functionality, but trying to use them in an expression causes type inference to fail, which just makes them a pain in the ass to use.
This means that if first_thing used to have a non-overlapping value space so that converting it to other_thing might fail, so you wrote
... if you later refactor and now first_thing is a subset of other_thing so that the conversion can never fail, the previous code still works fine, the try_into() call just never fails. In fact, the compiler even knows it can't fail, because its error type is now Infallible, a sum type with nothing in it, so the compiler can see this never happens, and optimise accordingly.So for instance the following code generates an error (I add to add the function call indirection otherwise rustc would even refuse to compile the straightforward `200u8 + 100u8`):
IMO that's a more robust and generic solution than hoping that `int` promotion will prevent the overflow. The drawback is that it incurs a performance hit and is therefore only enabled in debug builds by default.D has core library functions to check for integer overflow. In my not-so-humble opinion, a targeted solution like this is better than a global switch to turn it on and off. Adding is deeply embedded into computation, and very very few are at any risk of overflowing.
Not sure I like that D promotes implicitly and then complains about the result of the addition not being assignable to a `char`.
But in theory, every integer could just have the maximum range of values encoded in its type, and for every arithmetic operation the compiler could check if the maximum range of the integer type is exceeded.
You could set the range explicitly:
Or make the function generic over any range of inputs where the possible range of outputs provably fits the output type, which would be the default.There are crates that add ranged integers as types, but I haven’t found them satisfactory to work with.
C should also make char unsigned (D does). Optionally signed chars are an abomination.
Unisys OS 2200 uses one's complement[1] and Unisys MCP uses signed magnitude[2]. Both are still around.
[1] - https://public.support.unisys.com/2200/docs/CP19.0/78310422-... (page 108) - "UCS C represents an integer in 36-bit ones complement form (or 72-bit ones complement form, if the long long type attribute is specified)."
[2] - https://public.support.unisys.com/aseries/docs/ClearPath-MCP... (page 304) - "ClearPath MCP C uses a signed-magnitude representation for integers instead of two’s-complement representation. Furthermore, ClearPath MCP C integers use only 40 of the 48 bits in the word: a separate sign bit and the low order 39 bits for the absolute value."
Rust gets points in my book for being more explicit and avoiding arbitrary promotion rules. Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
True. I've done a lot of assembler programming. Getting the right jxx instruction after a compare was a prolific source of bugs. I've had a lot fewer bugs with signed types :-)
> Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
D does, too:
https://dlang.org/phobos/core_checkedint.html
along with integer types that automatically check for overflow:
https://dlang.org/phobos/std_checkedint.html
The peculiar vagueness of the early C standards with respect to integers was probably due to the variety of hardware we used in the 1970s. For example:
The CDC 6000/7000s had 60-bit words, ones compliment arithmetic, 6 bit chars (but machine addresses were only of the 60 bit words).
The peripheral processors of the CDC machines (responsible for I/O) had 12-bit words as I recall.
IBMs very popular 360 mainframes supported several native hardware number formats: twos complement, unsigned, zoned decimal (one decimal digit per 8-bit byte), and packed decimal (two digits per byte).
I believe C was first developed on the DEC PDP-11. This was a very popular computer with 16 bit words and 8 bit bytes. It was twos complement, but mixed endianess (big endian for the 16 bit words making up a 32-bit long and little endian for the bytes in the 16-bit words).
Other DEC computers of the 1960s used 12 bit program counters and accumulators; I did some assembly language programming on the PDP-6 (it cost $300,000 back then and 24 were sold).
I seem to recall using a system with 18 bit registers and 9 bit bytes, but I can’t remember the details.
One system that I did assembly language programming on for a year was the TI 960. This machine was notable because it had two separate 64KB address spaces, one for data and one for instruction, known as a Harvard architecture instead of Von Neumann architecture. The advantage of the Harvard architecture was that a program couldn’t inadvertently modify the code it was running, a frequent and difficult to untangle occurrence when programming assembly. The Von Neumann architecture became dominant because it didn’t chop the available memory in two and partly because self modifying code was believed to be sometimes useful (yikes).
Sort of like putting a nut on a grade 8 bolt rather than a hardware store bolt.
People don't always agree on what's better:
https://digitalmars.com/d/archives/digitalmars/D/Movement_ag...
and sometimes the rationale for things isn't obvious at all, and even counter-intuitive.
And hence unsigned shorts are promoted to signed ints.