180 comments

[ 4.1 ms ] story [ 231 ms ] thread
(comment deleted)
Aah, this felt like a much-needed piece. The original text was so long, it kind of made my eyes glaze over and I never found the time to do this level of detailed critique. Well done!
Not to be confused with Ken Thompson.
Almost fell for that....
> Zeroing memory often means that buggy code will have consistent behavior; by definition it will not have correct behavior. And consistently incorrect behavior can be more difficult to track down.

I agree that using calloc should not be an excuse for writing sloppy initialization code. But in my experience, inconsistently incorrect behaviour (e.g. heisenbugs that tend to appear and disappear depending on the content of some random chunk of uninitialized memory that happens to be in an unfortunate state after malloc) are one of the hardest type of bugs to fix. It's the type of the bug that occasionally happens in production and typically can't be reproduced in a controlled manner in a development environment. I certainly prefer consistently incorrect behavior.

It's also more secure. We've just this week now had an SSH bug where random chunks of memory get dumped somewhere they shouldn't (not that I'm clear on if calloc would help in this case, but the issue remains).

Zeroing memory is just good habit - it's means you don't wind up accidentally leaking things when you make a mistake. And you should assume you will.

calloc is not relevant to that issue - it's an out-of-bounds read.

(Good ASLR does help, by turning such issues into crash bugs.)

https://www.qualys.com/2016/01/14/cve-2016-0777-cve-2016-077...

>- "out_start == out_last" (lines 205-206): no data was ever written to out_buf (and both out_start and out_last are still equal to 0) because no data was ever sent to the server after roaming_reply() was called, but the client sends (leaks) the entire uninitialized out_buf to the server (line 214), as if out_buf_size bytes of data were available.

Heartbleed wasn't an out-of-bounds read.
If OpenSSL had used calloc() everywhere instead of malloc(), Heartbleed would have still happened.
The tools for rooting out these problems have gotten a lot better, thankfully. I'm very fond of the address sanitizer. Using calloc() everywhere can prevent the address sanitizer from finding problems.
You could try to split the difference by using a wrapper around malloc that zeroes in production and doesn't under testing.

Of course, running significantly different code in production and test has its own issues.

I mean, reading zeroed out memory is not really incorrect behaviour anyway. If you know that it was zeroed out, then you're really just reading memory that was initialised with a specific default value.

That being said, zeroing out everything you allocate will make it difficult for the kernel's paging system to defer allocating you a physical page until you actually need it, so you will need to be a lot more careful with your allocations if your application uses relatively large amounts of memory and you need to work with memory constraints.

Unless the zeroing is defined by the language, assuming it t is incorrect and shouldn't be done.
Calloc is defined to return memory that's all-bits-zero, which in turn defines the value for many datatypes. The overwhelming majority of C implementations additionally define the values of all-bits-zero floats and pointers. IMO it's reasonable to write programs targeting only such implementations, provided such a dependency is clearly documented (just as e.g. it's reasonable to write C programs targeting only implementations on which floating-point arithmetic is IEEE 754, despite this not being required by the C standard).
> The overwhelming majority of C implementations additionally define the values of all-bits-zero floats and pointers.

So... Clang and GCC?

Regardless, you're ignoring the obvious difference: that isn't POSIX- or ISO-defined at all. If you choose to use it, you're using a weird dialect. Might as well just switch to Cyclone.

> that isn't POSIX- or ISO-defined at all. If you choose to use it, you're using a weird dialect.

People used C (even on multiple compilers) long before POSIX or ISO. You're putting the standardization cart before the compatibility horse.

Yeah, let's revert to 1988! Great solution! Muchly of advance!
Great point.

This is the part I disagree with most strongly:

> And consistently incorrect behavior can be more difficult to track down.

All of that useless zeroing is very painful from a performance perspective. It just isn't a reasonable option in performance-relevant code like kernels and core infrastructure. The Moore's Law argument doesn't apply because memory bus latencies and case pressure are lasting problems.
On high-performance code for an embedded PPC system I used to work on, we made all our control block a multiple of the L1 cache width. Our allocation routines then all had inline assembler to run the dcbz instruction (data cache block zero) on all the cache blocks for the control block as it was allocated. This meant the control block was always zeroed, and the memory bus wasn't touched in order to do so. Yes, things were evicted from the cache, but since we're about to start writing things into the control block, the lack of fetch was a net gain.
A better solution in this case is to fill the memory with junk -- 0xdeadbeef is a popular solution.
I'm divided.

I think it is good to share our ideas and our theories with each other because the discourse can stretch our minds and help us better imagine the perspective of another. Dialogue can foster creativity and invention, and an honest argument can embiggen friendships like nothing else.

On the other hand, opinions are like assholes.

This, ladies and gentlemen, is why C gives me the willies. And why this happens:

http://www.cvedetails.com/vulnerability-list/year-2016/month...

Not that it would eliminate security exploits caused by logical bugs, but most those exploits were already taken care of with Burroughs Algol variant (1961), Modula-2 (1978), Cedar (1981), Ada (1983), .... but oh well.
Those are due to programmer error. The fact that C doesn't lie to you means that it takes a bit more experience to program properly; as with all things, with great power comes great responsibility. You don't blame the gun for killing people, you blame the person for shooting it.
Not even the very best C programmers will write memory safe code. When you write C you will write exploitable code, no matter who you are. Don't blame the programmer for stepping onto a mine in the minefield, blame the minefield.
> Those are due to programmer error.

How does that help? Almost all security vulnerabilities are due to programmer error.

> You don't blame the gun for killing people, you blame the person for shooting it.

It's more like choosing between a gun that makes it really easy to shoot yourself in the foot by accident and one that is safer.

I disagree with many of the arguments. Many of them are ignoring the reality of today.

One example:

> If you want bytes, use unsigned char. If you want octets, use uint8_t

Bytes and octets are the same today. I never came across a system where this wasn't true. I was never told about a system where this wasn't true. I wasn't even told that there once were systems where this wasn't true until fairly recently.

What was the original distinction between bytes and octets? Byte = word vs octet = 8-bits, i.e. a byte is not necessarily 8-bits long?
A byte is the smallest addressable unit. An octet is a unit of 8 bits, which is a common size for a byte.
A byte is the smallest individually-addressable unit.
Oh, well, if you haven't seen something then it must not exist.

"I wasn't even told that there once were systems where this wasn't true until fairly recently."

Wait, so someone filled in a piece of your ignorance and your response is still "Bytes and octets are the same"?

To be fair, what they actually said was, "Bytes and octets are the same today". It's a little disingenuous, I think, to quote everything but the last word, especially since the qualifier "today" was kind of their entire point.
In the words of SLJ, allow me to retort. Bytes and octets are the same today on the hardware the OP is used to working with; I'm guessing whatever the latest generation of commodity x86 derivatives is.

The real crime here isn't lack of knowledge; it's thinking that one's own narrow experience is all-encompassing and that what's true in one part of the programming world is true everywhere.

I'm not disputing your last statement. I'm saying that, if you are going to quote someone, quote the whole sentence, at least, and don't leave out words that are substantial to the point they are making.
>Bytes and octets are the same today. I never came across a system where this wasn't true.

The fact that all modern systems use eight bit bytes is no reason to assume that a byte is eight bits. The fact is that a byte is not eight bits large.

People may wish to run code on "historical" systems, and someone may wish to create a 9-bit byte system for fun.

To write C code with an assumption that things are one way, when the standard does not define this assumption, screams of unportable code. You might as well use GCC extensions - after all, for a long time, what would it matter? Everyone used GCC, right?

> People may wish to run code on "historical" systems, and someone may wish to create a 9-bit byte system for fun.

I bet there will be then bigger problems with porting the code than assuming byte is 8 bit long if someone is creating a 9-bit system "for fun".

As a matter of fact, GCC extensions are de facto standards at this point. Clang had to copy most of them.

Same with bytes not being 8 bits. Nobody will ever make a system where that's not the case anymore. You'd break de facto compatibility with almost all C code for no reason. Market realities trump making systems "for fun".

>You'd break de facto compatibility with almost all C code for no reason.

The authors of that code should have written portable code if they didn't want it to break.

Texhnically, they did. The go to compiler for years was GCC, with all the GNU extensions turned on. The problem is that those extensions were classically on by default, so it was very easy to just use them without thinking too much about it.
On GNU/Linux, yes.

Outside world, there are tons of C compilers to chose from.

If a few people write unportable code in your language, they're bad programmers. But if everyone writes unportable code in your language, it's a bad language.
If you're designing a new ISA, it's extremely difficult to succeed in the market no matter what you do. But one excellent way to guarantee you won't succeed is to have that attitude toward your developers.
By the same argument we shouldn't assume that bits have only two values - what, you mean I can't write C for my ternary logic computer?

You need to make a decision about which simplifying assumptions you make - C has decided not to assume there are 8 bits to the byte. I think it's arguable that in 2016 this is not the best decision, and they could get rid of some cognitive overhead by making the simplifying assumption.

Unfortunately, there are real world systems with C compilers that do not have 8 bit bytes. It's easy to forget the embedded world.

C is one of the very, very few languages that you can reasonably expect to run on your weird embedded microprocessor, microcontroller, DSP, or whatever.

Bit, however, means "binary digit". That's the literal definition of the word. Historical usage of the word "byte" has varied.

> C is one of the very, very few languages that you can reasonably expect to run on your weird embedded microprocessor, microcontroller, DSP, or whatever.

I'm not an expert on DSP processing in any way, but I'd argue that if the environment is so peculiar that it doesn't have 8-bit bytes, you won't be able to use so many of the (formally optional) parts of C that you might as well call your code "weirdC" and do without char. It's not like you're going to use normal C libraries, and you are most probably relying on a specific compiler implementation and a lot of platform specific weirdnesses.

I think you're underestimating the amount of legacy code that people have. We're not talking about "I have a new processor, let's write code for it." That's unrealistic. It's more like, "I have a new processor, let's get our existing code base working on it, because if we don't, we're going to lose millions of dollars."
That doesn't touch my argument. If you're code is meant to run on DSPs it will already enforce all the relevant rules; if not, you'll have to port it anyway. It's not like your compiler backend that outputs code for DSPs will just compile standard C.
All the code using non-optional c constructs will compile. If you're doing something that uses floating point math and expecting to run it on a 16-bit DSP, you're probably doing something wrong. The biggest hurdle isn't compiler support, rather converting all the floating point operations to fixed-point and ensuring that you have enough resolution such that rounding errors don't make your IIR filter unstable for example. That's really outside the scope of the discussion though, as that is just an aspect of practical DSP using fixed point math.
You can use char, and for the most part you won't notice any difference, until you start looking at the memory display in your IDE and have a short-lived WTF moment. A c-string looks strange in the memory view because there's a NULL octet between each character but that's an implementation detail that you don't really notice 99% of the time, because everything on the processor is designed to use 16-bit words. If you write a buffer of chars to the UART tx register, it will only put the half of the word with the character data onto the wire, just like with any other microcontroller, just like with PySerial, and just like with JS and websockets. That's why the C standard doesn't say a char is 8-bits, it just says that sizeof(char) is 1. I would argue that any code doing anything under the assumption that a char variable occupies 8 bits in memory and that the next char in an array is physically adjacent to it with no wasted space in between is just poorly-written and inherently not portable.

As for why this is the case, Generally DSPs are designed to do a few tasks very well (multiply-accumulate being the most obvious example, simultaneous reads from X and y memory, etc.) and are generally optimized for a specific word size. If you really need to access an octet on E.g. A 16-bit TI DSP you can, you just need to shift and mask. That's obviously not very efficient, because that's not really what it's optimized for. If that's what you really need, you picked the wrong part. You're gonna be filter-/FFT-/DCT-/whatever-ing way more 12 or 16-bit ADC samples per second, per dollar, and per milliWatt on that 16-bit DSP than you would be on your octet-addressable MSP430.

Regarding "Normal" c libraries, If they're valid C they should work just fine for the most part, With some obvious exceptions: you're not going to have a fun time with your "floats" on a processor without FP HW. That's why there's a standard, if you're not relying on undefined or implementation-defined behavior, you'll be fine(ish)

> I would argue that any code doing anything under the assumption that a char variable occupies 8 bits in memory and that the next char in an array is physically adjacent to it with no wasted space in between is just poorly-written and inherently not portable

I don't think that's fair at all. It's well written and perfectly portable, under that assumption. If that assumption happens to hold for all the systems that could is meant to run on, why shouldn't you make it? Why assuming that unsigned integers overflow to 0 is fair, and assuming 8 bit bytes is not? They are both pragmatic choices. I think arguing that the C standard didn't make the best choice is at the least a reasonable position. Saying that people who disagree with you on this are writing bad code is not the right way to approach the subject, IMO.

> If they're valid C they should work just fine for the most part

I've yet to find a library that is valid C, in the strictest sense of the term - i.e., no floats or doubles, no 8-bit char, no uint8_t, etc. I'm sure there are, among libraries meant to run on DSPs. Does it make sense for them and libraries meant to run on bigger processors to adhere to the same exact standard? I'm not sure.

> If that assumption happens to hold for all the systems that could is meant to run on, why shouldn't you make it?

Because what's going to happen two years down the road when you need to get that code working on a different platform.

(In the old days, they called that thinking "all the world's a VAX", which became "all the world's linux" in the not-quite-as-old days)

> Because what's going to happen two years down the road when you need to get that code working on a different platform.

And what happens when the different platform handles overflow differently? Or has non-IEEE floats? Or breaks whatever assumption the C standard already makes?

Here's the thing: C is already built on assumptions, which might or might not hold for a specific platform.

Choosing which assumptions we hold true is a pragmatic choice that must be done. C does it, and does it consciously - but it doesn't mean that it nails every choice.

If you're in a situation where you need specific implementation details of floating point arithmetic, then may the devil have mercy on your soul, because god certainly won't.
ehmmmm standard C has floats and doubles. so they are valid C

>Why assuming that unsigned integers overflow to 0 is fair

because the standard says so

In your other posts I noticed some more misunderstanding of the C standard, so I suggest not to talk about things you are not aware about.

> ehmmmm standard C has floats and doubles. so they are valid C

Sure, but the standard says hardly anything about their behaviour, making it practically impossible to write code that actually does anything with floats and doubles that will behave correctly on any standards-compliant C implementation.

I suggest you refrain from personal attacks, especially based on your own misreadings of the comment you're replying to.

>but the standard says hardly anything about their behaviour, making it practically impossible to write code that actually does anything with floats and doubles that will behave correctly on any standards-compliant C implementation.

No, not really. This reminds me the common "you can't write anything serious in C without undefined behaviour" bs.

However, if you DO have a real need of IEEE-compatible floats for whatever reason, then there is nothing wrong with checking __STDC_IEC_559__ and reporting an error if it is not defined. In fact, if you are using a system that doesn't have an IEEE float implementation, using another language won't save you.

>I suggest you refrain from personal attacks, especially based on your own misreadings of the comment you're replying to.

Feel free to show me what I misread. I would also suggest that you try to read the nonsensical bullshit he has been spreading in this thread.

> However, if you DO have a real need of IEEE-compatible floats for whatever reason, then there is nothing wrong with checking __STDC_IEC_559__ and reporting an error if it is not defined.

Sure - but in that case why not do the same thing for insisting that char be 8 bits?

Except than the fact that it does not limit you except in some very specific cases, unlike the IEEE incompatible floats thing.

If it does limit you somehow and you really need char be 8 bits, then sure.

Maybe "poorly-written" was a bit strong, but if you're making that assumption, you are by definition relying on un- or implementation-defined behavior.

The fact that a char is stored on silicon in the lowest 8 bits of the smallest addressable memory space really makes no difference. char arrays act just like they do on any other platform.

There aren't a lot of libraries floating around that you'd be using on a DSP, aside from vendor-supplied ones. Generally you're using a DSP because you have specific signal processing needs and power and cost requirements, which are pretty orthogonal to the requirements of most desktop development. As such, there probably isn't a huge amount of overlap between the libraries you'd be using.

Another reason for that being that quite a bit of the time you're not running an OS. There is no transparent abstraction layer that magically maps an ADC IC connected via SPI to an arbitrary group of remappable peripheral pins to /dev/adc0. I think that's kind of the big disconnect between desktop and embedded, so much of the code is specific to the processor/DSP and the arbitrary external hardware connected to it. You can't take code written for a DsPIC that uses the internal ADC and run it on an MSP430 with an internal ADC for the same reason. There isn't necessarily a POSIX layer, so it's a very different beast than writing code that works on UNIX, Linux, and OSX. Look at the Linux source and see how many architecture-specific files there are in each architecture's folder. That's what you need to get a consistant interface across different processors. Without all that non-portable abstraction code running below yours, posix- and standard-compliant code still wouldn't be portable.

As far as using the same standard, I think it's a good thing that there is agreement on what the effect of a given construct is. I think the issue comes when you expect that code written for one platform to just work(TM) on another.

As a male human of questionable maturity, I'd just like to say that I am really looking forward to working with ternary digits!
I would be very surprised if some non-trivial C code written for x86 compiles and runs unmodified on exotic platforms. Besides, I don't want to be hold back by people wishing to run code on historical or experimental systems.
Yeah at least since the 80s bytes has been taught as "8 bits" for as long as I've ever heard it.

That said, uint8_t is more explicit (and less to write).

char is apparently still not always 8-bit on current DSP chips, where everything is 32-bit. Historically, PDP-8's used 12-bit bytes and Crays also used 32-bit bytes.

(EDIT: fixed PDP-11 -> PDP-8, thanks kabdib!)

PDP-11s always used 8-bit bytes, you're probably thinking of the PDP-8.

I don't remember what the popular sizes for the PDP-10 were. A 36-bit machine, none of the choices were really great. (I know that CLU on the PDP-10 used 36 bits).

Just one counterexample: Texas instrument C28x core is 16-bit addressable. Each byte is 16 bits.

>The TMS320C28x byte is 16 Bits. By ANSI/ISO C definition, the sizeof operator yields the number of bytes required to store an object. ANSI/ISO further stipulates that when sizeof is applied to char, the result is 1. Since the TMS320C28x char is 16 bits (to make it separately addressable), a byte is also 16 bits.

http://processors.wiki.ti.com/index.php/Byte_Accesses_with_t...

C is the closest thing to universally portable assembler.

I fail to see how C is more portable than any other memory safe systems programming language, other than having the fortune that for 30 years OS and hardware vendors decided to only support C compilers.
Compilers. Compilers for everything.

You can hype your favorite systems language all you want, but if there is no compiler, it wont compile.

>OS and hardware vendors decided to only support C compilers.

If you want your favorite language X to work everywhere C does. Just make X-to-C compiler.

X-to-C compiler isn't as nice as it could be, because you're writing code for some weird chip and need to twiddle some bits for IO using a macro defined in a header somewhere, and that doesn't play nicely with your pet X-to-C compiler.

Not an ideal situation, for sure, but a real one.

It's possible for translator understand headers and macros and generate bindings for them. Even for function macros. For example, GNAT has macro binding although it's not complete.

Btw. There is nothing "pet" in X to C translators. You pay good money for a good Ada to C compiler.

Or an RPG II to C translator/compiler :-)

A job I had in the early 90s, maintaining and extending such...

Of course, but that is an issue of implementation availability not language quality.

I remember the days when using C meant getting an account to the university (or work) UNIX system.

But that's the most important thing, isn't it? Being able to compile and run your code? I mean, the English language sure isn't popular for its intrinsic merits either, but here we are.
Sure, but in a way C is the JavaScript of systems programming languages, in a deep need of a "The Good Parts" book as well.
I'm sorry, but is this comment a joke? Have you ever read "The C Programming Language", and seen how simple C is? These arguments that the two OPs are having are important, but only when faced with obscure architectures that nearly nothing but C will run on.

Conversely, JavaScript's failures are not because of the wide variety of architectures that it supports (a tiny fraction of C's), but instead on the monolithic and bad design of the language. If you read "The C Programming Language", you'll realize that in the ~240 pages of text, you know basically every feature of the language. It's small, and it's elegant, and it's portable. JavaScript is none of those.

It is not a joke, rather the conclusion of someone that was already dismayed with C back in the mid-90's.

> If you read "The C Programming Language", you'll realize that in the ~240 pages of text, you know basically every feature of the language.

Except for those little compiler specific little issues, it seems:

http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

http://blog.regehr.org/archives/213

http://blog.regehr.org/archives/226

http://blog.regehr.org/archives/232

> It's small, and it's elegant, and it's portable

Have you had the pleasure to port C code across multiple C compilers and OS in the mid-90's?

Agreed. (yes, did a lot of C in the mid/early 90s that ran on sundry unices, DOS, OS/2, Windows & VAX). Ouch.

"The C Programming Language" isn't that great of a book, either. I had FAR better luck making sense of C (after having been in school for almost 4 years, and working part time on another language for almost 2 years) with the book "C, An Advanced Introduction" (1st version, not "ANSI Edition"), by Narain Gehani. For somebody transitioning from real high level languages, the array/pointer/string swamp was a real source of anti-productivity fun and games until explained by somebody other than its creators.

When you're done with The C Programming Language, read Expert C Programming, C Traps and Pitfalls, and The C Puzzle Book. Then tell me how simple C is.
Have you read "Expert C Programming: Deep C Secrets"? That book will show you that a simple language does not mean that programming in it is simple.

On a related note, I believe "The C Programming Language" is not a good book for the realities of 2016.

There's two recommendations here for "Expert C Programming".

I read it cover to cover fairly recently; I don't think there was really anything I disagreed with, but there also wasn't anything I didn't already know and wasn't already practicing. While I've been programming C a while now, that surprised me. I've decided that the book isn't bad but is probably more "help get you from beginner to intermediate" than "help get you from intermediate to expert", much less something of use to an expert (sadly, in my position).

An alternative interpretation would have it that there is just not enough to say about C that an expert-level book is likely to say things a C expert would not already know. That would argue against your point about the complexity of C. As I say, though, this has not been my conclusion.

[1] Except that Eliza was named after the character from Pygmalion - I'd somehow never put that together!

There are tons of such books.
> Bytes and octets are the same today.

Not in DSPs. Today.

Hi, I'm working on one. It's a recent DSP with pretty massive processing power that's powering some really, really expensive machines (six figures a piece). A byte is 32 bits here. This is common on a lot of VLIW machines.

I guess uint8_t et co. appeared out of the plight of many systems programmer who needed to do things like "hand these precisely 8 bits to this peripheral". On this machine, getting only these specific 8 bits out of a char requires bit twiddling trickery (it doesn't support unaligned memory access), but if the peripheral you're talking to (e.g. a temperature sensor) calls 8 bits a byte, you're going to have to send it 8 bits, not teach it that a byte doesn't always mean 8 bits.

Prior to stdint.h (and on any platform without a C99-enable compiler, which are neither few nor unused), we used to work around this crap with a bunch of custom macros defined in hardware-specific header files. It was a mess and you needed to learn a lot more about every specific platform's compiler than you ever cared about. And it was particularly beautiful on bi-endian platforms, oh yes, those were so beautiful.

C99 came and said you know what, this is so platform-and-compiler-specific that the compiler should worry about it and that accounted for a massive relief of about 20% of my daily stress dose.

You should use it when you know that you need something to be exactly 8 bits long. Which may or may not be a byte (that's why it frickin' says int8_t and uint8_t, not byte and ubyte).

Edit: I'm going to piggyback on my comment and gently ask anyone to consider not blindly following the advice to inline the declaration of b in code like this:

    void test(uint8_t input) {
        uint32_t b;

        if (input > 3) {
            return;
        }

        b = input;
    }
In many (most?) cases, that's good advice, but declaring it (and any other local variables) in the beginning has the major advantage of allowing me to read the first line of the functions and tell how much space is being allocated on the stack when the function is called.

This is not only nice to know for performance reasons, it's also useful information for platforms that have limited stack size or -- my favourite! -- platforms that lack a MMU, so when your stack runs into useful data, your system doesn't crash, it just starts acting funny.

The XAP processor architecture use in CSR Bluetooth Chips (design orginally from Cambridge Consultants) is one such processor.

On it: A 'byte' is 16 bits. A word is 16 bits (so sizeof(int) == 1) A function ptr is 24 bits (so sizeof(Fx) == 2)

CSR have sold well over a billions chips. There is a strong argument that this is the 3rd most popular architecture family after ARM and X86/64.

Things that x86 programmers do that won't work because they make assumptions the C standard doesn't specify: Cast a uint16_t* to uint8_t* to extract the upper and lower octets of the uint16 (a uint8_t array is actually in memory a uint16_t array using only the lower 8 bits) Store a function pointer in a void* (throws away upper 8 bits of the 24 bit func pointer).

Would you port x86 code, like say the Apache web server, to this chip? Or is it basically a memory constrained little device that you are writing a driver for, from scratch?

Seems like you would learn the rules, even though they are a bit odd, easily enough while writing something from scratch, or maintaining an existing (somewhat) small driver program.

Agreed, though, be aware (or just beware?) that this chip is quite different, and new skills MUST be learned.

What's up with creating "empty" repositories for articles? Isn't there a better solution? I think using gist would be more suitable if you want the article to be attached to your github username...
Ironic that he begins by taking issue with the idea that you should avoid writing C if you can, then proceeds to provide the best evidence possible for why it's true. Why on Earth would you voluntarily code in a language where people can debate something as simple as which type to use for integers, unless you absolutely had to?
Because it's simple and low-overhead and compilers for it exist on every lump of hardware I've ever had to work with, from a clunky x64 to a PIC to a TigerSHARC to a bizarre DSP thingy from a tiny fab lab somewhere.

A great many of the problems that people experience with C code can actually be nullified (zing!) by finding a competent C programmer who does the job properly. A tool being easy to misuse doesn't mean we shouldn't use it; it just means we should demand high standards of the programmers and not employ some chancer who learned how to program from the back of a cereal packet. I give young children plastic safety scissors. I give competent adults chainsaws and scalpels.

See, that is the problem right there.

Even if one does find such competent programmers, I doubt they the way C culture works it would help.

Most C devs I have met, if given a memory safe systems programming language like Modula-2 or Rust, would just use SYSTEM or unsafe {} everywhere without any proof of a profiler if it really matters to the customer.

As for being available everywhere, that is an historical accident of C's adoption. I remember when C was only available in UNIX and we had things like RatC.

Slightly off-topic, I'd say that the majority of programmers, C devs included, are barely competent. They know enough to bash through to something clunky and dangerous that kind of does what's needed at a demo. Whilst it's true that taking away all the dangerous tools would make it harder for this great sea of perpetual amateurs to do so much damage, I think it's not right to blame the tools.

By simply being here and reading this, you're almost by definition in the top 20% of programmers. Hyperbole, yes, and counter-examples exist, but anyone who routinely comes here will improve; the majority of programmers never do.

Again, if people can't agree what type to use for integers, then it's not "simple".

And yes, a tool being easy to misuse absolutely means we shouldn't use it unless we have to. A competent adult would never use a chainsaw or a scalpel except for tasks where nothing else would work.

Pick the integer type that is best suited for your use, given your circumstances.

I can't make it any simpler. There is no one true magical perfect solution for all circumstances in all places at all times. This is why I employ competent programmers; to make these choices.

A competent adult would never use a chainsaw or a scalpel except for tasks where nothing else would work.

Of course they would. I do, frequently; not chainsaws and scalpels, but there are dangerous tools that I am really good with that I can produce really good work with, quickly and efficiently, and there are safe tools that would mean I take longer and can't do exactly what I want to do and produce a less satisfactory result. I don't pick the safe tool; I pick the right tool for the job, given the circumstances and the requirements.

Are there people who deliberately pick the tool that will give a worse outcome? If one of the tools available is something that the person using it isn't competent to use, that doesn't mean the problem is with the tool.

I freely accept that a good C programmer needs knowledge and experience beyond that of many programmers. This is not somehow a bad thing.

>Pick the integer type that is best suited for your use, given your circumstances.

>Pick the fission reactor type that is best suited for your use, given your circumstances.

I cannot tell if you are in favour of, or against, the principle that one should pick the right tool for the job.
Saying "just pick the right tool" is a worthless statement when the question is the difficulty of doing that. Would you enter a discussion about curing cancer or depression and announce "just pick the right cure for the disease" and then get upset when people don't agree with you?
Now I can understand your argument. I upvoted you because I like to encourage clarity. This sounds patronising but that's the limitations of a text interface for you.

I think you're arguing that sometimes it's difficult to work out what the right tool is for the job. That's true, and not what I'm talking about. I'm saying that the principle of selecting the right tool for the job is sound, and that sometimes C is the right tool for the job and should be selected.

The chain of discussion:

>Because [C] is simple

>Again, if people can't agree what type to use for integers, then it's not "simple".

>Pick the integer type that is best suited for your use, given your circumstances.

It had nothing to do with deciding to pick C. It was about the difficulty of picking integer types in C being a demonstration of how C isn't simple. That doesn't mean C isn't sometimes the right tool for the job. It does mean that C isn't simple.

Look, nobody cares how clever and talented a programmer you are, or think you are. Please just drop the unsubtle boasting and passive aggressive insults, they only make you sound like an insecure egotist. Concentrate on the actual issue.

In 99.9% of languages, there is one true magical perfect solution to a question as basic as "what type should I use for integers?", and it's "use the integer type". The fact that C not only has options in this case, but that experienced developers seemingly cannot even agree on best practices, is a very good reason to avoid using the language unless you have to.

The reason why C has several integer types, is because there are different ways to represent integers for computers. Everything else is convention.

I mean, if that bothers you, then meta-programming in Ruby and Python, as well as different lisps should really bum you out.

As a society, we still avoid tools that are easy to misuse whenever we can. Modern chainsaws have several safety features that were missing from earlier ones and that doesn't make them less useful.

Just because the most popular systems language is dangerously unsafe doesn't mean a safe alternative couldn't replace it (like, say, Rust).

Those compilers exist because of path dependency from unix not from any unique qualities of C
This makes no sense and I believe you are incorrect. When Analog Devices churned out the TigerSHARC processor, they did not decide to write a C compiler for it because of unix path dependencies.
C become widespread because of UNIX, just like JavaScript got widespread because of the browsers ubiquity.

Eventually developers wanted to take home to their 8 bit computers what they were coding at the university and work.

So RatC, Small-C and many other C dialects were born.

Just like node.js nowadays, with UNIX adoption becoming the basis of the 80's graphical workstations, the pressure to have easy access to C compilers increased.

Also back in the day, one paid for compilers, so the majority went to whatever what the OS official system programming language.

So this is how we ended up in this situation.

When Analog Devices churned out the TigerSHARC processor they wrote a c compiler because "C is what you use for low level code" not Forth, Pascal, Modula, Ada, Fortran, Algol, a lisp or any of the other hundreds of programming languages that have been used to write programs directly on metal on computers that the TigerSHARC outclasses by magnitudes.

and "c is what you use for low level code" because decades of Unix. Decades of C being the lowest (non-assembly) level of programmers' systems, decades of being the only language that doesn't make you install an additional runtime, decades of accumulated wisdom...

But those are a side effects of Unix winning, not C qua C.

I think what the discussion about this statement misses is the "unless you need to" part. There are cases where you should use C! It is not "never ever use C".

If the use-case is being able to run on everything from TigerSHARC to x64, you probably should use C. If you want a GUI based Windows App or a webapplication, that might not be a "you need to use C" case.

The key phrase here is "if you can avoid it".

When your target platform is an 8-bit microcontroller or an embedded system with hard real-time requirements, your options are very limited.

But there are plenty of situations where there are alternatives, and often these allow you to solve your problem in less time, with much lower risk of bugs than in C. As always in life, one has to evaluate one's options, consider their respective advantages and weaknesses and then make a choice.

If I had to write a document like that, I would use a different phrasing from "if you can avoid it". But the sentiment - to avoid using C where superior alternatives exist[1] - is something I agree with.

> A great many of the problems that people experience with C code can actually be nullified (zing!) by finding a competent C programmer who does the job properly.

That is kind of like saying the solution to traffic accidents is only letting people drive that don't cause accidents or demand of people that they drive sufficiently carefully. Sure, that is something worth wishing for - but it is not very realistic to expect this to actually happen.

Not that I disagree that C is a very demanding language. Even the best programmers make mistakes. But this is the reason the author of the original document the linked article refers to advised people to avoid C where possible. Most programmers will be better off for most of their projects with a language like Go, or Python or whatever.

[1] Of course, if by "alternatives" I mean languages like Perl, Python or Ruby, it is worth keeping in mind that their main implementations are written in C.

> to avoid using C where superior alternatives exist is something I agree with.

Who wouldn't?

> When your target platform is an 8-bit microcontroller or an embedded system with hard real-time requirements, your options are very limited.

You mean like the CPUs supported by MikroElektronika's Pascal and Basic compilers?

http://www.mikroe.com/8051/

http://www.mikroe.com/pic/

http://www.mikroe.com/avr/

I get your point. However, I did not say there were no alternatives, just that they were "very limited" (when compared to, say, Debian running on amd64).
This comment only fans the flames of language war, it doesn't contribute to the discussion.
This is simply not true. The barrier to a NEW person starting to write C code in 2016 - imagine some 20 year old college student - is "Am I ready to pick a denomination and religion and read hundreds to thousands of pages of gospel?" There is no such barrier with any other language.

You literally cannot give any examples of correct C code as a solution to a problem: literally, experts will disagree with you.

I've had experts tell me that cplusplus.com/reference gives horrific examples you should never follow. The newest addition of The C++ Programming Language says that the examples are not meant to be production code you can actually use as-is. (I can dig this up later if you want.)

C and C++ are more debates in the form of a language than languages per se.

This is exactly my point. You're basically telling me that C is a horrible language. Why? Do you think I disagree with you? Do you think that you're giving me new information? Or are you just doing your part in the language war, perhaps repeating talking points that we're frankly a bit tired of? There are better things we could be talking about.
I didn't say C++ is a horrible language, and I'm sure I didn't say it because I don't really think this.

I would think you're being downvoted because there aren't really any language wars around the choice of C/C++ - it's not like PHP vs Ruby or Python or something. You're imagining a war where there really isn't one.

Usually people writing C or C++ don't have much of a choice, and usually people choose these languages due to the requirements. For example, writing embedded microcontroller code that will run on a $0.70 chip. What are you going to do, write it in Rust?

Nobody would dream of telling Linus Torvalds to refactor the Linux kernel into some other language - it's not just a ridiculous suggestion, it's like asking the United States to move to a different continent - it's a total non sequitur, it's not even in the realm of possibility.

Basically the advice "you should avoid writing C if you can" is totally uncontroversial, and for I would say 99% of people who woke up today, January 15th, 2016, a weekday, and wrote C code, the answer to "can you avoid it" is no. (Not just due to their position/employment but due to the very requirements of the project itself.)

Sure you should avoid writing C and C++ when you can, but for some classes of problems that is "never."

> imagine some 20 year old college student - is "Am I ready to pick a denomination and religion and read hundreds to thousands of pages of gospel?" There is no such barrier with any other language.

Haskell comes to mind. There's a denomination and a religion. I'm not sure that the gospel runs to hundreds of pages, but there are many prophets who tell you that you need to know category theory, so for that sect, yes, it's hundreds or thousands of pages.

Lisp also has a denomination and a religion, but the sacred texts may be shorter.

> but there are many prophets who tell you that you need to know category theory

I challenge you to show me one!

If I knew of a decent search function for HN comments, I could show you several. As it is, I can only offer my memory of what I have read in the comments here.

My impression is that Haskell "experts" here say some variant of "of course all this is obvious because category theory" (implying "you have to know category theory or you're an uneducated heathen") at least three times as often as they say "you don't really need to know category theory".

(not to your point but on "If I knew of a decent search function for HN comments"-- just so you know the search https://hn.algolia.com/ is fantastic - I've never failed to find a HN comment)
That is the basic problem with show and tell with a low-level language. You either have a simple working example, with a dozen edge cases not caught. Or you have the usual check-hedge growth around Input and Output - which makes the Code hard to read. They chose the correct examples for the task- thus they where right, and your expert was wrong.
What? This comment is directly related to whether advice "don't use C if you can avoid it" is good or not.
And arguing that point is just the language war, nothing more, nothing less. I'd get as much mileage telling people that they shouldn't use Emacs unless they have to.
I think you're wrong. It's valid to discuss the pros and cons of languages, and it's valid to discuss whether a language is a good tool for any job.

But your thoughts are still valid and contribute to the conversation.

You voluntarily code in C for the same reason that your OS kernel's 'yield' is written in assembly. How do you push the addressing mode register to the stack when switching to another task in your high level language of choice?
I like how the old Borland Turbo Pascal had little extensions to let you move data between program variables and registers, as well as put in little bits of assembler.

Of course, that only worked on x86 on MS-DOS :-(

"Why on Earth would you voluntarily code in a language where people can debate something as simple as which type to use for integers, unless you absolutely had to?"

Because we have no alternatives so far. I do C programming for a living, but I'd be more than happy if there were some type and memory safe language I could use to program any tiny, obscure MCU, that could stay at least in the same order of magnitude of speed and let access hardware easily. Rust might be a step in a right direction, but it still has a looong way to go to become a viable alternative to C in embedded world. Of course, for a modern, high level development (web, mobile etc.), I see no reason to use C (maybe except for tiny libraries or co-routines where speed is critical). C is simply a tool -- it's neither good, nor bad -- and as every tool, it has it's use. You can easily break your fingers with a hammer, but that doesn't mean that it's a bad tool for driving nails in.

You're aware that "we have no choice" does not answer a question about why you would do something voluntarily, right?
I just confirm, that majority of C programmers nowadays do that because of "they absolutely had to".
What do you think Rust needs to do to become viable in the embedded world?
Ecosystem and community. Most of the time you can find what you need in C already written, in Rust it's do-it-yourself now.
> But zeroing a structure with memset, though it will set any integer members to zero, is not guaranteed to set floating-piont members to 0.0 or pointers to NULL (though it will on most systems).

Why is that? On those systems, is a sequence of 0 bits not considered 0.0f or NULL?

Not sure about fp, but the C standard very carefully and explicitly notes that NULL pointers doesn't mean a runtime value of 0. The comp.lang.c FAQ 5.17[0] further provides examples of nonzero NULL pointers e.g.

> The CDC Cyber 180 Series has 48-bit pointers consisting of a ring, segment, and offset. Most users (in ring 11) have null pointers of 0xB00000000000.

This is a bit confusing because the "null constant" is an integral 0 e.g. `(void ) 0` is a null pointer, but it's defined as an arbitrary compiler instruction. `(void ) foo` where foo is 0 at runtime may not be a null pointer at all.

[0] http://c-faq.com/null/machexamp.html

One of the most important reasons to use calloc, other than the zeroing of the allocated memory, is the fact that it checks for integer overflows (at least on most implementations).

For example, when allocating an array of n elements, in malloc you would do something like the following:

ptr = malloc(sizeof(int) * n);

Which could potentially lead to an overflow of the size passed to malloc, hence allocating a significantly smaller buffer and end up accessing adjacent memory which hasn't been allocated to the buffer (buffer overflow)

Calloc requires two arguments, one for size and one for number of elements, which allows it to check for an overflow before performing an allocation (e.g. by using SIZE_MAX):

ptr = calloc(n, sizeof(int));

This is not to say that zeroing memory is not useful. There are many situations in which a zero in a memory block represents the end of the usable data, as for example in a string, so zeroing memory in those situations is definitely recommended.

Using calloc instead of malloc, just for checking arithmetic overflow, is superfluous. You can always write a check or a wrapper for malloc that does that automatically. It is so easy I can write it here:

  _Bool Check( const size_t a , const size_t b )
  {
    if( a > SIZE_MAX / b )
    {
       return false;
    }
    return true;  
  }
(If proper warnings are enabled, it also provides extra integer type checking.)
Agreed, but there's little point in reinventing the wheel in this particular case.
There is no reinventing going on here. In C you have to write relatively low level code. This includes manually calling allocs for objects, which includes the code for checking your arithmetic. At some point you will have to write that code. If you are smart you will wrap it into a function.
I don't see the point of the discussion, that piece of code (or relatively equivalent), is already being performed by a libC function (calloc, in this case), how is that less convenient than you writing it yourself?

It has nothing to do with how smart you are or how low level C is, you are effectively reinventing that part of calloc by wrapping malloc.

There's a point to be made that people will be idiots and risk bugs for the sake of efficiency.

That's why OpenBSD has reallocarray(3) (which serves as a fine malloc, though the kernel not having reallocation has mallocarray(9))

>size_t is defined as "an integer capable of holding the >largest array index"

>>No, it isn't.

Could anyone elaborate?

I don't see how could someone allocate more than SIZE_MAX bytes with (m/c/re)alloc since they all take size_t as an argument, and size_t is the type used for defining array sizes. Thus size_t can hold the largest array index (in fact it can hold the largest_index+1 since arrays are zero based). Any counterarguments?

Maybe he's taking it very literally, as I think size_t is literally defined as 'the integer type of the result of the sizeof operator'. So it literally isn't defined as 'an integer capable of holding the >largest array index' as that's not the definition and words they use.
But if sizeof returns size_t, then your can't have indexes larger than that. Otherwise you could point beyond the largest in-memory object possible. (Using smallest (byte) indexing)

But yes, he may be arguing the definition rather than meaning.

The counterargument is actually relatively simple, because it's an argument over what the definition of size_t is, and you can simply go look to the C standard and find the definition there.
I understand.

If I change the argument to: size_t is capable of holding the largest array index", then this must be correct, or is there still something I missed?

> Converting to uintprt_t will give you some result -- but that result won't be useful. Unless you're writing low-level system code, don't do any pointer comparison or arithmetic unless both pointers point into or just past the end of the same object.

I disagree. There are situations where you need an ordering between unrelated objects, but the precise order doesn't matter, only that it's consistent.

An example is when you have a complex data structure with one lock per node, and you need to lock more than one node at a time for some algorithm.

To prevent a deadlock, you must always take the locks in the same order. Comparing the address of the locks gives you a total order over all the locks, which is enough.

I think comparing (using <) pointers to unrelated objects is undefined behavior. So you do not get a consistent ordering.
The part I quoted is talking about casting the pointers to uintptr_t (which is defined behavior) and comparing the resulting uintptr_t (since it's an integer, comparing it is defined behavior). It's not talking about comparing the pointers directly (which is undefined behavior). His argument is that, while defined behavior, "(uintptr_t)ptr_a < (uintptr_t)ptr_b" is useless; I disagree, since it's a well-known trick to prevent AB/BA type deadlocks.
I find myself to disagreement with almost ALL the points this "critique" makes.

They are either minor pedantic corrections, "it's how it's always been done" things, or "you might have your reasons for doing it in a bizarro way" affairs.

Sorry, but for 90% of use cases, the original article has better advice.

Agreed. Most of the integer stuff is pedantry ("doesn't make it non-standard" -- who cares, the point being made was that these types are better). There are some valid and useful points being made, but most would serve better as caveats on the original article rather than straight out "don't listen to this".
> int in particular is going to be the most "natural" integer type for the current platform. If you want signed integers that are reasonably fast and are at least 16 bits, there's nothing wrong with using int

The slight performance gain on a few systems that you get from using int rather than int16_t is very rarely worth the hugely increased risk of introducing platform-specific undefined behaviour, IMO.

> float and double are very commonly IEEE 32-bit and 64-bit floating-point types, particularly on modern systems, but there's no guarantee of that in the language.

True. Is there an alternative? Something like float32_t?

> But more often all you need is a particular range of values. For that, you can use either the [u]int_leastN_t or [u]int_leastN_t types, or one of the predefined types.

Sure - but again, how much performance do you gain from using a _leastN type, and how much reliability do you lose?

> It's capable of holding the size of the largest object your implementation supports. (There's an argument that that's not necessarily guaranteed, but for practical purposes you can rely on it.) It can hold the largest memory offset if all offsets are within a single object.

But you just said in the previous section that it only makes sense to perform pointer arithmetic within a single object anyway.

> There is a widespread convention, particularly in Unix-like systems, for functions to return 0 for success and some non-zero value (often -1) for failure. In many cases different non-zero results denote different kinds of failure. It's important to follow this convention when adding new functions to such an interface. (0 is used for success because typicallyi there's only one way for a function to succeed, and multiple ways for it to fail.)

The convention is not as widely accepted as this section implies. When adding to an existing system one should follow that system's conventions. When writing for Unix one should follow the Unix conventions. But there are other conventions (e.g. the VMS one) that also see use in C.

> I don't often use automatic formatting tools myself. Perhaps I should.

Yes, you should.

> Zeroing memory often means that buggy code will have consistent behavior; by definition it will not have correct behavior. And consistently incorrect behavior can be more difficult to track down.

Disagree strongly. Consistently incorrect behaviour is much easier to track down than inconsistently incorrect behaviour.

> if you're trying to program defensively, you might consider initializing allocated memory to some value that's known to be invalid rather than one that might be valid.

Agreed - but how? One of the biggest problems with C is that it is extremely hard to mark a state as invalid (hence the whole integers for error codes discussion earlier, where really an error code should have a different type from a valid integer result).

>The slight performance gain on a few systems that you get from using int rather than int16_t

int16_t is also unportable as he mentions later

>the hugely increased risk of introducing platform-specific undefined behaviour

What risk? Please explain how int introduces undefined behaviour.

>how much performance

It's not about performance. intN_t is NOT guaranteed to exist.

>But you just said in the previous section that it only makes sense to perform pointer arithmetic within a single object anyway.

Yes and that's right.

> int16_t is also unportable as he mentions later

It's theoretically unportable. I have yet to see a real system to which it's unportable.

> What risk? Please explain how int introduces undefined behaviour.

(Signed) integer overflow is undefined behaviour. It is very common to develop on platforms on which int is 32 or 64-bits. This carries a high risk of accidentally overflowing the limits of a 16-bit integer and not realizing. When you do this you don't notice anything (because it doesn't show up on your development machines), but you've introduced undefined behaviour that can easily manifest in practice on machines with 16-bit int.

Isn't another problem of "ptrdiff_t diff = (uintptr_t)ptrOld - (uintptr_t)ptrNew;" that you're assigning an unsigned value to a signed variable?

If ptrNew is larger than ptrOld this can give a very large value via wrapping around, so large that it will probably be bigger than the max value of ptrdiff_t, making the assignment a form of signed overflow.

The assignment is a conversion from unsigned to signed, if the value cannot be represented by the signed type, the result is implementation defined or an implementation defined signal is triggered.
Right. Implementation defined, not undefined. Still not great for portable code, which is what the article is all about.
It's also very dangerous.
> The first rule of C is don't write C if you can avoid it.

My C skills are non-existent, but it occurs to me while reading the response that when two people who have so carefully considered the subject can arrive at such different places... then yes, you probably shouldn't write C if you can avoid it.

C is the only language that will run on certain strange systems, and that is the only reason that most of these quibbles even matter. It's more of an argument of "What is the most portable way to write C?"

C is a very small and elegant language that never lies to you. There aren't any useless functions that are specific to one use case, and things that you don't really, really need are simply left out of the language. This means that instead of spending hours learning the language and all of its quirks, you get to focus on the problem that you're trying to solve, and come up with the most efficient solution possible (there's a reason why almost all performance tests use C as 1). You don't have any language quirks like those that are discussed in this article, unless you are running an architecture that has similar quirks, in which case you're going to have quirks no matter what you do. Edge use cases, as with everything, may cause a bit of quirkiness.

That being said, don't be scared off from C just because of these pedantic arguments.

That's a great point; thanks for the context.
> C is a very small and elegant language that never lies to you.

I would call undefined behavior and compiler optimizations lies.

The discussion usually boils down to

a. We need a better C

b. We need better programmers.

Personally I believe C tries to hard to be high level instead of what it really is. Newer revisions should try to make it lower level. For example I never could understand why we need enums. Enums are very useful for Java or Golang which are high-level. C never needs enums. Why should I ever put const in the argument of a function? It makes sense for D but for C no. Why should I ever use complex in C. Is there any HW that supports complex numbers? If it becomes standardized, then why not?

C should be good to write kernels and simple utilities and simple compilers. For the rest, use a high-level language.

The thing is, such higher level systems programming languages were already a thing in the 60's.

C designers just wanted to hack something for their toy OS and here we are.

> For example I never could understand why we need enums.

> Why should I ever put const in the argument of a function?

Reducing potential state and conveying programmer intent. Both lead to safer and more performant code (by way of compiler optimizations).

What you say is incompatible with the "Cross-platform assemble", simple language ideals.

If that was the intent why not throw namespaces and other goodies in?

Well those ideals are pretty subjective, I'd say that the ideal language would be flexible, performant, safe and productive. As those goals conflict with one another, the ideal language would find the perfect balance. I don't know if C has found that balance, but it has had a pretty good run.

Defining sets (enums) and memory access constraints (consts). I wouldn't describe that as high level from a conceptual or practical perspective. Namespaces are simple conceptually but not in practice. The same can be said about the beautiful logical consistency of S-expressions.

We've had "better C" since the 70s (since 68 depending on your view of Algol68). Programmers didn't want to use them. I can't tell you how many times I heard "having to to type BEGIN and END sucks" as though that was a valid argument against a language that had none of the issues C has.
Reading this as someone who neither loves not loathes C, I would not have written this as a series of rebuttals. It makes the essay seem defensive and nit picky. I get more of a sense of emotional attachment to a particular way than any sort of reasoning.