> Simple rule: include files should never include include files. If instead they state (in comments or implicitly) what files they need to have included first, the problem of deciding which files to include is pushed to the user (programmer) but in a way that's easy to handle and that, by construction, avoids multiple inclusions. Multiple inclusions are a bane of systems programming. It's not rare to have files included five or more times to compile a single C source file. The Unix /usr/include/sys stuff is terrible this way.
> There's a little dance involving #ifdef's that can prevent a file being read twice, but it's usually done wrong in practice - the #ifdef's are in the file itself, not the file that includes it. The result is often thousands of needless lines of code passing through the lexical analyzer, which is (in good compilers) the most expensive phase.
> Just follow the simple rule.
This is a ridiculous stance, in my opinion.
Part of the appeal of libraries is abstracting away implementation details. I want to be able to have foo.h #include a hash table, or a binary search tree, depending on #ifdef's or its internal implementation. The user of foo.h shouldn't even have to know about it.
Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to resolve the problem.
Rob's resolution is an instance of worse-is-better: Instead of slightly complicating the lower-level tool (in this case, the preprocessor), let's complicate all user code in existence.
> Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to resolve the problem.
There are reasons why actual C/C++ compilers don't do this. Additionally, recursive includes is a major reason why C++ compilers are slow.
This construct is commonly known as a wrapper #ifndef. When the header is included again, the conditional will be false, because FILE_FOO_SEEN is defined. The preprocessor will skip over the entire contents of the file, and the compiler will not see it twice.
CPP optimizes even further. It remembers when a header file has a wrapper ‘#ifndef’. If a subsequent ‘#include’ specifies that header, and the macro in the ‘#ifndef’ is still defined, it does not bother to rescan the file at all.
If a file is intended to be read multiple times with different results, then obviously you don't want to prevent it being read twice. However, we were talking about preventing the hit from reading a file twice unnecessarily. The quoted text says it doesn't bother rescanning if the macro is still defined. How do you get a different outcome from scanning the same file with the same define? Can you provide an example where this breaks things?
I do not understand why you are arguing about this.
There are two scenarios: in one you want to prevent a file from being read multiple times because it wastes resources (and in some sad cases, might cause issues with redefinition). In this case, you use the include guards and the compiler can do a trick to optimize out unneeded scanning.
You have this other hypothetical scenario in which you purposefully want a file to be read multiple times, with a different outcome (this kind of trickery isn't usually possible at all in other languages with module system so it is funny to argue about C being broken). Now I don't think this is a common scenario or even a good one to be in, but it is possible. The solution is simple: don't try to prevent the file from being read multiple times. Why would you try to prevent it, if it is meant to be read multiple times?
You are arguing that a door is broken because you cannot walk in through it if you close the door. Don't close the door if you want to walk in...
Because it is an optimization that relies on well behaved developers and we all know how good such assumptions end, specially in big teams with high attrition.
As I read this, if the compiler determines that (with the #define set) the file is effectively empty, it knows it can just check the #define next time rather than re-reading. Without this optimization, you get the same result slower. It doesn't depend on well-behaved anything to get a result just as correct as without the optimization.
With the single possible exception of a file that changes on disk between the multiple includes in the same build step. That is not a situation I have ever encountered and I am not at all sure it is supported and there aren't a bazillion other things that would break it in any given compiler chain (certainly at least line directives would be wrong, but that's arguably minor I guess).
When the preprocessor sees that, it marks it off and never reads the file again (Unless you were to #undef INCLUDE_FOO_H, but that's an extremely unlikely scenario since it's likely to break your code). If you want to be able to include a file multiple times, just don't use include guards. Even without this optimization your idea still wouldn't work if you placed include guards in your #include file. All this does is speed up preprocessing when we already know that the file has no new lines to be added.
I am not talking about include guards, rather macros that affect the outcome of specific file.
It is not a very clean way of coding, but there are many enterprise developers that provide files without include guards and include them multiple times.
An example would be generic data structures in C done with macros.
It's pretty unlikely that list.h would have a trivial include guard, so I don't see how the above optimization would break it. An include guard in list.h would prevent you from using it properly with or without the optimization.
If list.h has no include guard, it gets included with or without this optimization. If it has an include guard and you #undef it, then it gets included again with or without this optimization. If you it has an include guard and you do not #undef it, it does not get included again, with or without this optimization.
In no case does this optimization change the code emitted.
Note that this only partially helps with a subset of the problem (and modules as proposed don't fix the issue, they just help a bit more): CPP still needs to make separate passes for each translation unit, for example. I'd recommend reading the link I put in my original post...
"Implements the simple state machine that the Lexer class uses to detect files subject to the 'multiple-include' optimization.
The public methods in this class are triggered by various events that occur when a file is lexed, and after the entire file is lexed, information about which macro (if any) controls the header is returned."
Modules improve things, but that doesn't mean one can do nothing to speed up lexing without them.
It can also be dangerous. Take (for instance) a header file that relies on a specific integer size. Using this scheme one could include a different/wrong typedef and get yourself into trouble.
It would be faster, but this would still cause the compiler to open and start parsing the file unnecessarily (possibly multiple times from multiple included files).
Another approach would be to have a pragma that means: "just don't bother including this file again if you see it", and have the compiler remember it. Perhaps there is - its a long time since I programmed in C.
I wish for a standard #once directive. It should be very simple to implement, increase preprocessing speed and reduce the size of visually disturbing boilerplate in header files.
Good point. Opening a file has non-trivial cost, so perhaps the added cost of burdening the compiler with avoiding needless includes is worth it. As a long time C hacker I hate to say this, but I guess the whole include mechanism is a bad design or, perhaps, a design that didn't scale well.
I always find it humorous to find posters here and reddit and other boards say these things and I believe they would make the same statement about the Ten Commandments from God.
There are plenty of things Rob says I strongly disagree with, and even if you're very impressed with his work, your comment is still a very silly appeal to authority.
Maybe you need someone to believe in. Stronger minds decide for themselves (interestingly enough which is the advice given by Rob in the intro of this article).
I think you've merely re-stated my point. A strong, independent mind makes determinations on ideas objectively without respect to the sources of those ideas.
And, I find Rob Pike completely unimpressive. I would suggest that you find other folks smarter than yourself (and preferably smarter than Rob as well) to blindly follow (if that is your thing).
I think you're confused. What is your point exactly? This whole discussion has been about how your original post was a baseless appeal to authority (which you still seem intent to make). My point is merely: if that is your sort of thing, then you can find someone better than Rob Pike to fill the role. How is this in any way the same point that you yourself are making?
I have plenty of people I believe in. In the sense that almost whatever they say, I'd think really really hard before disagreeing. Even then, if I find them unconvincing, I'll eventually disagree.
I personally don't hold Rob to be such a figure. I believe he's quite ignorant when it comes to many important topics in computer science (e.g: he made some comments that make it clear he does not understand parametric polymorphism). I find his work on Go to be a minor variant of Algol69 (and not necessarily an improvement, at that!), throwing away 40 years of real improvements in PLT.
I'm not a fan of Go, its syntax, or Rob Pike in the slightest but aren't go-routines as first-class/ default at least an improvement over most Algol-derived languages' support for concurrent execution models?
In the final analysis, this is hardly enough to convince me to use the language (as I've never found concurrency difficult in any other language to begin with and there are better ways than go-routines to make it 'easy'); however, I'm probably not in the target audience.
My impression is that Pike is endeavoring to create for his new corporate overlords a language that fungible enterprise developers can be at home in. And, it probably is better than Java.
Have you looked at the code for plan9? It follows this principle and I think it works fairly well. Its much nicer than wading through a large number of preprocessor directives.
All Plan9 C programs are written in this way. Paradoxically the number of includes in your code is minimal compared with the usual POSIX aproach. All C programs begin with two includes: u.h and libc.h and usually libio.h and it is often all you need for the startdard library.
The man pages of the libraries describe the libraries you need to include.
* Along with preprocessor optimizations to avoid reparsing protected headers, it can actually parse fewer headers in each compilation (reversing the performance argument by Rob)
* Can aid with greps for where modules are used
The comparison is really a strawman -- and the length of the GNU one and its many dependencies relate more to feature creep than to the #include issue here.
At least with GNU's #includes, I can see that it depends on the various stuff it needs for its feature creep.
Todays compilers may be fast enough so that using #ifndef that way would not make much of an impact, but back in 1989 when this article was written, presumably the time spent was much greater.
> Part of the appeal of libraries is abstracting away implementation details. I want to be able to have foo.h #include a hash table, or a binary search tree, depending on #ifdef's or its internal implementation. The user of foo.h shouldn't even have to know about it.
I agree with your premise (re: abstraction of implementation details), but I think you've applied it incorrectly to reach the wrong conclusion.
If the implementation of foo (contained in foo.c) uses a hash table or binary search tree, then foo.c may of course include whatever it likes. On the other hand, foo.h is (by definition) the interface to foo; any headers the interface requires (e.g. for function argument types) ought to be explicitly understood by the user of the library.
It depends on whether you delegate all memory allocations to the C file.
In the C code I work on, dynamic memory allocations are considered expensive and problematic (fragmentation, runtime errors) so we prefer to move the responsibility for allocation as far up the call-stack as possible, where it is often possible to minimize or coalesce dynamic allocations.
To do this, the interface files must expose the correct sizeof() so that the caller can allocate. We do this by splitting the header file into foo.h and foo_private.h (foo.h #includes foo_private.h). This separation is not for enforcement, but for API clarity and convention (if it's in a private.h, you have no business assuming anything about it).
This forces us to rebuild when the sizes of private structures change (which can be a significant downside) but allows us to avoid dynamic allocations, their costs and their many potential error paths.
In this approach, foo_private.h may very well #include a hash_table or whatever header which should not* be exposed to the #includer.
The ugly "includes hack" seems ridiculous today, but in the 90's such techniques were the only way to compile large C/C++ systems in a reasonable time.
Messing with includes and reducing file dependencies while less important nowadays are underappreciated elements of software design (called physical design as opposed to logical design).
For better historical context see:
John Lakos - "Large scale C++ software design" (1996)
" Procedure names should reflect what they do; function names should reflect what they return." - Never thought about naming this way, but is what I do (:
It's interesting to see how some of these opinions are reflected in Go, because Rob Pike helped design Go about 20 years after writing this essay:
Typography: Automatic formatting is part of the standard Go tooling and culture.
Function pointers / Protocols: This section seems to be decsribing a technique similar to Go's interfaces.
Include Files: Rob Pike talked about this in his presentation at SPLASH [1,8:00-12:30]. He mentioned that Plan 9 had fast compile times in part due to following this discipline. He contrasted that with one of Google's C++ code bases in which the compiler had to read 2000 bytes for every byte of source. That was largely because of includes, and was a problem for compile times. A little later he talks about Go's method of importing dependencies, and presents them as a big improvement to the situation.
52 comments
[ 5.2 ms ] story [ 125 ms ] thread> There's a little dance involving #ifdef's that can prevent a file being read twice, but it's usually done wrong in practice - the #ifdef's are in the file itself, not the file that includes it. The result is often thousands of needless lines of code passing through the lexical analyzer, which is (in good compilers) the most expensive phase.
> Just follow the simple rule.
This is a ridiculous stance, in my opinion.
Part of the appeal of libraries is abstracting away implementation details. I want to be able to have foo.h #include a hash table, or a binary search tree, depending on #ifdef's or its internal implementation. The user of foo.h shouldn't even have to know about it.
Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to resolve the problem.
Rob's resolution is an instance of worse-is-better: Instead of slightly complicating the lower-level tool (in this case, the preprocessor), let's complicate all user code in existence.
There are reasons why actual C/C++ compilers don't do this. Additionally, recursive includes is a major reason why C++ compilers are slow.
For more on this, see http://clang.llvm.org/docs/Modules.html
http://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html#Onc...
This construct is commonly known as a wrapper #ifndef. When the header is included again, the conditional will be false, because FILE_FOO_SEEN is defined. The preprocessor will skip over the entire contents of the file, and the compiler will not see it twice.
CPP optimizes even further. It remembers when a header file has a wrapper ‘#ifndef’. If a subsequent ‘#include’ specifies that header, and the macro in the ‘#ifndef’ is still defined, it does not bother to rescan the file at all.
C based toolchains are just broken.
There are two scenarios: in one you want to prevent a file from being read multiple times because it wastes resources (and in some sad cases, might cause issues with redefinition). In this case, you use the include guards and the compiler can do a trick to optimize out unneeded scanning.
You have this other hypothetical scenario in which you purposefully want a file to be read multiple times, with a different outcome (this kind of trickery isn't usually possible at all in other languages with module system so it is funny to argue about C being broken). Now I don't think this is a common scenario or even a good one to be in, but it is possible. The solution is simple: don't try to prevent the file from being read multiple times. Why would you try to prevent it, if it is meant to be read multiple times?
You are arguing that a door is broken because you cannot walk in through it if you close the door. Don't close the door if you want to walk in...
With the single possible exception of a file that changes on disk between the multiple includes in the same build step. That is not a situation I have ever encountered and I am not at all sure it is supported and there aren't a bazillion other things that would break it in any given compiler chain (certainly at least line directives would be wrong, but that's arguably minor I guess).
This preprocessor optimization is only for the case where include guards block off the entire #include file. Ex:
When the preprocessor sees that, it marks it off and never reads the file again (Unless you were to #undef INCLUDE_FOO_H, but that's an extremely unlikely scenario since it's likely to break your code). If you want to be able to include a file multiple times, just don't use include guards. Even without this optimization your idea still wouldn't work if you placed include guards in your #include file. All this does is speed up preprocessing when we already know that the file has no new lines to be added.It is not a very clean way of coding, but there are many enterprise developers that provide files without include guards and include them multiple times.
An example would be generic data structures in C done with macros.
In no case does this optimization change the code emitted.
"Implements the simple state machine that the Lexer class uses to detect files subject to the 'multiple-include' optimization.
The public methods in this class are triggered by various events that occur when a file is lexed, and after the entire file is lexed, information about which macro (if any) controls the header is returned."
Modules improve things, but that doesn't mean one can do nothing to speed up lexing without them.
Another approach would be to have a pragma that means: "just don't bother including this file again if you see it", and have the compiler remember it. Perhaps there is - its a long time since I programmed in C.
It's not standard, but it is supported by many compilers.
I always find it humorous to find posters here and reddit and other boards say these things and I believe they would make the same statement about the Ten Commandments from God.
And, I find Rob Pike completely unimpressive. I would suggest that you find other folks smarter than yourself (and preferably smarter than Rob as well) to blindly follow (if that is your thing).
I personally don't hold Rob to be such a figure. I believe he's quite ignorant when it comes to many important topics in computer science (e.g: he made some comments that make it clear he does not understand parametric polymorphism). I find his work on Go to be a minor variant of Algol69 (and not necessarily an improvement, at that!), throwing away 40 years of real improvements in PLT.
In the final analysis, this is hardly enough to convince me to use the language (as I've never found concurrency difficult in any other language to begin with and there are better ways than go-routines to make it 'easy'); however, I'm probably not in the target audience.
My impression is that Pike is endeavoring to create for his new corporate overlords a language that fungible enterprise developers can be at home in. And, it probably is better than Java.
The man pages of the libraries describe the libraries you need to include.
Example:
Plan9 cat: http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cat.c
GNU cat: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f...
Plan9 du: http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/du.c
GNU du: http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f...
I think it is nicer to have quite specific #include's:
Because it:* Gives a list of dependencies at a glance
* Along with preprocessor optimizations to avoid reparsing protected headers, it can actually parse fewer headers in each compilation (reversing the performance argument by Rob)
* Can aid with greps for where modules are used
The comparison is really a strawman -- and the length of the GNU one and its many dependencies relate more to feature creep than to the #include issue here.
At least with GNU's #includes, I can see that it depends on the various stuff it needs for its feature creep.
[1] http://plan9.bell-labs.com/magic/man2html/2/exits
This is an introduction to programming in Plan9 C:
http://doc.cat-v.org/plan_9/programming/c_programming_in_pla...
> Part of the appeal of libraries is abstracting away implementation details. I want to be able to have foo.h #include a hash table, or a binary search tree, depending on #ifdef's or its internal implementation. The user of foo.h shouldn't even have to know about it.
I agree with your premise (re: abstraction of implementation details), but I think you've applied it incorrectly to reach the wrong conclusion.
If the implementation of foo (contained in foo.c) uses a hash table or binary search tree, then foo.c may of course include whatever it likes. On the other hand, foo.h is (by definition) the interface to foo; any headers the interface requires (e.g. for function argument types) ought to be explicitly understood by the user of the library.
In the C code I work on, dynamic memory allocations are considered expensive and problematic (fragmentation, runtime errors) so we prefer to move the responsibility for allocation as far up the call-stack as possible, where it is often possible to minimize or coalesce dynamic allocations.
To do this, the interface files must expose the correct sizeof() so that the caller can allocate. We do this by splitting the header file into foo.h and foo_private.h (foo.h #includes foo_private.h). This separation is not for enforcement, but for API clarity and convention (if it's in a private.h, you have no business assuming anything about it).
This forces us to rebuild when the sizes of private structures change (which can be a significant downside) but allows us to avoid dynamic allocations, their costs and their many potential error paths.
In this approach, foo_private.h may very well #include a hash_table or whatever header which should not* be exposed to the #includer.
Typography: Automatic formatting is part of the standard Go tooling and culture.
Function pointers / Protocols: This section seems to be decsribing a technique similar to Go's interfaces.
Include Files: Rob Pike talked about this in his presentation at SPLASH [1,8:00-12:30]. He mentioned that Plan 9 had fast compile times in part due to following this discipline. He contrasted that with one of Google's C++ code bases in which the compiler had to read 2000 bytes for every byte of source. That was largely because of includes, and was a problem for compile times. A little later he talks about Go's method of importing dependencies, and presents them as a big improvement to the situation.
[1]http://www.infoq.com/presentations/Go-Google