From my perspective, all the complaints are solved by using a more modern build system, like Bazel. If you are already encoding dependencies between modules in your build system, then when you are searching for an import, it's trivial... you just look at the declared module dependencies. Bazel (and other similar systems) already force you to do this for other reasons, and so you theoretically get modules "free".
It's not like the same problem doesn't already crop up with generated sources. Like, if you see this:
#include "file.h"
If file.h is generated, then your build system needs to generate it before you compile anything that includes it. Generated sources are a major pain point in C++ build systems. It throws a wrench in the typical compilation flow... where you don't know which header files you need until you run the compiler, so you have to manually specify header dependencies in your Makefile if any are generated (instead of getting the dependencies from the compiler output). To keep things consistent (and more reliably cacheable), in Bazel, you have to specify all the dependencies in your build system, not just the ones that the compiler can't figure out for you. This solves some other problems with cached builds--like how you can have multiple files named "file.h", and the insertion of a new file earlier in the include search paths should trigger a rebuild, or else you end up with a poisoned build cache.
Obviously there are going to be some people stuck on build systems that can't handle this stuff gracefully. Too bad. You can keep using headers, it's not like those are going to stop working.
Does anyone know of a good example of C++ module usage out in the wild? Not Hello World examples, but I mean an actual project with a serious build process using it? I'd love to see how they're currently used.
So, apparently it can work. I don't don't how much Google's internal Module incarnation differs (if at all) from what was ultimately published in the standard, but I'd expect it to be similar.
The article discusses a problem of resolving dependencies between compiling different c++ files. I'm no expert, but I think this dependency graph is implicitly baked into the build system, with tool-assisted user-written bazel rules.
The module system from that video would have been great; there was/is a sound implementation that is based on Objective-C's module system, has been battle tested extensively and works without all kinds of silly quirks, makes packaging and distributing modules relatively easy, and provides performance benefits similar to precompiled headers which are known to improve build times substantially.
Of course, for those reasons it did not get standardized and instead what did get standardized was a mostly theoretical module system that has a great deal of ambiguity and implementation divergence across different compilers, places a great burden on build systems, none of which fully support modules yet, does nothing to help with packaging libraries, and only sometimes provides performance benefits, sometimes provides performance regressions.
Such is the way of C++, a language whose standardization process has been hijacked by politics and corporate stacking of the committee, legacy interests who insist on holding back key features that would improve the language for the vast majority of developers for the sake of ABI compatibility, and whose advancement has been plagued by numerous defect reports and broken or buggy features that get adopted without any kind of battle testing, user experience or broad community feedback.
Edit: I just notice now that you said "TU", my mind autocompleted your sentence with "file" instead the first time I read it. Still keeping my comment here, module :private is not well known and a nice way to declare a distinct interface Translation Unit while keeping everything in one file.
You can do this, but you can also use the "module :private" fragment if you want to have everything in one file while still keeping the distinction between interface and implementation.
Example:
// File "mymodule.ixx"
module;
#include <string>
// declare interface
export module mymodule;
export namespace mymodule {
// templates have to be declared
// in interface to be known at
// compile-time
template<typename T>
T add(T x, T y) {
return x + y;
}
std::string greetings();
}
// implement interface
module :private;
std::string mymodule::greetings() {
return "hello";
}
How do C++ programs / programmers enforce and manage modularity in the general sense if they don't even have simplest module system? I'm not even talking of having module functors like Ocaml, but something like Rust has. It seems to be one of the most obvious steps to reduce complexity and compose program of smaller parts.
Mostly we rely on the system package manager. If I'm trying to compile a C++ program and I need somelib I `sudo apt install libsomelib-dev`
Works about 80% of the time. The other 20% I gotta go digging through either github or sourceforge or someone's personal website that hasn't been updated in 15 years.
It's not "enforced;" it is managed through various means including namespaces, header includes, and prefixes. For example, SQLite is a library whose functions all start with 'sqlite3'. In practice it works well enough: name collisions are uncommon and can be managed.
Wait, does the C++ module proposal allow modules names to be different from their filenames? This sounds so strange, I've never used such a module system in other languages. I've always assumed C++ modules would somehow restrict the excessive liberty it has had from its headers and preprocessors. How did they arrive at such a weird idea?
C++ modules have little to do with the concept of "modules" you find in other languages. You can think about it as a more standardized way to define and create what is known as pre-compiled headers. But that's basically it.
C++ modules are not like pre-compiled headers. They give you freedom to specify what you are and are not exporting from a module. This is unlike header files (including pre-compiled headers), where you are constrained to re-export any dependency your header file has, as well as any code internal to your module which is necessary to declare your exports.
C++ headers are a total mess, and modules are the cleanup effort.
> Wait, does the C++ module proposal allow modules names to be different from their filenames?
It sort of makes sense. The calling program specifies `foo.bmi`, but compiling for ARM might look in "armv5/foo.bmi" while targeting x64 might look in "x64/foo.bmi", and that doesn't even take into account that there might be different debug and release versions.
That is very easily solved by not using modules for those things that cause issues, or by using a module facade over traditional C++.
For your example, you could make a 'foo' module that pulls in an implementation from a traditional C++ source file, "armv5/foo_impl.h" or "x64/foo_impl.h", using preprocessor #if/#ifdef to choose the right one. Or you could even use a module backend, pulling in the implementation from either "foo_armv5.bmi" or "foo_x64.bmi".
There is really nothing to see here. The "(2019)" should clue you in.
There are lots of ways to skin this cat. C++ has what they call a "large installed base" -- i.e., literally hundreds of billions of lines of code in active use. So, the solution for C++ is not what we might have done for a new language. It had to be reasonably easy to move an existing project to modules without breaking everybody that might depend on it, and without needing everything you depend on to be moved already. Everybody has their own upgrade schedule, many of which must be "never".
So, it looks odd compared to a green-field approach, but it works and is not a burden to maintain, which is what matters.
> Do C++ folks still not have the equivalent of Turbo Pascal Units (which were from 1987)?
Well, it's an easy problem to solve when a single compiler is used for everything. I doubt that Turbo Pascal Units could be used with other Pascal compilers.
With C++, your module compiled with Clang might be used in a program compiled with gcc. I don't think that C++ modules specify a binary interface in the spec.
With new languages it is easier to implement modules, because a new language, by definition, only has a single implementation. If the new language specifies a modules system it does not break existing software because, by definition, it is a new language.
> I can compile almost anything short of rebuilding my IDE in less than a second, every time, thanks to units in Lazarus/Free Pascal.
I know, I sometimes use Lazarus. I'm not really sure that a FreePascal Unit is compatible with other Passsl implementations (maybe Delphi, because the goal is to be a drop-in replacement for Delphi?).
My understanding of the C++ modules specification is that there is no requirement for modules from different compilers to be compatible.
>With C++, your module compiled with Clang might be used in a program compiled with gcc.
Why do you need more than one C++ compiler installed? The idea is to compile everything only once, and save all the time when you need to use the code later.
It makes no sense not to use the same compiler against all the source when building an executable.
> Why do you need more than one C++ compiler installed?
[WARNING: I don't really know that much about the C++ modules proposed by ISO. Please feel free to jump in and correct me.]
Okay, lets assume that everyone is using the same compiler.
The problem is that, when someone hands me a `foo.bmi` to use in my project, even if I am using the same compiler that they did, the `foo.bmi` might not be compatible because they used different compile-time flags (Maybe they've removed padding between structs, maybe they specified a 64bit `time_t` using a compile-time flag).
This problem makes it no different to handing someone a `foo.o` file and a `foo.h` file (header having templates in them, or similar).
Modules were supposed to solve the problems with headers, not replicate them.
> The idea is to compile everything only once, and save all the time when you need to use the code later.
I agree, but there are real problems with C++ other than using a different compiler.
Command-line/compile-time flags will produce `.o` files that are incompatible with each other. Templates are challenging to put into compiled libraries/modules. `#ifdef` directives have to be supported in any source for a module, making repeated compilations of the same source result in different output. `#include` directives might include `system/headers.h` that also cause different output with the same module name.
All-in-all, it may be a lost cause; ISO C++ modules are far too abstracted (as I understand it) to be practically useful. If the specification included some sort of readable header in the output file `foo.bmi` that contains practical but non-portable information like compiler name, compiler version, compiler flags, host name, target name, etc then C++ modules become easier in practice because a compiler can simply reject a module that it is not compatible with, with reasonable error messages ("User specified -DTIMET_SIZE=64 but foo.bmi is compiled with -DTIMET_SIZE=32").
It's hard to do C++ modules both usefully and portably[1]. When trading off between utility and portability, ISO is almost always going to weight portability more than utility.
[1] I once got a support ticket from a user who was porting my FLOSS library to zOS, and when trying to solve the compilation problems I learned that C++ (the language) is insanely portable. The build systems are not.
All the problems that we had were build-time problems (#ifdef for different archs, compile-time flags that needed to be used to make the library compatible, etc).
In any environment where compilation and linking is separate (this covers both static and dynamic linking, it's not just a shared library problem), you have many situations where a final application is assembled from a set of binaries that were compiled on different machines.
This means that realistically, even if every machine only has one C++ compiler installed, binaries compiled with different compilers will need to be able to interface with one another. Changing a language's ABI is a really, really big deal and not having a defined ABI at all is even worse.
Yes, modules finally, although not without issues, as the stardard is freshly baked and there are 40 year of history to drag.
On the context of Turbo Pascal units (actually they are UCSD Pascal units), C++ Builder did support the concept of C++ packages with a similar purpose.
Or if one wants to delve into the COM dungeons on Windows, that is also a path although naturally the COM ABI only covers the basics of an OOP ABI.
Visual C++ support for modules is already quite good, although still has some warts.
I was just tonight watching Bjarne’s cppcon 21 talk[1] and he does not seem to think they’re dead.
OTOH, from what I can tell my compiler of choice, clang, has not improved its support since last time I dabbled with it unsuccessfully. But I could be Doing It Wrong. There seems to be very little stuff on the internet about actually using them which is not a great sign.
I can't help but notice that of all the names in the linked docs, there doesn't seem to be any from the companies that implement the actual C++ compilers. What's the stance of Clang/gcc/MVSC/... developers on this?
Eh, the assumption here is that you don't know what the dependencies of a C++ file are before you start compiling. That's not a great assumption.
That assumption allows you to write really lazy Makefiles. You just throw a bunch of C++ filenames in your Makefile, add the correct dep flags, and add an "include" line, and your Makefile will correctly rebuild the object files when header files change, without needlessly rebuilds. Here's how such a lazy Makefile works:
Sure, that's nice, but there are a lot of reasons why you might want a more sophisticated build system than a lazy Makefile. You might have a project with long build times, so you might want a good build cache. You might have generated header files. As your project gets larger and more complicated, the idea of building everything with a Makefile really loses its appeal, and you look for better build systems.
The new generation of build systems is Bazel, Buck, Pants, Please, Meson, etc. Plenty of choices to go around. These systems require that you specify the dependencies up-front, rather than relying on -MF & friends. This means that if you add a generated header somewhere, nothing breaks. If you add a new file earlier in the include search path with the same name as a file later in the path, nothing breaks. (By "nothing breaks" I just mean that your incremental builds and clean builds will still both generate the correct output.) The -MF option is then used to trim the dependency set (in a particular way to keep the build correct... rather than record the header dependencies, you record which headers were ignored during compilation).
With these build systems, modules are going to work just fine.
I think this post has a clearly missing solution that is (essentially) a first stage in a build system.
Yes, C++ can be slow to build. But it's slow to build because of the includes and the code generation (and we still have to do code generation).
How much time does the author think it takes to pipe code through the preprocessor (for defines, not includes since in this case there are no includes), and tokenize?
I just checked the dependency files of a fairly large project (GNU Radio), and each cc file transitively depends on 414 files on average. In total in order to compile its 1339 cc files the compiler needs to open and read over half a million files.
So just in terms of data to read on this first pass it's faster by a factor over 400x. Add to that that you don't have to do anything in this pass except look for module dependencies, I bet it's several thousand times faster than a compile.
So in other words, if it takes you an hour to build your big project, are you saying you don't have one single second left over to calculate the dependency tree as a first pass?
And you only have to do it once (and then re-do files as source timestamps change).
If the compiler and the build system is an integrated product, then right, this shouldn't be too hard. But that's not usually the case.
If they're not, what's the interface for the initial build step? Is it still something like "gcc -MD"?
But "gcc -MD" can't just spit out a list of files. It will have to output not only file names but also module names (both referenced and declared), because resolving module names to file names requires global knowledge.
That means every build tool that wants to work with C++ must be changed to take in this extended information and resolve module dependencies. Because otherwise, if there's even a single module in your 1339 file project, then current tooling breaks.
Which will put pressure on developers to not add that first module, slowing adoption.
Make doesn't understand gcc -MD output. Make has stayed basically the same for half a century. There's nothing language-specific in make. For example, when C++ was invented, nothing needed to be added to make, because the general mechanism could easily support adding a cfront step that compiled .cxx files into .c.
It looks like that's changing now. Yesterday's make will not be able to build tomorrow's C++.
I'm not going to try to design the best way to do the equivalent for C++ modules here in a HN comment, but it doesn't seem like a particularly hard problem.
Of course it could be exactly because I've not seriously written a compiler or build system that I mistake the problem for not being hard.
But it shouldn't be a problem for explicit dependency build systems like Bazel. And people have written automatic Bazel dependency tooling, so it really looks to me like "what's the best way to do this" rather than "we can't do this", which the article implies.
Yes, gcc produces dependency information while compiling, and is adapted for Make. But in theory this could be done by `./configure` or something in a dedicated script.
It'd not even be hard. And it's super quick:
cpp -I. src/foo.cc | grep '^# ' | sed 's/^[^"]"//;s/"[^"]$//'
| sort | uniq | grep -v \<
It's not without precedence to have a pre-stage. Just look at the Ninja build system.
So how would you change that incantation to accommodate C++ with modules?
Edited to add:
Okay, you did say "I'm not going to try to design the best way to do the equivalent for C++ modules here in a HN comment", but then all you've shown is that it's easy to implement for C++ without modules.
What I meant with that oneliner was to show that you don't even need compiler integration. A much simpler tool (in this case the preprocessor) could extract just the needed parts without parsing the rest.
Sure, it's harder to tokenize C++, but once you have the yacc for it it's pretty simple to get a list of provided and needed modules, right?
We have pretty advanced tools nowadays even for rewriting source code. The Go standard library has a package for parsing a Go file. You can use that and easily extract the import list and package name definition. I don't see (perhaps due to my ignorance of C++ module details) why this doesn't translate. Yes, Go's compiler is integrated with its build system. But I'm saying that's not a prerequisite.
I'm saying extracting the necessary information from a C++ file is quick and easy. What I'm not trying to do is design build system and compiler passes.
Am I missing something? Is it inherently not possible to create a tool that will take C++ source code and output its imports and exports, or that it's even hard?
(again, aside from writing the YACC correctly, once. But the complexity of C++ is mostly not its tokenization)
D's module system benefited greatly by getting rid of the preprocessor. I implemented precompiled headers for C++, and so knew that a robust and successful module system wasn't compatible with preprocessing.
How does this design handle cyclic dependencies? Or it doesn't, which would be even worse than headers in which you can at least forward declare some things?
58 comments
[ 5.4 ms ] story [ 126 ms ] threadCMake support is an open issue: https://gitlab.kitware.com/cmake/cmake/-/issues/18355
Did anyone ever respond to this? What do the anti-SG15 (and/or pro-module) people say? I want to see the other side of this argument.
It's not like the same problem doesn't already crop up with generated sources. Like, if you see this:
If file.h is generated, then your build system needs to generate it before you compile anything that includes it. Generated sources are a major pain point in C++ build systems. It throws a wrench in the typical compilation flow... where you don't know which header files you need until you run the compiler, so you have to manually specify header dependencies in your Makefile if any are generated (instead of getting the dependencies from the compiler output). To keep things consistent (and more reliably cacheable), in Bazel, you have to specify all the dependencies in your build system, not just the ones that the compiler can't figure out for you. This solves some other problems with cached builds--like how you can have multiple files named "file.h", and the insertion of a new file earlier in the include search paths should trigger a rebuild, or else you end up with a poisoned build cache.Obviously there are going to be some people stuck on build systems that can't handle this stuff gracefully. Too bad. You can keep using headers, it's not like those are going to stop working.
https://github.com/pjmlp/ppm2png
https://github.com/pjmlp/AStarDemo
I am also in the process of porting Raytracing Weekend to C++ modules.
C++ Modules Might Be Dead-On-Arrival - https://news.ycombinator.com/item?id=19015079 - Jan 2019 (193 comments)
So, apparently it can work. I don't don't how much Google's internal Module incarnation differs (if at all) from what was ultimately published in the standard, but I'd expect it to be similar.
The article discusses a problem of resolving dependencies between compiling different c++ files. I'm no expert, but I think this dependency graph is implicitly baked into the build system, with tool-assisted user-written bazel rules.
Of course, for those reasons it did not get standardized and instead what did get standardized was a mostly theoretical module system that has a great deal of ambiguity and implementation divergence across different compilers, places a great burden on build systems, none of which fully support modules yet, does nothing to help with packaging libraries, and only sometimes provides performance benefits, sometimes provides performance regressions.
Such is the way of C++, a language whose standardization process has been hijacked by politics and corporate stacking of the committee, legacy interests who insist on holding back key features that would improve the language for the vast majority of developers for the sake of ABI compatibility, and whose advancement has been plagued by numerous defect reports and broken or buggy features that get adopted without any kind of battle testing, user experience or broad community feedback.
You can do this, but you can also use the "module :private" fragment if you want to have everything in one file while still keeping the distinction between interface and implementation.
Example:
Works about 80% of the time. The other 20% I gotta go digging through either github or sourceforge or someone's personal website that hasn't been updated in 15 years.
C++ headers are a total mess, and modules are the cleanup effort.
It sort of makes sense. The calling program specifies `foo.bmi`, but compiling for ARM might look in "armv5/foo.bmi" while targeting x64 might look in "x64/foo.bmi", and that doesn't even take into account that there might be different debug and release versions.
For your example, you could make a 'foo' module that pulls in an implementation from a traditional C++ source file, "armv5/foo_impl.h" or "x64/foo_impl.h", using preprocessor #if/#ifdef to choose the right one. Or you could even use a module backend, pulling in the implementation from either "foo_armv5.bmi" or "foo_x64.bmi".
There are lots of ways to skin this cat. C++ has what they call a "large installed base" -- i.e., literally hundreds of billions of lines of code in active use. So, the solution for C++ is not what we might have done for a new language. It had to be reasonably easy to move an existing project to modules without breaking everybody that might depend on it, and without needing everything you depend on to be moved already. Everybody has their own upgrade schedule, many of which must be "never".
So, it looks odd compared to a green-field approach, but it works and is not a burden to maintain, which is what matters.
I can compile almost anything short of rebuilding my IDE in less than a second, every time, thanks to units in Lazarus/Free Pascal.
Well, it's an easy problem to solve when a single compiler is used for everything. I doubt that Turbo Pascal Units could be used with other Pascal compilers.
With C++, your module compiled with Clang might be used in a program compiled with gcc. I don't think that C++ modules specify a binary interface in the spec.
With new languages it is easier to implement modules, because a new language, by definition, only has a single implementation. If the new language specifies a modules system it does not break existing software because, by definition, it is a new language.
> I can compile almost anything short of rebuilding my IDE in less than a second, every time, thanks to units in Lazarus/Free Pascal.
I know, I sometimes use Lazarus. I'm not really sure that a FreePascal Unit is compatible with other Passsl implementations (maybe Delphi, because the goal is to be a drop-in replacement for Delphi?).
My understanding of the C++ modules specification is that there is no requirement for modules from different compilers to be compatible.
Why do you need more than one C++ compiler installed? The idea is to compile everything only once, and save all the time when you need to use the code later.
It makes no sense not to use the same compiler against all the source when building an executable.
[WARNING: I don't really know that much about the C++ modules proposed by ISO. Please feel free to jump in and correct me.]
Okay, lets assume that everyone is using the same compiler.
The problem is that, when someone hands me a `foo.bmi` to use in my project, even if I am using the same compiler that they did, the `foo.bmi` might not be compatible because they used different compile-time flags (Maybe they've removed padding between structs, maybe they specified a 64bit `time_t` using a compile-time flag).
This problem makes it no different to handing someone a `foo.o` file and a `foo.h` file (header having templates in them, or similar).
Modules were supposed to solve the problems with headers, not replicate them.
> The idea is to compile everything only once, and save all the time when you need to use the code later.
I agree, but there are real problems with C++ other than using a different compiler.
Command-line/compile-time flags will produce `.o` files that are incompatible with each other. Templates are challenging to put into compiled libraries/modules. `#ifdef` directives have to be supported in any source for a module, making repeated compilations of the same source result in different output. `#include` directives might include `system/headers.h` that also cause different output with the same module name.
All-in-all, it may be a lost cause; ISO C++ modules are far too abstracted (as I understand it) to be practically useful. If the specification included some sort of readable header in the output file `foo.bmi` that contains practical but non-portable information like compiler name, compiler version, compiler flags, host name, target name, etc then C++ modules become easier in practice because a compiler can simply reject a module that it is not compatible with, with reasonable error messages ("User specified -DTIMET_SIZE=64 but foo.bmi is compiled with -DTIMET_SIZE=32").
It's hard to do C++ modules both usefully and portably[1]. When trading off between utility and portability, ISO is almost always going to weight portability more than utility.
[1] I once got a support ticket from a user who was porting my FLOSS library to zOS, and when trying to solve the compilation problems I learned that C++ (the language) is insanely portable. The build systems are not.
All the problems that we had were build-time problems (#ifdef for different archs, compile-time flags that needed to be used to make the library compatible, etc).
This means that realistically, even if every machine only has one C++ compiler installed, binaries compiled with different compilers will need to be able to interface with one another. Changing a language's ABI is a really, really big deal and not having a defined ABI at all is even worse.
Sometimes you need to upgrade your compiler but can't/don't want to rebuild the whole dependency graph.
Because sometimes you need to import prebuilt libraries/modules provided by a third party, and you don't control how those were built.
Keep in mind that C++ does not define a standard ABI.
On the context of Turbo Pascal units (actually they are UCSD Pascal units), C++ Builder did support the concept of C++ packages with a similar purpose.
Or if one wants to delve into the COM dungeons on Windows, that is also a path although naturally the COM ABI only covers the basics of an OOP ABI.
Visual C++ support for modules is already quite good, although still has some warts.
OTOH, from what I can tell my compiler of choice, clang, has not improved its support since last time I dabbled with it unsuccessfully. But I could be Doing It Wrong. There seems to be very little stuff on the internet about actually using them which is not a great sign.
[1] https://youtu.be/15QF2q66NhU
That assumption allows you to write really lazy Makefiles. You just throw a bunch of C++ filenames in your Makefile, add the correct dep flags, and add an "include" line, and your Makefile will correctly rebuild the object files when header files change, without needlessly rebuilds. Here's how such a lazy Makefile works:
Sure, that's nice, but there are a lot of reasons why you might want a more sophisticated build system than a lazy Makefile. You might have a project with long build times, so you might want a good build cache. You might have generated header files. As your project gets larger and more complicated, the idea of building everything with a Makefile really loses its appeal, and you look for better build systems.The new generation of build systems is Bazel, Buck, Pants, Please, Meson, etc. Plenty of choices to go around. These systems require that you specify the dependencies up-front, rather than relying on -MF & friends. This means that if you add a generated header somewhere, nothing breaks. If you add a new file earlier in the include search path with the same name as a file later in the path, nothing breaks. (By "nothing breaks" I just mean that your incremental builds and clean builds will still both generate the correct output.) The -MF option is then used to trim the dependency set (in a particular way to keep the build correct... rather than record the header dependencies, you record which headers were ignored during compilation).
With these build systems, modules are going to work just fine.
Yes, C++ can be slow to build. But it's slow to build because of the includes and the code generation (and we still have to do code generation).
How much time does the author think it takes to pipe code through the preprocessor (for defines, not includes since in this case there are no includes), and tokenize?
I just checked the dependency files of a fairly large project (GNU Radio), and each cc file transitively depends on 414 files on average. In total in order to compile its 1339 cc files the compiler needs to open and read over half a million files.
So just in terms of data to read on this first pass it's faster by a factor over 400x. Add to that that you don't have to do anything in this pass except look for module dependencies, I bet it's several thousand times faster than a compile.
So in other words, if it takes you an hour to build your big project, are you saying you don't have one single second left over to calculate the dependency tree as a first pass?
And you only have to do it once (and then re-do files as source timestamps change).
Modules could be precompiled, but I found that it was faster (and easier) to just recompile them.
(And not having a preprocessor eliminates several passes.)
If they're not, what's the interface for the initial build step? Is it still something like "gcc -MD"? But "gcc -MD" can't just spit out a list of files. It will have to output not only file names but also module names (both referenced and declared), because resolving module names to file names requires global knowledge.
That means every build tool that wants to work with C++ must be changed to take in this extended information and resolve module dependencies. Because otherwise, if there's even a single module in your 1339 file project, then current tooling breaks.
Which will put pressure on developers to not add that first module, slowing adoption.
Or is there a step to this I'm missing? Can't they be analyzed individually and then joined?
Conceptually this merging isn't much different from the fact that gcc -MD needs to output data formatted in a way that Make understands.
But fair enough. I feel like I'm for the second time am saying "just add another stage to the build". Still. "Dead on arrival" is a bit much.
It looks like that's changing now. Yesterday's make will not be able to build tomorrow's C++.
How do you mean? It's literally Makefile syntax that's being output, and it's directly included by the Makefile as-is.
What I meant is that make doesn't have special code to handle "gcc -MD" output. gcc is adapted to the needs of make, not the other way around.
Of course it could be exactly because I've not seriously written a compiler or build system that I mistake the problem for not being hard.
But it shouldn't be a problem for explicit dependency build systems like Bazel. And people have written automatic Bazel dependency tooling, so it really looks to me like "what's the best way to do this" rather than "we can't do this", which the article implies.
Yes, gcc produces dependency information while compiling, and is adapted for Make. But in theory this could be done by `./configure` or something in a dedicated script.
It'd not even be hard. And it's super quick: cpp -I. src/foo.cc | grep '^# ' | sed 's/^[^"]"//;s/"[^"]$//' | sort | uniq | grep -v \<
It's not without precedence to have a pre-stage. Just look at the Ninja build system.
Edited to add:
Okay, you did say "I'm not going to try to design the best way to do the equivalent for C++ modules here in a HN comment", but then all you've shown is that it's easy to implement for C++ without modules.
Sure, it's harder to tokenize C++, but once you have the yacc for it it's pretty simple to get a list of provided and needed modules, right?
We have pretty advanced tools nowadays even for rewriting source code. The Go standard library has a package for parsing a Go file. You can use that and easily extract the import list and package name definition. I don't see (perhaps due to my ignorance of C++ module details) why this doesn't translate. Yes, Go's compiler is integrated with its build system. But I'm saying that's not a prerequisite.
I'm saying extracting the necessary information from a C++ file is quick and easy. What I'm not trying to do is design build system and compiler passes.
Am I missing something? Is it inherently not possible to create a tool that will take C++ source code and output its imports and exports, or that it's even hard?
(again, aside from writing the YACC correctly, once. But the complexity of C++ is mostly not its tokenization)
- no circular dependency allowed
- you have to fw declare everything, yet again
i gave up on C++ personally, for a little while i went back to C only, now i only use D, and some GO for my backend needs