The ratio of times autotools checked my system for a Fortran compared to the number of times I compiled something with a Fortran dependancy must be at least 200:1.
This is one of my other peeves with the whole GNU build system; they will have serious bugs, they will fix those serious bugs promptly, but then take years to publish a release with the bug fix. My favorite example is when a new version of GNU libc broke GNU m4, then they quickly updated GNU m4 to support the new GNU m4, but they didn't actually release their fix for years. So for something like 2 or 3 years, you literally had a situation where the latest GNU libc couldn't be used to compile the latest GNU m4.
Cmake is just another hair ball, with its arcane syntax and incomprehensible errors. Plus, as mentioned it’s way less friendly to final users who have to compile the code from a tarball without knowing the intricacies of the build system. There is pretty much no advantage in going from autotools to cmake. If there is a jump to make, it’s to something better like meson.
Yes, the project indicator is nice, good point. Personally I haven’t really noticed any big difference in building time, parallel make seems good enough for my uses.
What we really need is a tool to convert from an arcane and underpowered silly build tool like cmake to proper autotools, like autoupdate. And then farm cmake to retirement. Or keep it to the poor windows folks.
A maintain dozens of mixed autotools, cmake and some header-only/makefile-only projects. The cmake build recipes are constantly behind and fragile.
The makefile-only projects are a dirty hack, and very limited.
autotools is easily the best, with support for everything. And if it's not in autoconf, then it's in autoconf-archive.
meson and ninja are cool, but overkill mostly. We had dozens of such build tools before, and they all died.
I even have to teach GNU maintainers how to use autotools/automake properly, as they cannot deal with recursive subdirs, rather have their own configure.in in each subdir. Of course this doesn't scale well.
"to proper autotools" ah yes, the usual "you're holding it wrong" defense
No, another hack on top of the auto-* ecosystem won't make things better. CMake's problems are cmakes's problems, but it doesn't make autostuff better in any way
But please tell me again why do I need a buildsystem that needs to check if I have Fortran or if I'm running AIX under MIPS again?
Makefile only is crude but it can work fine if your target is the usual distros and your project is small (read: less trouble than autotools)
Why would you need to check for it? The name of compiler is well known, the build will properly fail if it’s not there, and work otherwise. Autotools doesn’t serve any purpose here.
I dearly regret that I adopted autotools for some of my projects. As TFA identifies, the primary problem autotools solved is no longer a problem: Linux, BSD, Darwin, and Cygwin are the targets today and they're more-or-less portable and compliant. Meanwhile, autotools is a constant tax on the development cycle.
1. autoconf is awful to learn.
2. autoconf can be useful in determining the platform, dealing with a few quirks, and ensuring that system dependencies/libraries are satisfied.
3. autoconf's usefulness here is completely undermined by automake, as changing Makefile.am requires rerunning autoreconf, which then requires rerunning configure. So, developers must constantly rerun the configure script, even though nothing about their system environment has materially changed. For example, this happens when a new source file is created and added to Makefile.am.
4. Perhaps the most egregious missing weakness from TFA is acknowledgement that configure scripts are horrifically slow. It's 2022 and developers have their time wasted by configure checking whether printf() exists.
5. automake documentation discourages developers from globbing source files in source directories, and insists they list them individually. It's silly and just results in needless running of configure.
6. The complex interdependencies between generated files is nigh-impossible to understand and reason about. It's cruft.
7. libtool is completely braindead, the absolute worst of it all. It serves absolutely no purpose on contemporary systems, but the docs continue to admonish developers that they must use it to stand a chance of creating a shared library (no, just learn to specify -fPIC). Instead of analyzing a project once, as part of autoconf, libtool determines what to do on the fly for every source file--and all in bash. It doubles a project's compilation time.
8. Creating alternatives to libtool with new autoconf macros, deprecating libtool, and cleansing all traces of it from the planet should absolutely be the highest priority of the autotools project.
I find autoconf to be good at its job - building a standalone script which runs micro tests - and still use it, but that automake and libtool add very little value for the reasons you described.
I do agree that m4 is rather unpleasant to write, and that the resulting tests are too slow (often because user defined macros do a poor job of cacheing) but I also think that these things could be addressed (newer macro languages, bundled or threaded micro-tests) while still preserving the good bits of autoconf.
I once munged a configure.ac so that it would blast out its variables to a text file, i.e.,
CFLAGS="-g -O2"
LDFLAG="-l openssl"
I then replaced automake and libtool with tup, and the Tupfile would just source these variables from configure. It worked great! It's a pity that tup hasn't caught on.
The lack of parallelization is also a pain point. When I needed something to test platform specific stuff and set defines/etc, I found that writing something to generate ninja & integrate the define generation (which ended up in some generated headers) into the total build process. This was very snappy.
These days I use meson mostly, but I would love to see a complete build system really get into parallelizing platform checking and merge it into the normal build process.
Hasn't WSL replaced Cygwin these days? I'm surprised that it and even MinGW are still used. I even recently fixed a bug filed about a build failure on MinGW due it not having System V shared memory functions.
WSL is a Linux virtual machine. I'm developing low-latency audio & video software, I'm not going to have my users run it in a VM and require them to install WSL ... So MinGW it is (which is just a port of the GNU tools to windows and a free reimplementation of the windows API headers - the called code is still the MS's "universal runtime" shared libraries so it really does not change anything in my case except more sanity and final binaries faster than than the MSVC ones)
I use WSL with mingw-w64 to cross compile things for native Windows sometimes... e.g. I keep my own builds of ffmpeg and mpv around, as well as some of my own "legacy" tools that I use both on linux and windows natively.
And in this space, it's often a lot easier to get something autotools-based to build than something cmake or meson, at least in my experience. With autotools more often than not all I need to do is to specify the right target hosttriple and be done. With cmake and meson, it's often a fight to figure out how to cross-compile something like that.
This approach of cross-compiling for windows with mingw-w64, sometimes on "proper" linux, sometimes on WSL2, sometimes even on cygwin, seems to be still somewhat common in C/C++ land and in open source in particular, especially for projects that pull in a fair amount of dependencies (such as the aforementioned ffmpeg) - you will find a lot of Windows build guides that say only mingw cross-compilation from linux is supported and tested.
Maintaining MinGW builds of your deps must be annoying, would be nice if some distros had a MinGW partial port/architecture people could use for cross-building other things.
There are some projects that try to be partial dependency "package" managers for mingw, but for whatever reason I never really used them, so I cannot comment on how well they work and how up-to-date they are kept.
I guess, I just like fighting with the interwoven mess of build-systems and dependencies sometimes (and really, I don't spend more than maybe 1 or 2 hours a month on that)... a bit like people doing linux-from-scratch, except probably less time consuming than building an entire kernel and userland step-by-step.
>5. automake documentation discourages developers from globbing source files in source directories, and insists they list them individually. It's silly and just results in needless running of configure.
No it's not silly, realistically you can't actually do that anywhere without re-running configure. Any build system based on directory trees will have this limitation. See meson's docs for more on this: https://mesonbuild.com/FAQ.html#why-cant-i-specify-target-fi...
I also came here to comment that libtool is totally useless on modern systems and I hope it gets removed or made optional, but you beat me to it :)
The other problem is that during a Git conflict resolution, other files matching that glob get dropped into your source tree. Globbing these up is…never going to go well.
> 4. Perhaps the most egregious missing weakness from TFA is acknowledgement that configure scripts are horrifically slow. It's 2022 and developers have their time wasted by configure checking whether printf() exists.
I'd say on some platforms "horrifically slow" is an understatement. At my day job we were building software in Cygwin on Windows machines infected with Symantec's virus scanner which adds hundreds of milliseconds to every process launch. The configure script for Protocol Buffers took something like half an hour to run.
We eventually switched to CMake but not without a lot of pain first. For what it's worth, CMake fixes the speed issue but not much else. It still has its own totally esoteric custom language, it still discourages globbing source files and requires re-running upon adding any file or changing anything, and it has even more insane default settings than autotools (for example file(DOWNLOAD) from an https:// URL by default ignores certificates!)
There is something wrong with all of these buildsystems. I don't know what the solution is but I feel we are very far from it.
> it still discourages globbing source files and requires re-running upon adding any file or changing anything
Globbing is discouraged exactly for the reason that then cmake can't discover when files should be added or removed from the build process, or build options have changed. But if globbing isn't used there's no reason why manually re-running cmake would be required (there have been some problems in the past with Visual Studio not automatically reloading a modified project, but I haven't seen those problems in a long time).
I'm all for bashing autotools, but in this one case, I'd throw Symantec (SEP?) into the fire first.
And the solution to all build systems being bad is imho not to use C/C++. It's model for platform independence is fundamentally broken. It's not like snafu doesn't exist elsewhere too (Python), but other systems are clearly less troublesome (Go, NPM, Cargo).
> It still has its own totally esoteric custom language, it still discourages globbing source files and requires re-running upon adding any file
We use cmake, with Windows as our main (but not the only) target platform. We found that not globbing is slow and impractical for us.
1. We generate VS project files, which then guides the build process. When you add source files using the IDE, the generated project(s) will be updated, no regenerate is triggered, the build system will do the correct thing: only the added file, not everything is (re-)built.
2. If we didn't glob, then given the nice local workflow above, people would often forget to update the cmake files and get CI failures.
3. When injecting cmake regeneration into the build process, then for some strange reason we found a bad serialization of build steps, which slows everything down.
4. The slow part on Windows seems to be that cmake has to use some system call to figure out if source file name casing is correct, which is very slow (for us the slowest part of the generation process by far). I don't recall the details here, but I think globbing somewhat reduces this issue.
All in all, I can say: figure out what works best for you and don't trust experts that discourage globbing altogether.
The part I always struggled with regarding 4. is that if printf() didn't exist, it's not like it bundles its own compat library. It just errors out, with a very similar error to what a user would get if they simply tried to compile the code that called a library you didn't have. This "pre compile test" didn't actually add anything to the picture.
Would you rather your build failed during configure or 10 minutes into the build, where parallelism will make the compile still print tons of things after the error message?
I would rather the printf check be handled by the compiler than in the configure check because the compiler will of course know that printf is present and not waste your time.
Honestly? If the question is if I would A) make everyone everywhere spend minutes (or potentially hours on Windows) all the time running the configure script any time anything changes, or B) not spend all that extra time running configure but maybe get a compile error a few minutes later if I'm compiling on extremely weird systems (such as ones without printf), I think I'd take B.
Especially since configure doesn't actually make anything any faster even in most cases where it _does_ detect errors. It aborts at the first error. So when I'm missing packages for example, I will wait half a minute while `./configure` runs, then it errors with a single missing package, then I will install that package and re-run `./configure`, wait another half a minute until it prints another single missing package, etc. Repeat until I have all packages.
I hate autotools so much. Configure scripts should either run in parallel, or cache its results, or both (ideally both), and they should report as many errors as it can before aborting.
> Linux, BSD, Darwin, and Cygwin are the targets today
They're the most popular targets today. They certainly weren't yesterday, and very probably won't be tomorrow. I've seen a lot of changes over the decades and the attitude that "nobody will ever need more than 640 k" has always been popular. I also say this from the point of view of someone who makes a living with porting software to a reasonably popular platform "today" that is not one of those listed. Also someone who has spent many years doing "maintenance programming" as opposed new development on the leading edge. "Maintenance" accounts for something like 80% of revenue in the software industry, and the old machines in maintenance mode are not always identical to the new sleek affairs on developer's desks.
> 1. autoconf is awful to learn.
Yes. Like vim, it has a learning curve. It's not actually that bad, and the internet is chock full of documentation and examples. It's about on a par with alternatives like CMake and meson in terms of learning difficulty.
> 3. autoconf's usefulness here is completely undermined by automake, as changing Makefile.am requires rerunning autoreconf, which then requires rerunning configure. So, developers must constantly rerun the configure script, even though nothing about their system environment has materially changed. For example, this happens when a new source file is created and added to Makefile.am.
Ah, you got stuck at point 1. You don't have to re-run autoconf after re-running automate (autoreconf knows this), so you don't have to re-run configure. The makefiles generated by automake know this. That's why when you just type "make" after modifying a Makefile.am it will do the right thing automatically, and that is to re-run the config.status script which contains all the autodetected configuration already cached. It's seriously faster than running configure although the printed output is identical (because configure itself generates and runs that script).
> 4. Perhaps the most egregious missing weakness from TFA is acknowledgement that configure scripts are horrifically slow. It's 2022 and developers have their time wasted by configure checking whether printf() exists.
Yes. You pay the price for portability. You have argued "but I don't port my software why should I pay the price? That only helps other people and there's nothing in it for me". OK, fair enough.
> 5. automake documentation discourages developers from globbing source files in source directories, and insists they list them individually.
The documentation also says why. It's a well-reasoned argument. Also, despite it being a pain in the butt it turns out that explicitly showing your work is a benefit in the long run.
> 6. The complex interdependencies between generated files is nigh-impossible to understand and reason about.
They're a DAG; they should be straightforward for someone with a background in programming to grasp. The relationships are there a priori but the autotools document them and make them explicit.
> 7. libtool is completely braindead
OK. Agree with you there. Libtool was a godsend in its time. It's improved massively (yes, I see the complaints about frustrations with it over a decade ago) but it's largely superfluous today.
> Yes. You pay the price for portability. You have argued "but I don't port my software why should I pay the price? That only helps other people and there's nothing in it for me". OK, fair enough.
Portability to what, though?
It's 2022, not 1982. Most of the commercial UNIXes that autotools was written to handle are long dead now. When was the last time anyone saw a (non-embedded) system with CHAR_BITS ≠ 8, for example?
As an end user, I like how simple autotools is to build a project:
./configure && make && make install
And I like that I can very easily see all the various build options:
./configure --help
I don't like that configure takes a long time and checks things that don't seem like they really need to be checked. Nor do I like how difficult autotools is to use as a developer (I have some, but not much experience working on a project using autotools, over a decade ago).
I've yet to see a competing build system that makes the end user usage and option discovery as simple.
Edit: I've not necessarily used and explored all the various build systems, but the few I have used have generally not been as simple, or at least weren't to me obviously as simple.
I build a lot of cmake and meson projects. I was an expert at cmake for quite some time (bleh), but now I use meson for everything I can.
The end user experience is in every aspect superior to autotools. There is a small downside for cmake compared to autotools: while you have to install the bits for autotools it runs on a shell. There is a bigger downside for meson: you need a minimal python infrastructure up and working. Over time you need that python infrastructure to keep working. Sometimes it doesn't.
I still vastly prefer meson over cmake and much more so autotools. If I have to fix cmake or meson builds, well, really, if you know their languages, it's just no comparison to debugging the 1000s of lines of shell code autotools generates.
I'm not very familiar with meson. I can use cmake to build a project but I don't know how to find various build options without searching through that project's documentation. I probably haven't explored cmake enough.
I would argue this confirms what the OP said cmake is really not very friendly to the user (not developer). Reading through the cmake sources for options is equivalent to "read the source".
But you only have to do this if you don't want to run an actual GUI or ncurses UI which does this for you for some reason ? What would you suggest instead ?
(and if your meta-build system does not allow to do that, I'm sorry but I'll have to build a meta-meta-build-system on top of it which implements it and no one will like it)
In the build directory, "ccmake ." Note the "." I need to do this in almost every cmake project I don't know well. Also toggle on the advanced options listing. Occasionally you'll see stuff in the CMakeLists.txt files that don't show up in that menu. This is usually variables you can set on the command line to influence configuration. People are nowadays calming down their desire to program up their own flavor of a gigantic build system, and sticking to simpler idioms. That's a big improvement. However there's a lot of legacy nightmare cmake out there. Meson is much harder to build different dialects. Some people don't like this. After 35 years of build systems, I say yes please!
I forget what I do in meson. Which means I just read the meson.build and meson_options.txt. They are quite straightforward, and very concise, if you're not Doing It Wrong.
And yet, meson just routinely doesn't work. Sure: if you are just trying to do some trivial local compile job it seems fine, but it keeps having issues with even trivial cross compilation cases (which are used even on Linux as you generally want to do a Linux-to-Linux cross compile to control your glibc target) and linker detection (as it is assuming way too much direct control over the linker that it should frankly be delegating to the compiler) that make you have to jump through ridiculous hurdles to make things compile (and that's assuming you aren't on one of the commonly-deployed semi-recent versions of meson where it is a truly lost cause). In contrast, autotools always works: I have never had a blocking or serious issue compiling something that uses autotools, no matter how complex the scenario.
Here are a couple issues I have run into (one being fixed, though too late for some key versions of Ubuntu; and the other just languishing, as I guess "working" is asking for too much).
Could be; I'm not an evangelist for meson and if it doesn't work for cross compilation I'd use something else, likely cmake. I don't cross compile very often, so for me it works great. I have run into sadness with meson subprojects, but I'm unconvinced they are a big win.
The problem is that I don't get to make the decision to use meson or cmake: you make that decision when you develop your project around meson; which ties back to the point of this thread: as a user attempting to compile something (which FWIW should essentially always be a cross compile... a native compile is essentially never what you want as it bleeds way too much random machine state into your build), autotools is still king (and certainly much better than the haphazard piles of programmable cmake that people tend to throw together when they use cmake; at least with cmake,npeople tend to develop stuff where I just give up on their compilation scripts and compile all their code with my own replacement scripts... sadly, I haven't managed to do that reasonably for glib yet so I could avoid their meson hell).
For us(https://buildroot.org/) cross compiling with meson tends to be significantly more reliable than autotools already as it tends to be much better at separating host/target artifact builds within a project and setting up the toolchain correctly, autotools tends to need lots of package specific hacks in my experience.
We have a fairly customized integration(https://github.com/buildroot/buildroot/blob/master/package/p...) which we've iterated on for a while that works quite well these days. The cross compilation support with upstream meson has improved a lot over the years and while it's quite mature now IMO it still takes a bit of env setup to work correctly(but once it works it tends to not break much).
For the second, https://github.com/mesonbuild/meson/issues/6662#issuecomment... is the right solution and I will try to have it documented. Unfortunately, clang disagrees that it's the right solution and meson might have to hack something around that disagreement.
Hum... Configure routinely doesn't work for me either, on even more trivial scenarios than cross-compilation.
When I care to look how to fix it, it's almost always some error on the autotools configuration and I can edit the Makefile to run things correctly. Also, when it breaks, there are normally more broken things on the project. But autotools are always among the first things to break.
Not really - from my experience autotools can’t really be expected to work properly on any system it wasn’t tested on. And it’s way easier to hack the generated script - usually just commenting out the problematic part, as almost none of those scripts generated by autoconf serve any purpose.
The equivalent for a cmake project is roughly (from within a subdirectoy to not pollute the project directory with build files):
cmake .. && cmake --build . && cmake --install .
...and discovering/changing the build options can be done interactively:
ccmake .
...not that much different (and it usually works on Windows too). IMHO cmake for 'end users' (who just want to build a project into an executable) is mostly fine, it's just not great for the project maintainers because the cmake scripting language is a PITA.
The problem with Autotools isn't the end user experience, it is the developer experience - and guess who has all the power to make a build system choice? :-P
IMO there needs to be a replacement for Autotools that from a user's perspective behaves exactly like Autotools (following the same GNU standards that specify how the configure script, etc is to be used, what parameters it accepts, etc) but provides a better experience for the developer.
My solution for C software that I maintain is handwritten ./configure of 200-ish lines of code, dealing with specific issues that I've encountered, supporting specific systems that are supported (Linux, macOS, Emscripten, MinGW). It looks and feels like Autotools script, outputs a file for make to include later, etc.
If you're a user, familiar
./configure && make && sudo make install
will do the trick for you.
If you're a package maintainer, ./configure supports all the --libdir flags that packaging system sets, (handwritten) Makefile respects CFLAGS, DESTDIR, etc. All of it is extremely hackable since there are no silly layers of preprocessors between you and make.
I guess the secret sauce here is having only OpenSSL as a dependency and the project being a fairly small library.
Hum... Good thing you got a distribution package, and not a development one. From your comment, I expect you to have no idea what to do if you get a development package, as `configure` isn't even there.
You just said that half of the procedure is simple enough. And then, I don't get your claim that nothing is comparable. For example, how is `cargo install` more complex than that? Or less discoverable?
(Oh, and notice that there is no distinction between distribution and development packages for any other system, it's autotools only. Also, you may say "cargo is Rust-only", what is true, but autotools are C-only while pretending all over the docs that they are cross language.)
The biggest problem I ever encountered with autotools when working on a distributed open source project was that every developer must have the same version of autotools installed. Backward compatibility was never a priority, and if one developer was a single release step ahead of anyone else, they would need some macro changed, removed or added that broke things for everyone else.
We switched to waf, which is customarily embedded within the source tree, and that just ends this problem forever.
There is a really great book for everyone who seriously wants to learn autotools: Autotools: a practitioner's guide to GNU Autoconf, Automake, and Libtool / by John Calcote (2nd edition from 2020)
This is not specific to Autotools, but I like learning from well-structured books written by experts in its fields. The book provides a good path on how to handle this rather complex collection of tools without searching and jumping from one obscure internet resource to the other. And after a few hours of reading and experimentation, Autotools are not so unapproachable any more as the common preconception suggests. I agree with others that, once understood and the entry barrier removed, Autotools can be used quite pragmatically and are probably not harder to grasp as competitors in the field.
As someone who started coding cross-platform C/C++ stuff in the mid-90's with Windows being the main platform, I never understood the appeal of autotools. Trying to compile any C/C++ project which used autotools+make on Windows never worked out of the box, best one could was trying to generate a Makefile (and config headers) on a Linux system and hope that it works on Windows (which it actually did surprisingly often because most C/C++ code was portable enough to not actually need any system-specific configuration).
Later it dawned on me that autotools did actually make sense at one time, when there have been dozens of competing UNIX implementations which hadn't been standardized yet. But that time is long gone, autotools no longer has a purpose, just let it rest.
But here's the thing: AFAIK lua builds from just a Makefile. libopencm3 also builds from just a Makefile, and it will build multiple examples for multiple microcontrollers. The BSD guys seem to do just fine with Makefiles.
So, could we not just add a few features to Makefiles so that we can do away with all this other infrastructure?
Personally, I just try to use Makefiles wherever possible. My heart sinks when someone comes out with a new build tool. If you must have a build tool, use either autotools or cmake. Please please please don't introduce some new shiny toy that is "autotools done right" but in practise is at least as much pain. And, for the love of all that is holy, don't write your build tool in Python. There's nothing wrong with Python. It just doesn't belong in build systems. Build systems should be written in C or C++, and nothing else. Not Rust, not Haskell, not anything else.
Well if you have a library or app without dependencies producing working Makefile for multiple platforms can work as with lua. But the moment you start introducing dependencies on external libraries everything breaks very easily even between different flavors of Linux and build tools such as automake or cmake can mitigate the problem of finding correct paths and names.
No-one loves such tools but they are not used without a reason.
I've been using WAF for years and it is indeed written in Python. For complex configuration+build+deployments you need a scripting language and Python is probably the best language for that purpose. Because if you already know Python you don't need to learn a new language.
My main grievance with guile scheme is that libraries are often distributed with autotools. The installation scripts are very brittle and I've had to spend a lot more time than I'd like picking apart how they work.
I gave up on autotools and wrote a small c program that compiles the project along with a configure script that compiles the compile program. Never again will I waste hours diagnosing the latest inexplicable autotools script error. Yes, someone compiling the project on some weird system will have to spend one or two minutes editing my "make.c" program. I wish them well.
I have a simple Ruby script that invokes clang to compile each .c file and then link them together. It's extremely fast for plain C code, even without doing the smart thing and only recompiling changed files.
89 comments
[ 3.3 ms ] story [ 145 ms ] threadUnfortunately the defaults are what they are and many people don't care to fine tune their builds.
It took years for the GNU maintainers to publish a libtool that fixed this bug, and then years for libtool users to upgrade to the new version.
I hate GNU so much.
What we need is a converter from Autotools to cmake, or some other system and then take Autotools to a farm upstate for retirement
- faster builds
- progress indicator
A maintain dozens of mixed autotools, cmake and some header-only/makefile-only projects. The cmake build recipes are constantly behind and fragile. The makefile-only projects are a dirty hack, and very limited. autotools is easily the best, with support for everything. And if it's not in autoconf, then it's in autoconf-archive. meson and ninja are cool, but overkill mostly. We had dozens of such build tools before, and they all died.
I even have to teach GNU maintainers how to use autotools/automake properly, as they cannot deal with recursive subdirs, rather have their own configure.in in each subdir. Of course this doesn't scale well.
No, another hack on top of the auto-* ecosystem won't make things better. CMake's problems are cmakes's problems, but it doesn't make autostuff better in any way
But please tell me again why do I need a buildsystem that needs to check if I have Fortran or if I'm running AIX under MIPS again?
Makefile only is crude but it can work fine if your target is the usual distros and your project is small (read: less trouble than autotools)
If you need Fortran support you need to check for it. Ditto for AIX. Hardcoding assumptions is fragile.
1. autoconf is awful to learn.
2. autoconf can be useful in determining the platform, dealing with a few quirks, and ensuring that system dependencies/libraries are satisfied.
3. autoconf's usefulness here is completely undermined by automake, as changing Makefile.am requires rerunning autoreconf, which then requires rerunning configure. So, developers must constantly rerun the configure script, even though nothing about their system environment has materially changed. For example, this happens when a new source file is created and added to Makefile.am.
4. Perhaps the most egregious missing weakness from TFA is acknowledgement that configure scripts are horrifically slow. It's 2022 and developers have their time wasted by configure checking whether printf() exists.
5. automake documentation discourages developers from globbing source files in source directories, and insists they list them individually. It's silly and just results in needless running of configure.
6. The complex interdependencies between generated files is nigh-impossible to understand and reason about. It's cruft.
7. libtool is completely braindead, the absolute worst of it all. It serves absolutely no purpose on contemporary systems, but the docs continue to admonish developers that they must use it to stand a chance of creating a shared library (no, just learn to specify -fPIC). Instead of analyzing a project once, as part of autoconf, libtool determines what to do on the fly for every source file--and all in bash. It doubles a project's compilation time.
8. Creating alternatives to libtool with new autoconf macros, deprecating libtool, and cleansing all traces of it from the planet should absolutely be the highest priority of the autotools project.
I do agree that m4 is rather unpleasant to write, and that the resulting tests are too slow (often because user defined macros do a poor job of cacheing) but I also think that these things could be addressed (newer macro languages, bundled or threaded micro-tests) while still preserving the good bits of autoconf.
I then replaced automake and libtool with tup, and the Tupfile would just source these variables from configure. It worked great! It's a pity that tup hasn't caught on.
These days I use meson mostly, but I would love to see a complete build system really get into parallelizing platform checking and merge it into the normal build process.
And in this space, it's often a lot easier to get something autotools-based to build than something cmake or meson, at least in my experience. With autotools more often than not all I need to do is to specify the right target hosttriple and be done. With cmake and meson, it's often a fight to figure out how to cross-compile something like that.
This approach of cross-compiling for windows with mingw-w64, sometimes on "proper" linux, sometimes on WSL2, sometimes even on cygwin, seems to be still somewhat common in C/C++ land and in open source in particular, especially for projects that pull in a fair amount of dependencies (such as the aforementioned ffmpeg) - you will find a lot of Windows build guides that say only mingw cross-compilation from linux is supported and tested.
There are some projects that try to be partial dependency "package" managers for mingw, but for whatever reason I never really used them, so I cannot comment on how well they work and how up-to-date they are kept.
I guess, I just like fighting with the interwoven mess of build-systems and dependencies sometimes (and really, I don't spend more than maybe 1 or 2 hours a month on that)... a bit like people doing linux-from-scratch, except probably less time consuming than building an entire kernel and userland step-by-step.
No it's not silly, realistically you can't actually do that anywhere without re-running configure. Any build system based on directory trees will have this limitation. See meson's docs for more on this: https://mesonbuild.com/FAQ.html#why-cant-i-specify-target-fi...
I also came here to comment that libtool is totally useless on modern systems and I hope it gets removed or made optional, but you beat me to it :)
I'd say on some platforms "horrifically slow" is an understatement. At my day job we were building software in Cygwin on Windows machines infected with Symantec's virus scanner which adds hundreds of milliseconds to every process launch. The configure script for Protocol Buffers took something like half an hour to run.
We eventually switched to CMake but not without a lot of pain first. For what it's worth, CMake fixes the speed issue but not much else. It still has its own totally esoteric custom language, it still discourages globbing source files and requires re-running upon adding any file or changing anything, and it has even more insane default settings than autotools (for example file(DOWNLOAD) from an https:// URL by default ignores certificates!)
There is something wrong with all of these buildsystems. I don't know what the solution is but I feel we are very far from it.
Globbing is discouraged exactly for the reason that then cmake can't discover when files should be added or removed from the build process, or build options have changed. But if globbing isn't used there's no reason why manually re-running cmake would be required (there have been some problems in the past with Visual Studio not automatically reloading a modified project, but I haven't seen those problems in a long time).
And the solution to all build systems being bad is imho not to use C/C++. It's model for platform independence is fundamentally broken. It's not like snafu doesn't exist elsewhere too (Python), but other systems are clearly less troublesome (Go, NPM, Cargo).
We use cmake, with Windows as our main (but not the only) target platform. We found that not globbing is slow and impractical for us.
1. We generate VS project files, which then guides the build process. When you add source files using the IDE, the generated project(s) will be updated, no regenerate is triggered, the build system will do the correct thing: only the added file, not everything is (re-)built.
2. If we didn't glob, then given the nice local workflow above, people would often forget to update the cmake files and get CI failures.
3. When injecting cmake regeneration into the build process, then for some strange reason we found a bad serialization of build steps, which slows everything down.
4. The slow part on Windows seems to be that cmake has to use some system call to figure out if source file name casing is correct, which is very slow (for us the slowest part of the generation process by far). I don't recall the details here, but I think globbing somewhat reduces this issue.
All in all, I can say: figure out what works best for you and don't trust experts that discourage globbing altogether.
https://github.com/xmake-io/xmake
Perhaps the error printing during a parallel build could also be improved somehow.
Especially since configure doesn't actually make anything any faster even in most cases where it _does_ detect errors. It aborts at the first error. So when I'm missing packages for example, I will wait half a minute while `./configure` runs, then it errors with a single missing package, then I will install that package and re-run `./configure`, wait another half a minute until it prints another single missing package, etc. Repeat until I have all packages.
I hate autotools so much. Configure scripts should either run in parallel, or cache its results, or both (ideally both), and they should report as many errors as it can before aborting.
You are not supposed to understand it. Stop wasting your time and blaming autotools instead of your process.
They're the most popular targets today. They certainly weren't yesterday, and very probably won't be tomorrow. I've seen a lot of changes over the decades and the attitude that "nobody will ever need more than 640 k" has always been popular. I also say this from the point of view of someone who makes a living with porting software to a reasonably popular platform "today" that is not one of those listed. Also someone who has spent many years doing "maintenance programming" as opposed new development on the leading edge. "Maintenance" accounts for something like 80% of revenue in the software industry, and the old machines in maintenance mode are not always identical to the new sleek affairs on developer's desks.
> 1. autoconf is awful to learn.
Yes. Like vim, it has a learning curve. It's not actually that bad, and the internet is chock full of documentation and examples. It's about on a par with alternatives like CMake and meson in terms of learning difficulty.
> 3. autoconf's usefulness here is completely undermined by automake, as changing Makefile.am requires rerunning autoreconf, which then requires rerunning configure. So, developers must constantly rerun the configure script, even though nothing about their system environment has materially changed. For example, this happens when a new source file is created and added to Makefile.am.
Ah, you got stuck at point 1. You don't have to re-run autoconf after re-running automate (autoreconf knows this), so you don't have to re-run configure. The makefiles generated by automake know this. That's why when you just type "make" after modifying a Makefile.am it will do the right thing automatically, and that is to re-run the config.status script which contains all the autodetected configuration already cached. It's seriously faster than running configure although the printed output is identical (because configure itself generates and runs that script).
> 4. Perhaps the most egregious missing weakness from TFA is acknowledgement that configure scripts are horrifically slow. It's 2022 and developers have their time wasted by configure checking whether printf() exists.
Yes. You pay the price for portability. You have argued "but I don't port my software why should I pay the price? That only helps other people and there's nothing in it for me". OK, fair enough.
> 5. automake documentation discourages developers from globbing source files in source directories, and insists they list them individually.
The documentation also says why. It's a well-reasoned argument. Also, despite it being a pain in the butt it turns out that explicitly showing your work is a benefit in the long run.
> 6. The complex interdependencies between generated files is nigh-impossible to understand and reason about.
They're a DAG; they should be straightforward for someone with a background in programming to grasp. The relationships are there a priori but the autotools document them and make them explicit.
> 7. libtool is completely braindead
OK. Agree with you there. Libtool was a godsend in its time. It's improved massively (yes, I see the complaints about frustrations with it over a decade ago) but it's largely superfluous today.
Portability to what, though?
It's 2022, not 1982. Most of the commercial UNIXes that autotools was written to handle are long dead now. When was the last time anyone saw a (non-embedded) system with CHAR_BITS ≠ 8, for example?
I've yet to see a competing build system that makes the end user usage and option discovery as simple.
Edit: I've not necessarily used and explored all the various build systems, but the few I have used have generally not been as simple, or at least weren't to me obviously as simple.
The end user experience is in every aspect superior to autotools. There is a small downside for cmake compared to autotools: while you have to install the bits for autotools it runs on a shell. There is a bigger downside for meson: you need a minimal python infrastructure up and working. Over time you need that python infrastructure to keep working. Sometimes it doesn't.
I still vastly prefer meson over cmake and much more so autotools. If I have to fix cmake or meson builds, well, really, if you know their languages, it's just no comparison to debugging the 1000s of lines of shell code autotools generates.
I forget what I do in meson. Which means I just read the meson.build and meson_options.txt. They are quite straightforward, and very concise, if you're not Doing It Wrong.
Here are a couple issues I have run into (one being fixed, though too late for some key versions of Ubuntu; and the other just languishing, as I guess "working" is asking for too much).
https://github.com/mesonbuild/meson/issues/6057
https://github.com/mesonbuild/meson/issues/6662
We have a fairly customized integration(https://github.com/buildroot/buildroot/blob/master/package/p...) which we've iterated on for a while that works quite well these days. The cross compilation support with upstream meson has improved a lot over the years and while it's quite mature now IMO it still takes a bit of env setup to work correctly(but once it works it tends to not break much).
Thanks for the report!
When I care to look how to fix it, it's almost always some error on the autotools configuration and I can edit the Makefile to run things correctly. Also, when it breaks, there are normally more broken things on the project. But autotools are always among the first things to break.
It's usually not broken on my own code because I will write a DAG builder in bare C before I use them personally, so I don't know the breakage timing.
If you find yourself debugging generated scripts, you have done something wrong.
cmake .. && cmake --build . && cmake --install .
...and discovering/changing the build options can be done interactively:
ccmake .
...not that much different (and it usually works on Windows too). IMHO cmake for 'end users' (who just want to build a project into an executable) is mostly fine, it's just not great for the project maintainers because the cmake scripting language is a PITA.
IMO there needs to be a replacement for Autotools that from a user's perspective behaves exactly like Autotools (following the same GNU standards that specify how the configure script, etc is to be used, what parameters it accepts, etc) but provides a better experience for the developer.
If you're a user, familiar
will do the trick for you.If you're a package maintainer, ./configure supports all the --libdir flags that packaging system sets, (handwritten) Makefile respects CFLAGS, DESTDIR, etc. All of it is extremely hackable since there are no silly layers of preprocessors between you and make.
I guess the secret sauce here is having only OpenSSL as a dependency and the project being a fairly small library.
You just said that half of the procedure is simple enough. And then, I don't get your claim that nothing is comparable. For example, how is `cargo install` more complex than that? Or less discoverable?
(Oh, and notice that there is no distinction between distribution and development packages for any other system, it's autotools only. Also, you may say "cargo is Rust-only", what is true, but autotools are C-only while pretending all over the docs that they are cross language.)
We switched to waf, which is customarily embedded within the source tree, and that just ends this problem forever.
As an end user and for writing build stuff it both seems to be more complicated than its competitors.
This is not specific to Autotools, but I like learning from well-structured books written by experts in its fields. The book provides a good path on how to handle this rather complex collection of tools without searching and jumping from one obscure internet resource to the other. And after a few hours of reading and experimentation, Autotools are not so unapproachable any more as the common preconception suggests. I agree with others that, once understood and the entry barrier removed, Autotools can be used quite pragmatically and are probably not harder to grasp as competitors in the field.
Later it dawned on me that autotools did actually make sense at one time, when there have been dozens of competing UNIX implementations which hadn't been standardized yet. But that time is long gone, autotools no longer has a purpose, just let it rest.
But here's the thing: AFAIK lua builds from just a Makefile. libopencm3 also builds from just a Makefile, and it will build multiple examples for multiple microcontrollers. The BSD guys seem to do just fine with Makefiles.
So, could we not just add a few features to Makefiles so that we can do away with all this other infrastructure?
Personally, I just try to use Makefiles wherever possible. My heart sinks when someone comes out with a new build tool. If you must have a build tool, use either autotools or cmake. Please please please don't introduce some new shiny toy that is "autotools done right" but in practise is at least as much pain. And, for the love of all that is holy, don't write your build tool in Python. There's nothing wrong with Python. It just doesn't belong in build systems. Build systems should be written in C or C++, and nothing else. Not Rust, not Haskell, not anything else.
No-one loves such tools but they are not used without a reason.