Hey!! No, the regex on the sed command takes care of it. It breaks the target into three parts. One before the ":", one between the ":" and the "##" and one for the rest, after the "##". Then it outputs only the first and the third part
Autotools projects tend to have recursive makefiles. Look up "recursive make considered harmful" for a good explanation of why that's bad. Edit:
https://pic.blog.plover.com/prog/lib/auug97.pdf
Then, the actual benefit that autotools provides, which is portability for C code running on mostly Unix-like systems, is a pretty dead topic today. I am not talking about all the C bashing people do in a place like HN, what I mean here is that C99 and POSIX compliance is pretty universal now in a way it wasn't 20 years ago. And there are much fewer Unix-like systems in common use. (Linux has taken over most.) Nobody has to run a check for whether it's safe to include string.h or the return type of getpid() anymore.
But then why would somebody have so many targets on a makefile? What are the common scenarios for having so many targets in a makefile if not for doing different installation options?
That's like saying "why are there so many shell scripts?" Make is a generalized scripting environment.
So in one project you might have a target that builds an app, a bunch of rules that generate manpages, another that puts it in /usr/local, another that code signs, another that uploads to some app store.
* Because the rules are generated. Building a subpart sometimes means invoking a generated rule.
* Because make is used for other things: building binaries, libraries, examples, test suite, documentation -- then executing them, preparing a test environment, installing them, generating distribution packages, etc.
* Because the system is a framework, and each rule relates to one of many plugins (Buildroot for example).
* Because the system is just very large and complex (linux).
Now to go back to automake. It is still very useful to have automated checks of compliance -- either about some extended warnings supported only by some compilers and not others (clang vs GCC), allowing extensions, static analysis, coverage, etc. That being said, automake is terrible. The two passes to generate the actual build systems means most people are actually reading generated makefile and debugging that. As was said by GP, it is also sub-optimal regarding use of make.
Frankly, it is just much simpler to directly work with makefiles, or change build tool altogether (meson + ninja is nice). Autotools has always been a nightmare to work with in my experience.
Keeping with make, I much prefer musl way for example, where a POSIX shell script examines the dependencies and configures a single makefile giving the project state, then the actual makefiles will consume it to operate properly.
> Frankly, it is just much simpler to directly work with makefiles
agree.
pmake (colloquially bsd make though it arrived late in the CSRG era) allows for the concept of a 'makefile library' that handles common things (e.g. '.include <bsd.prog.mk>' to build a program if the project directory is laid out appropriately) which I think would have been a much better approach for autotools instead of generating makefiles in multiple stages. Unfortunately the autotools approach is many peoples 1st exposure to anything make-like so they are left with a bad impression of makefile complexity.
IMHO pmake also has a nicer and more flexible syntax extension over posix vs gnumake (e.g. can define a dynamic target in a for loop). Unfortunately pmake is not very well known outside of the BSD community.
Reading and debugging the generated makefile should not be the norm IMHO. For the same reason that is isn't usually required to read and debug generated assembly (from C), Java/Python bytecode etc.
Sure, except that the language used to generate the makefile is substantially harder to read in a lot of cases. And it doesn't tell you important things such as how libtool paths are configured or what paths are included in the gcc commands. These aren't that important unless you're cross-compiling for a system that doesn't have the same libc, libgcc, or library set and for which you need to set a "sysroot." Different projects have different ways of doing this, and sometimes a project will use something that sets sysroot appropriately for libtool but then doesn't generate the --sysroot command for gcc.
For historical reasons Cross Compilation against a different root filesystem for a different target architecture is somewhat of a second-class citizen in most automake/autotools build systems and you really do have to read the makefile output to figure out how they've configured their particular instance.
A Makefile that's basically a tool with multiple entry points, like `git'? The one I've got for my current work project has 36 such targets. Some people might have an individual shell script for each one, but this is just what I prefer to do. Everything's conveniently in one file, and it's easy to share code between targets.
The convention I use is for all entry points to be phony targets. Non-entry point phony targets have name starting with _. The helper I can then use is:
I've recently started using Make for my Python project as a convenient way of running arbitrary code checks. I like having targets like the following:
test - runs pytest with whatever config flags are needed
coverage - run test and then check for minimum code coverage
run - run the application with sane defaults
mypy - does type checking (mypy)
lint - run flake8
And I usually have `make all` setup to run everything (static analysis before tests, then code coverage checks).
In another project, I built out some testing infrastructure for a Rust project (setting up a testing database and tearing it down at the end).
I sometimes feel like I "should" use a dedicated tool for this, but Make is ubiquitous and easy enough to use for simple use cases like this.
In my experience, with very large software projects, the multiplicity of targets tends to be related to a combination of multi-stage builds (where you need to build some tools to generate files that are needed to compile the project), installation insanity (e.g., needing to build a large set of locales for upload to the FTP server), and testing (differentiating between multiple test suites!).
Make is not just a build system; it's also an interactive development tool. It's useful to be able to build parts or variants, run tools such as static analyzers, build and run unit tests, flash embedded devices, etc. Make can orchestrate all that without duplicating the effort to track source lists, dependencies, platforms, etc.
Here's a typical example that happens to be my current hobby project. Take a look at the first block comment and the "help" target.
When you install bash-completion on your system, usually a function is defined to autocomplete make rules for you. It works with all projects, as it does not rely on the rules being specially marked. (https://github.com/scop/bash-completion/blob/master/completi...)
Before I knew about it, I had written a similar thing:
# Autocomplete make rules
_make () {
local cur prev words cword cmdline
_init_completion || return
# command to generate the rules
words="$(make -pRrq |\
awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($1 !~ "^[#.]") {print $1}}' |\
sort | egrep -v -e '^[^[:alnum:]]' | xargs)"
COMPREPLY+=( $( compgen -W "$words" -- $cur) )
}
if declare -f _init_completion > /dev/null; then
complete -F _make make
fi
It relies on `make -pRrq`, that is a very useful command to debug your makefiles, especially in a big project. The option `-p` prints the make data-base, -R and -r removes the implicit rules and variables, and -q indicates only asking make a question about current state, avoid executing anything.
Well, it worked for a time, but the bash-completion one is nicer (though much, much more complex). Very nice however to know about `-pRrq`, definitely helps when working on big projects.
This is a good point. One big problem with solutions that match the text of the makefile is that they don't catch dynamically-generated targets.
I have makefiles used for data-crunching that are parametrically-generated using $foreach in gnu make. One typical use case is "make these plots for all the data, for 10% of the data, or for just one example" - with targets generated by a $foreach(...) construct.
Asking make to regurgitate the targets it knows about catches these guys, grep doesn't.
There are many other examples of dynamically-generated rules, such as using macros that create rules via gnu make's "define".
if i'm remembering correctly, and autoconf-style project guidelines have the default target as 'all', so 'make all' for the second step should usually work
The `sed` and `grep` syntax isn't always portable to non-Gnu versions of the tools. The following version doesn't use gnu-isms for `grep` and `awk` for and works on both BSD's and MacOS, sorting and printing comments printed after the ##. Does require on Gnu make for $(MAKEFILE_LIST), but the filename of the Makefile can be used instead:
help: ## Print help for each target
@grep '^[[:alnum:]_-]*:.* ##' $(MAKEFILE_LIST) \
| sort | awk 'BEGIN {FS=":.* ## "}; {printf "%-25s %s\n", $$1, $$2};'
The "remake" tool is "GNU make" + some extra bells/whistles, most notably a REALLY usefule interactive debugger. It's 100% compatible with GNU make because it is GNU Make. And you can ask it for --tasks and --targets.
I have another version based on awk[1], which instead of listing all targets, it will only list targets with a help message in the format of `@: #`, e.g.
install:
@: # Install the application to the given PREFIX.
@: # Unless specified otherwise, PREFIX defaults to /usr/local
which will generates
Usage: make TARGET
You must run make with one of the following targets to continue:
Run make VERBOSE=1 TARGET for verbose log.
help:
Print this help
install:
Install the application to the given PREFIX.
Unless specified otherwise, PREFIX defaults to /usr/local
Yes, also parameter directs execution to the correct subscript if/when needed. No conditional boilerplate needs to be written, arcane enough in shell script as to need looking up every time.
Make is kinda weird too but the complexity appears later. The traffic direction is trivial.
Most newer/younger people don't like or know Make; it's old, arcane software.
It differentiates between TABs and Spaces because each one signifies a different thing to follow: Lines starting with TAB indicate that a _recipe_ follows (which is shell syntax), and lines NOT starting with TAB indicate that a _rule_ follows (ie., native Makefile grammar/syntax).
Hate it or not, it's been a reliable workhorse for decades.
That said, the description above is accurate, but still describes an almost immediately regrettable mistake[0]. Many are used to it now, and the semantics are “fact” now, but could still have been avoided.
Thank you for the epic reference! Despite all the classic works in computer history, I've come to realize that TAOUP is honestly/possibly my favorite. ok, maybe tied for favorite...
(or `make -pn` when you're inside a directory with a Makefile and you don't want to trigger the default target as a side effect) lists all rules including implicit ones, which is great for discovering useful builtin variables. For instance,
This feels like functionality which should be handled within make itself.
The world has too many 'make' replacements. I feel like we just need an incrementally improved make. GNU Make has enough inertia that it isn't going away, but making it slightly easier to use while not confusing old folks like me seems like a win for the community.
'make' is the language that I hold up as an example when I say, "When you are developing a new programming language, do the debugger EARLY because you're going to need one anyway."
Debugging multi-million-line makefiles is not fun when all you have is echo, and one shot every few hours at a race condition that hopefully your debugging output won't perturb.
62 comments
[ 0.27 ms ] story [ 119 ms ] threadThen, the actual benefit that autotools provides, which is portability for C code running on mostly Unix-like systems, is a pretty dead topic today. I am not talking about all the C bashing people do in a place like HN, what I mean here is that C99 and POSIX compliance is pretty universal now in a way it wasn't 20 years ago. And there are much fewer Unix-like systems in common use. (Linux has taken over most.) Nobody has to run a check for whether it's safe to include string.h or the return type of getpid() anymore.
But then why would somebody have so many targets on a makefile? What are the common scenarios for having so many targets in a makefile if not for doing different installation options?
So in one project you might have a target that builds an app, a bunch of rules that generate manpages, another that puts it in /usr/local, another that code signs, another that uploads to some app store.
Frankly, it is just much simpler to directly work with makefiles, or change build tool altogether (meson + ninja is nice). Autotools has always been a nightmare to work with in my experience.
Keeping with make, I much prefer musl way for example, where a POSIX shell script examines the dependencies and configures a single makefile giving the project state, then the actual makefiles will consume it to operate properly.
agree.
pmake (colloquially bsd make though it arrived late in the CSRG era) allows for the concept of a 'makefile library' that handles common things (e.g. '.include <bsd.prog.mk>' to build a program if the project directory is laid out appropriately) which I think would have been a much better approach for autotools instead of generating makefiles in multiple stages. Unfortunately the autotools approach is many peoples 1st exposure to anything make-like so they are left with a bad impression of makefile complexity.
IMHO pmake also has a nicer and more flexible syntax extension over posix vs gnumake (e.g. can define a dynamic target in a for loop). Unfortunately pmake is not very well known outside of the BSD community.
For historical reasons Cross Compilation against a different root filesystem for a different target architecture is somewhat of a second-class citizen in most automake/autotools build systems and you really do have to read the makefile output to figure out how they've configured their particular instance.
The convention I use is for all entry points to be phony targets. Non-entry point phony targets have name starting with _. The helper I can then use is:
(The two spaces that replace ".PHONY" are just indentation, so the list stands out a bit more visually when scrolling back.)The phony target named "default" is the default one, that runs the above command, so I don't need to see its name.
test - runs pytest with whatever config flags are needed coverage - run test and then check for minimum code coverage run - run the application with sane defaults mypy - does type checking (mypy) lint - run flake8
And I usually have `make all` setup to run everything (static analysis before tests, then code coverage checks).
In another project, I built out some testing infrastructure for a Rust project (setting up a testing database and tearing it down at the end).
I sometimes feel like I "should" use a dedicated tool for this, but Make is ubiquitous and easy enough to use for simple use cases like this.
Here's a typical example that happens to be my current hobby project. Take a look at the first block comment and the "help" target.
https://github.com/kbob/MinimumViableSynth-5/blob/master/mak...
Before I knew about it, I had written a similar thing:
It relies on `make -pRrq`, that is a very useful command to debug your makefiles, especially in a big project. The option `-p` prints the make data-base, -R and -r removes the implicit rules and variables, and -q indicates only asking make a question about current state, avoid executing anything.Well, it worked for a time, but the bash-completion one is nicer (though much, much more complex). Very nice however to know about `-pRrq`, definitely helps when working on big projects.
https://www.freebsd.org/cgi/man.cgi?make(1)
https://news.ycombinator.com/item?id=21812897
I have makefiles used for data-crunching that are parametrically-generated using $foreach in gnu make. One typical use case is "make these plots for all the data, for 10% of the data, or for just one example" - with targets generated by a $foreach(...) construct.
Asking make to regurgitate the targets it knows about catches these guys, grep doesn't.
There are many other examples of dynamically-generated rules, such as using macros that create rules via gnu make's "define".
It's more complete, since it implements completion for bsdmake too, where there's no -p but a comparable debug option.[1]
It completes variables in addition to targets, too.
[1] https://www.freebsd.org/cgi/man.cgi?make(1)
But this breaks the classic incantation:
[1]: https://gist.github.com/sirn/6a2f4c0b1cb917ef54b1fb813330d8e...
Here are the full completions: https://github.com/fish-shell/fish-shell/blob/master/share/c...
Honestly, who came up with that `make run` nonsense? I see this quite often in the pythonverse.
Make is kinda weird too but the complexity appears later. The traffic direction is trivial.
It differentiates between TABs and Spaces because each one signifies a different thing to follow: Lines starting with TAB indicate that a _recipe_ follows (which is shell syntax), and lines NOT starting with TAB indicate that a _rule_ follows (ie., native Makefile grammar/syntax).
Hate it or not, it's been a reliable workhorse for decades.
That said, the description above is accurate, but still describes an almost immediately regrettable mistake[0]. Many are used to it now, and the semantics are “fact” now, but could still have been avoided.
[0] https://stackoverflow.com/a/1765566
The world has too many 'make' replacements. I feel like we just need an incrementally improved make. GNU Make has enough inertia that it isn't going away, but making it slightly easier to use while not confusing old folks like me seems like a win for the community.
make -p gets you mostly there.
Debugging multi-million-line makefiles is not fun when all you have is echo, and one shot every few hours at a race condition that hopefully your debugging output won't perturb.