wow i've been feeling like not knowing make has been a major weakness of mine, this article has finally tied all my learning together. i feel totally capable of using make now. thank you.
Makefiles are easy for small to medium sized projects with few configurations. After that it seems like people throw up their hands and use autotools to deal with all the recursive make file business.
Most attempts to improve build tools completely replace make rather than adding features. I like the basic simplicity and the syntax, (the tab thing is a bit annoying but easy enough to adapt to).
It'd be interesting to hear everyone's go to build tools.
> It'd be interesting to hear everyone's go to build tools.
A script written in whatever the primary language of the project is (usually with some library support), ideally; to reduce the minimum required knowledge to be a first time contributor to the project. Some kind of js build tool for js projects, fake for f#, and so on. I don't want people to need to learn "the one build tool DSL to rule them all" (be that makefile syntax, bazel rules, or whatever) to contribute to a project's infrastructure, on top of the project's primary language.
So for C? Fortran? You'd have users of those languages use those languages to write programs to do the build? But then, how would you keep them from generalizing their good ideas about builds into some kind of... system?
Currently, I use scons for C/C++ projects, though with a decent amount of stuff built on top of it. When I realized that I had a rudimentary package manager in it, I started thinking that maybe I should go searching for something that already does what I need.
I've used make quite a bit, and it is doable for individual projects. Where I start having trouble is managing libraries that may be used across multiple projects. When a library needs to supply its own configuration, and be subject to configuration specified of the project using that library, things get rather complicated, which is why I turned to scons.
For my current project in C++ I just use cmake.
It works fine on Linux, Windows, FreeBSD & OpenBSD, spits out stuff that integrates with the native/most commonly used build environment on the respective platform, provides facilities for config, build & installing, and if you avoid all legacy cruft it's even somewhat decent.
I don't really like those, because then it makes it harder to separate out the source and the build directory. VPATH is a partial solution, but I find it much easier to have a single explicit pattern rule, rather than using both implicit pattern rules and modification of those implicit pattern rules.
That's fair. In actual projects of more than a few source files I do copy/paste some boilerplate to give me a build dir, dependency files, and so on. I just thought this (actual real-world example that I use for smallish projects) would help someone.
Thanks... I suspected I was just redefining a built-in pattern rule, but I put it there anyway just so I can remember whether it's LDFLAGS (linker) vs LFLAGS (lexer).
Plus I was thinking maybe less magic for someone trying to make sense of it.
Is there no command needed for name_of_bin to be linked? Does it automatically figure it out based on *.o dependencies?
There are next to none good learning resources on Makefiles. Most are extremely tersely written, have next to no description of how various parts tie together and assume everyone builds C/C++.
The GNU make manual is a good place to start. It's not a tutorial, but it is fairly short. If you're on a different platform it's not exact, but most of the important material applies to POSIX make implementations.
Replace your own toolchain for the CC and related commands... really anything that takes multiple files as input and emits one file as output should fit the paradigm.
> If you're on a different platform it's not exact, but most of the important material applies to POSIX make implementations.
Which parts are "most of the material" though? ;)
> Replace your own toolchain for the CC and related commands... really anything that takes multiple files as input and emits one file as output should fit the paradigm.
I tried to. It's messy and undebuggable if you run into problems. Especially with tools that already look at the whole project (such as Typescript).
We ended up using Makefiles as just "command launchers" with targets that basically look like
That is only true if one does not believe in reading, or makes no effort at all to look for such resources. There are entire books on this subject.
* Clovis L. Tondo, Andrew Nathanson, and Eden Yount (1994). Mastering MAKE: a guide to building programs on DOS, OS/2, and UNIX systems. Prentice Hall.
* Robert Mecklenburg (2004). Managing Projects with GNU Make. Nutshell Handbooks. O'Reilly Media. ISBN 9780596552541.
* John Graham-Cumming (2015). The GNU Make Book. No Starch Press. ISBN 9781593276492.
I don't really know what to tell you, there. Are you generally familiar with the C build process, and bash?
The only "unusual" things I'm seeing there are the $@ (target) and $^ (dependencies) automatic variables, the wildcard and patsubst functions, which have the description right in the title, and the general ``target: dependencies \n\t command-list`` format of Makefile rules.
You might not be familiar with what C compiler and linker commands look like. Here's a listing of what "make" or "make all" actually executes in the example makefile (just take as given that these are normal and understandable commands to compile and link a binary):
cc -Wall -o file1.o -c file1.c
cc -Wall -o file2.o -c file2.c
cc -Wall -o file3.o -c file3.c
cc -Wall -o name_of_bin file1.o file2.o file3.o
and "make clean" executes:
rm -f *.o name_of_bin
The big win is that make looks at modification times of all targets and dependencies. If you type 'make' without changing anything, it completes within microseconds. If you edit only file1.c, then it only recompiles file1.o and re-links name_of_bin. You really miss this kind of speed when moving back to any other kind of dev environment!
> You've learned 90% of what you need to know about make.
That's probably in the ballpark, anyways.
The good (and horrible) stuff:
- implicit rules
- target specific variables
- functions
- includes
I find that with implicit rules and includes I can make really sane, 20-25 line makefiles that are not a nightmare to comprehend.
For a serious project of any scope, it's rare to use bare makefiles, though. recursive make, autotools/m4, cmake, etc all rear their beautiful/ugly heads soon enough.
But make is my go-to for a simple example/reproducible/portable test case.
> For a serious project of any scope, it's rare to use bare makefiles, though. recursive make, autotools/m4, cmake, etc all rear their beautiful/ugly heads soon enough.
If I didn't have to provide the option to build under Xcode or VS, I wouldn't have to live in the hell that is Cmake.
cmake has improved so much since 3.0 that it's almost unrecognizable. Though much of the cruft remains, thanks to backwards compatibility, and anything involving list or string operations is still absolute hell, I dare say it's actually pleasant to use, at least compared to 2.8.whatever-shipped-with-ubuntu-precise.
Personally, I'm glad we use cmake at work simply because we have a diversity of preferred build tools: some people like to work in XCode or Eclipse, others like to use a text editor and make, others like to use a text editor and ninja. While we all suffer a bit with cmake, we all benefit from its position as a "metabuild" tool.
For people who are more comfortable in Python, I highly recommend Snakemake[1]. I use it for both big stuff like automating data analysis workflows and small stuff like building my Resume PDF from LyX source.
One important tip is that the commands under a target each run sequentially, but in separate shells. So if you went to set env vars, cd, activate a Python virtualenv, etc to affect the next command, you need to make them a single command, like:
Make is awesome. I have always loved make, and got really good with some of its magic. After switching to Java years ago, we collectively decided, "platform independent tools are better", and then we used ant. Man was ant bad, but hey! It was platform independent.
Then we started using maven, and man, maven is ridiculously complex, especially adding custom tasks, but at least it was declarative. After getting into Rust, I have to say, Cargo got the declarative build just right.
But then, for some basic scripts I decided to pick Make back up. And I wondered, why did we move away from this? It's so simple and straightforward. My suggestion, like others are saying, is keep it simple. Try and make declarative files, without needing to customize to projects.
I do wish Make had a platform independent strict mode, because this is still an issue if you want to support different Unixes and Windows.
p.s. I just thought of an interesting project. Something like oh-my-zsh for common configs.
make does have a platform independent strict mode. Unfortunately the ".POSIX:" mode is sufficiently limited that you won't want to use it for anything you're writing by hand.
Don't know if there's a Win32 build of GNU make, but Microsoft has nmake. I haven't written a Makefile for it in years, but I have a hard time imagining there being much portability between the two. Not in the Makefile stucture, but rather in the commands it invokes to actually build targets.
I'd guess that nmake is POSIX-compliant (or close to it), simply because the POSIX requirements are so minimal that it would take quite a bit of effort to avoid being POSIX compliant.
Make should have included something like tcl, or shipped its own tcl and its own os/posix libs. It is very easy for makefiles to become tied to a particular OS/Distribution/dev environment.
This probably won't happen for quite a long time, but I'd like to expand my Oil shell to have the functionality of Make.
On the one hand, it's kind of ironic that you're asking for Tcl, when shell is Turing complete and already such an integral part of Make (every line of the Makefile spawns /bin/sh).
On the other hand, I understand that shell is a language with many sharp edges and most people dislike it. The point of Oil is to get rid of the sharp edges, and then maybe shell can take the place of Tcl/Guile.
It seems a little ridiculous to write build scripts in make, shell, and guile, all of which are Turing-complete (not to mention Awk or Perl, which often show up). And I don't actually think Lisp/Scheme are very good languages for Unix-like text processing and syscalls/OS integration.
What happens when /bin/sh is dash or bash? I despise /bin/sh, it is basically PHP. There isn't a terrible amount of basic functionality that would enable makefiles to cross platform (even cross distribution would be nice). Interacting with files, file metadata, processes, exit codes, etc.
Absolutely love your oilshell posts. Keep it up. I think a concatenative language would be great as the embedded logic for Make, but a Lisp would also work.
Yes that is one of the problems with Make -- /bin/sh is different on different machines, and that information appears nowhere in the Makefile. There's no way to test that the commands you're running will actually work on someone else's machine, other than by "memorizing the manual".
Combining make and shell would mitigate this problem. Then you would need one binary in sync instead of two. Well, I suppose you also need "busybox", e.g. cp, mv, mkdir -p, etc. But in practice I think that is less of an issue than shell and make colliding.
I've played with concatenative languages, and read a lot about Forth. I'm not sure I see them as great for "logic". They are very elegant for certain problems, but fall down for others (IIRC the quadratic formula was a popular example.)
The main kind of logic you need in Makefiles is expressing build variants -- e.g. for debug/release, internationalized builds, coverage builds, profile-directed feedback, running parameterized test suites, etc. I think that can be done fairly well with a Python-like imperative language with dicts and lists. (Some people have mentioned Bazel, and it is derived from Python and works fairly well for that.)
It goes through all the commands and common versions of Unix and which flags are likely to work, etc.
Even that book admits there's a lot of folklore, because nobody has actually gone and tested things recently. Something as simple as safely writing with "echo" is a problem. You can argue that any script that uses echo $foo is incorrect (because $foo might be a flag). Conversely 'echo --' is supposed to print -- by POSIX.
The only people who are likely to even attempt this are people whose full-time job it is to write shell scripts, or the authors of tools like autoconf, which must generate portable shell. autoconf shell is a good example of the anachronisms and hoops you need to jump through to support this style.
Nobody else has time for that, because they have to spend their mental energy writing C and C++, not shell and make. So that's why we have pretty low standards for the quality of build systems. The tools aren't there to support writing a robust and easy-to-use build system.
Meta, I really wish I could reply to both chubot and stephenr ...
There is way too much incidental complexity in /bin/sh and almost all build systems. Unix is awesome, but I also hate Unix for being "good enough". Unix Hater's Handbook and all. I hate it from above, not below.
We should always be seeking to reduce the cognitive burden in the tasks we do. One level of fail for /bin/sh is the level of semantic density and the lack of discoverability. It violates the principle of least surprise like nothing else I have used. Look at variable assignment!
export FOO2= "true"
export FOO3="true"
These are semantically different. One evaluates the string, the other does not. And by being so obtuse, the majority of folks randomly mutate their sh scripts until they appear to work. No knowledge gained and nothing that would qualify as engineering.
The biggest problems with sh and Make are that lack of debugging and introspection. Any follow on tool that would displace them should make developer ergonomics the highest priority.
It does, and I don't know why. I mostly ignore votes, and usually only vote when I think a viewpoint needs to get more exposure, not because I think it is "right".
> What happens when /bin/sh is dash or bash?
Dash is specifically designed to be a posix compatible shell for use as /bin/sh, i.e. in shell scripts (or called by makefiles).
When Bash is invoked as "sh", it runs in posix compatibility mode.
There is a reason some of us keep harping on about "don't write bash-scripts, write portable shell scripts".
> I despise /bin/sh, it is basically PHP.
Similarly, I despise chickens. They're basically the sub-prime mortgage crisis.
The guile integration in gmake is a bit shallow, but still quite useful (here's the whole implementation [1][2]). It could also use more real-life usage examples - there's really nothing out there besides a few short snippets.
As a learning experience, I wanted to implement parts of the GMSL[3] on top of guile. It looked like it could be useful to others, so I made it into its own project: the GNU Make Toolkit[4] (it's still very much a work in progress - I've put more effort into the test suite and the documentation generator than into actual functionality).
This is where cmake missed the boat, imo. They were even using Tcl/Tk for a medical imaging project[0] when they decided they needed their own build system. Then when they needed an expressive embedded language, wrote an ad hoc one from scratch.
I do like the straightforwardness of Make. But it doesn't do quite enough for it to be simple to use for C or C++ programs. The simplest generic Makefile I can come up with that handles header file dependencies correctly is 10-20 lines of relatively complex interaction with the compiler.
But then again maybe that's just an argument against C and C++'s compilation model. :)
To be fair, dependency tracking is properly a function of the compiler, you can't do it in make alone without the ability to parse the language. And it's not 10-20 lines. GCC and clang both support the -M family of arguments that do this for you. It's a matter of running it to generate some sort of dependency file and including it in the makefile.
Yes, that's annoying and definitely a problem with the inclusion-based dependency model of C. But really it's not such a big deal.
Right, I was referring to adding the right -M arguments, including the generated .d files, and handling the edge cases with adding/removing files to the build. Overall the minimal Makefile I copy-paste into every new C project is about that size, so I suppose 10-20 lines includes a few other things as well.
> It's a matter of running it to generate some sort of dependency file and including it in the makefile.
This is not as simple as it sounds. Make has a fundamental limitation that dependencies cannot be generated on the fly, and current workarounds for dependency-generating dependencies are very messy [1].
>Although it’s simple, there are serious problems with this method. First and foremost is that dependencies are only rebuilt when the user explicitly requests it; if the user doesn’t run make depend regularly they could become badly out-of-date and make will not properly rebuild targets. Thus, we cannot say this is seamless and accurate.
Not so fundamental, if you quickly make depend after changing includes. Why is that messy? I never got that "buy magic automation, sell simplicity" trading. Deps changed, deps have to be updated. Once updated, they just work.
Those arguments are not enough. For dependencies to be fully reliable, the compiler needs to emit information about where headers were not found, in addition to where they were found. Otherwise headers introduced earlier into search paths after a build do not properly trigger a rebuild.
Honestly, that's just intractable. That can't be solved for the same reason that changes to the makefile itself can't be tracked. Changes to the structure of the build system aren't "dependencies" as commonly understood.
Almost all other systems have this flaw in some way as well. You can fool everything, because a system capable of not being fooled would literally have to reparse and rebuild everything from scratch.
Almost all other systems have this flaw because they require dependencies before a build. If recording dependencies after the build, the entire problem becomes very simple. See here:
The example above was of a change to the build structure (include path, specifically) which changes the dependency tree in a way that doesn't involve changes to the files themselves and will be invisible to make or other tools that use file timestamps. I pointed out that this was just one of a whole class of intractable problems with dependency tracking that we all choose to ignore. It can't be fixed, even in principle. The only truly sure way to know you're building correctly is to build from scratch. Everything else is just a heuristic.
I would like to boast that I have clearly solved an intractable problem, since (as I said) I have tools that happily track this very thing. But the truth is that the idea that it is intractable is rubbish, rather than that I can achieve impossible things before breakfast. (-:
You've switched tack between "not such a big deal" and "can't be solved" in the space of two messages. Both of your extremes are wrong. The truth is that this is simply difficult and needs either improvements to compilers or, as in my case, add-on tools that replicate the compiler's pre-processing and emit both the redo-ifchange and the redo-ifcreate information.
The following sentence about my redo implementation is wrong:
> In 2014, Nils Dagsson Moskopp re-implemented Pennarun redo, retargetting it at the Bourne Again shell and BusyBox.
I targeted the Bourne Shell (sh), not the Bourne Again Shell (bash). Also, my redo implementation contains redo-dot that paints a dependency-tree – I have not seen this otherwere.
You only put /bin/sh as the interpreter. You still used quite a number of Bourne Again shell constructs in the scripts themselves. Some are not in POSIX sh and utilities, some are not in the Bourne shell. (The Bourne shell is not conformant with POSIX sh, so targetting POSIX sh is not the same as targetting the Bourne shell.)
These include amongst others local, $(), >&-, the -a and -o operators to test, and command. (You also failed to guard against variables expanding to operators in the test command, but that is a common general error rather than a Bashism.)
Beware of testing against /bin/sh, even the BusyBox one, and assuming that that means POSIX compliance, let alone Bourne shell compatibility. Even the OpenBSD Korn shell running as /bin/sh or the Debian Almquist shell running as /bin/sh will silently admit some non-POSIX constructs.
The real problem I have is that make generally relies on shell commands like cp, so there's no truly cross-platform (e.g., Windows and POSIX) way to copy files around in a makefile. I guess you could build your own utility, use it, and delete it when you're finished building. I've actually resorted to using ExtUtils::Command ( http://perldoc.perl.org/ExtUtils/Command.html ).
But because it is Make written in Java with a XML syntax, it has inherently the same problems.
Fun fact: They had for years a makefile rant on their home page which boilt down to "Ant was invented because its developer couldn't make Makefiles work as his editor didn't properly show tabs vs spaces": https://web.archive.org/web/20100203102803/http://ant.apache...
How is it more declarative than the Makefile language?
You declare targets and dependencies and the steps within are target a executed procedurally in sequence.
This sentence is 100% true for both Ant and Make.
Ugly, yes. It was developed during the times when everything had to be XML. But also Makefiles don't win a beauty contest.
Terrible. Maybe, but not worse than trying to write portable Makefile for something non-trivial (e.g. libraries).
Limiting? In what way? You are encouraged to use the supplied tasks which cover a lot. Creating new tasks would be done in the language you programming. You can still execute any arbitrary command if you really want to.
In ant, to declare a target for any no direct code generator is a royal pain. Even worse than in Make.
Ant is declarative in that you do not directly refer to files except in tasks that directly handle files. You cannot say that phony task x depends on file y.
I always suspected that ant was written by people who didn't first take a look at what already existed. Make's use of tabs was a mistake, but not a fatal one.
By using pseudo targets only in the example and not real files, the article misses the main point of targets and dependencies: target rules will only be executed if the dependencies changed. make will compare the time of last modification (mtime) on the filesystem to avoid unnecessary compilation. To me, this is the most important advantage of a proper Makefile over a simple shell script always executing lots of commands.
Personal blog spam, I learned make recently too and discovered it was good for high level languages as well, here is an example of building a c# project: http://flukus.github.io/rediscovering-make.html .
Thanks. I took the "making this pretty is the web clients job" approach, it looks nice in firefox reader mode. The two biggest issues are:
1. Syntax higlighting doesn't exist in reader.
2. The index page (http://flukus.github.io/) can't render in reader mode, it didn't hit the "must have a paragraph with at least 68 characters" or whatever the arbitrary limitation is.
For the later I'm hoping the add some sort of meta tag that allows me to enable it in future.
I feel like any discussion of make is incomplete without a link to Recursive Make Considered Harmful[0]. Whether you agree with the premise or not, it does a nice job of introducing some advanced constructs that make supports and provides a non-contrived context in which you might use them.
In seriousness, the linked paper describes Shake, a Make replacement with two party tricks: one, it is implemented as a DSL embedded in Haskell, thus giving a nice way of programmatic rule generation; and two, it supports monadic dependencies, unlike Make's applicative-only ones.
I think the primary thing that makes people fear Makefiles is that they try learning it by inspecting the output of automake/autoconf, cmake, or other such systems. These machine-generated Makefiles are almost always awful to look at, primarily because they have several dozen workarounds and least-common-denominators for make implementations dating back to the 1980s.
A properly hand-tailored Makefile is a thing of beauty, and it is not difficult.
They're very clearly hand made, do just enough to be immensely useful, and are well commented enough to be easy to modify for a Make beginner, just like I was back in the day. I'm sure there are other great examples floating around the 'net, but I cut my teeth on GBA and NDS development, and taught myself make by following in their footsteps.
I keep trying to get into GBA development but this is actually what keeps blocking me. Everything they've written is set up just for their environment and is not really explained how it works. Sure, they explain what to change for your game, but my arm compiler is over here and could you just tell me where this gba.spec file comes from or what it does?! (etc. etc.)
Any build system is as good as the local projects configuration for it. Any solution is good so long as when I press build everything that I wanted to build get built.
Makefiles tend to get really messy when you have to build libraries and components in separate directories. It's definitely tuned more towards building single target projects.
I've seen failures of such schemes to rebuild stuff when command line parameters (defines, environment) changed.
It also brings all sorts of trouble in parallel builds as Make is weak at handling dependencies that are not generated in the exact Makefile you run. (Thus non-recursive Makefile which still fail and have other warts.)
Even cmake and autoconf generated Makefile have trouble with complex projects.
(Part of the reason why cmake can now generate Ninja files instead.)
Eh, but by using includes you can still modularise.
I have a recursive-make project at the moment that suffers from none of the problems described in the paper. We will likely move to an include-based scheme before long, which will take some minor tweaking of targets in the leaf Makefiles.
Most of the things in an autogenerated Makefile are there for a reason: dependency tracking, stuff like "make help", proper cross compile support (trickier than you think!), test running, dist, distclean, configure flags... You could reimplement them yourself, but you're going to end up doing a whole lot of work for little gain.
I'll write Makefiles by hand for small C/C++ projects, but for anything serious I'll use cmake/etc.
Source: We made a "properly hand-tailored Makefile" for Rust. It started out short and elegant, but it quickly grew into a nightmare lasting 4 or 5 long years. Only about one or two people (Alex Crichton and Brian Anderson IIRC) had any clue how it operated. Proper cross-compiling support involved multiple levels of nested variable expansion all over the place to find the right build/host/target compilers so that Canadian cross builds worked. Alex ended up doing some heroic effort to throw away all the Makefiles and going to (effectively) a custom build system using Cargo, which was a massive simplification.
I've been told that, before KDE switched to CMake (becoming the first big open-source project to do so), there were only two or three people (out of hundreds of developers) who dared to touch the autotools stuff, except for peephole edits like adding a new source file to a list. For everything else, one of the experts needed to be brought in.
I only came into KDE when the switch to CMake had already happened, and remember it as reasonably approachable (even if quirky). Most developers were familiar with it and actively authoring the CMakeLists.txt files for their own projects. (That doesn't mean that there weren't some experts again, but they focused on implementing reusable modules that the others could easily integrate into their own build system.)
I'm already struggling to compare variables to strings. Sometimes the string will be interpreted as a variable instead of a string, and I don't know how to avoid that.
Most of the time in CMake, you want expansion, so:
my_function("${MY_VAR}")
Avoid the quotes if your variable contains spaces or other separators, and it's your intention for the separate pieces to go into the function or macro independently.
The other case is builtin macros that expect to be passed the name of a variable that they themselves are manipulating. For example the list and string operations. See: https://cmake.org/cmake/help/v3.0/command/list.html
* bad ordering, at least preventing parallel build (-j N)
* bad error handling which result in silent failures, I've seen it in Makefiles using for/while loops, if/then/else, or successions of commands (cmd1;cmd2) inside targets for example.
Makefiles are also a little too crude for somewhat complex projects. Doing things like searching header pathes, detecting which OS you're on, what is the CPU endianness, searching the required dependencies and recovering information such as version about said dependencies would be extremely painful to do with only plain Makefile.
CMake contains a lot of ready to use modules to tackle the most common issues, and you can easily write your own modules if you need to.
Aside from very simple project, I would not recommend using plain Makefiles, there are just to easy to do wrong.
I imagine anything short and simple enough can be finessed to be beautiful and elegant.
...but scaling from the small to the large elegantly is what make doesn't do well; and there's an astonishing number of tools designed to try to work around the problem.
A lot of people hate CMake for its strange language and (arguably) questionable design choices; but it scales to large projects without significant problems; you only have to look as far as the android native client makefiles to see how the truely heroic efforts to use make have resulted in makefiles that are only... moderately terrible, instead of absolutely aweful.
I feel like there's this nostalgic desire from many programmers to 'embrace simplicity' and avoid the complexity and annoyance of learning and using complex 3rd party tools when 'simple' solutions are good enough.
...but often those simple solutions are ever any good at a trivial scale.
Has anybody successfully used make to build java code? I realize there are any number of other options (ant, maven, and gradle arguably being the most popular).
In fact, I realize that the whole idea of using make is probably outright foolish owing to the intertwined nature of the classpath (which expresses runtime dependencies) and compile-time dependencies (which may not be available in compiled form on the classpath) in Java. I'm merely curious if it can be done.
A long time ago I had a build system for C++ and Java. It worked fine. The issue is generally that you want it to work on Windows and Unix, and then life is hell...
The problem with using make here is transitive dependency resolution.
Many of the new build systems have it "built in" now a days.
Maybe make could wrap something that does that? Either way...I wouldn't recommend it unless it wrapped something that understood maven central.
Sure it can be done. Long, long ago, before Ant existed, I worked on a big Java project built entirely with javac and GNU make. In fact javac's built-in dependency tracking makes this significantly easier than with C.
The trouble with "make" is that it's supposed to be driven by dependencies, but in practice it's used as a scripting language.
If the dependency stuff worked, you would never need
In my real world at work, the "dependency stuff" works just fine. I never clean for my iterative development, and `-B` is what you want for a pristine/paranoia build.
Yeah this is my pet peeve about how people use Makefiles. Make is supposed to be a dependency-driven "dataflow" language, but over time it's evolved into something like shell. And people tend to use it like shell.
The most glaring example is .PHONY targets, which are a hack and should just be shell functions. In 'make <foo>', <foo> should be data, not code. 'make test' doesn't make sense, but 'make bin/myprog' does.
I posted this link in another comment showing how Make, Shell, and Awk overlap:
Here are some more comments on Make's confused execution model. It's sort of functional/dataflow and sort of not. In practice you end up "stepping through" an imperative program rather than reasoning about inputs and outputs like in a functional program.
I recommend using plain shell scripts. Each .PHONY target is a simple shell function, and then the last line of the script is "$@", which means run function $1 with parameters $2 to $n.
So instead of 'make test', I just use './test.sh all', or './test.sh foo'. The test script can invoke make.
The idea is that 'dataflow' parts are done in Make, and the imperative parts are done in shell. This works out fairly well if you're disciplined about it. The only point of Makefiles is to support quick incremental builds. If there's no incrementality, then you might as well use shell. (Note: whenever you use Make, you're using shell by definition. There's no possibility of only using Make. So I take Make out of the picture where it's not necessary.)
For example, all the instructions here are of the form 'foo.sh ACTION':
Pretty much every shell script in the repo uses "the argv dispatch pattern"... I've been meaning to write a blog post about that pattern, i.e. using functions with the last line as "$@".
"Build systems are the bastard stepchild of every software project" -- me a years ago
I've work in embedded software for over a decade, and all projects have used Make.
I have a love-hate relationship with Make. It's powerful and effective at what it does, but its syntax is bad and it lacks good datastructures and some basic functions that are useful when your project reaches several hundred files and multiple outputs. In other words, it does not scale well.
Worth noting that JGC's Gnu Make Standard Library (GMSL) [1] appears to be a solution for some of that, though I haven't applied it to our current project yet.
Everyone ends up adding their own half-broken hacks to work around some of Make's limitations. Most commonly, extracting header file dependency from C files and integrating that into Make's dependency tree.
I've looked at alternative build systems. For blank-slate candidates, tup [2] seemed like the most interesting for doing native dependency extraction and leveraging Lua for its datastructures and functions (though I initially rejected it due the the silliness of its front page.) djb's redo [3] (implemented by apenwarr [4]) looked like another interesting concept, until you realize that punting on Make's macro syntax to the shell means the tool is only doing half the job: having a good language to specify your targets and dependency is actually most of the problem.
Oh, and while I'm around I'll reiterate my biggest gripe with Make: it has two mechanisms to keep "intermediate" files, .INTERMEDIATE and .PRECIOUS. The first does not take wildcard arguments, the second does but it also keeps any half-generated broken artifact if the build is interrupted, which is a great way to break your build. Please can someone better than me add wildcard support to .INTERMEDIATE.
Tup is really great. I wish it were more widely used. Tup can effortlessly handle a lot of situations that Make chokes on or requires deep magic to get right. A good example is the clean handling of automatically generated header files. This is all it takes to integrate protobufs into a Tupfile:
The "|>" is the pipe operator and I've elided the definitions of the "!protoc" and "!cpp" macros (but they're about what you'd expect). Tup detects whenever a .proto file is changed and does the right thing. Getting this to work with Make requires advanced tricks like .PHONY and .PRECIOUS.
I'm going to disagree slightly on redo; yes it punts, but the fact that it reuses sh means you don't have to learn a new, slightly different language that is barely (if at all) better suited to the task. Also my shell linting tools work with it for free.
From my point of view it does more than Make with much less. If all I want is a better make, redo is what I use.
Tup is not a make replacement, because it can do strictly less than what make does (e.g. implementing "make install" is impossible because tup only is for building outputs that are local to the project). However, generating correct build files is so much easier because of the guarantees it enforces. This is despite the fact that I after using it my feelings about its syntax have gone from "hate with fury" to "still don't like it, but with the docs open I can figure out the right syntax for what I want"
One very important thing missing from this primer is that Make targets are not 'mini-scripts', even though they look like it. Every line is 'its own script' in its own subshell - state is not passed between lines.
Make is scary because it's arcane and contains a lot of gotcha rules. I avoided learning Make for a long time. I'm glad I did learn it in the end, though I wouldn't call myself properly fluent in it yet. But there are a ton of gotchas and historical artifacts in Make.
Sneaky pro-tip - use Makefiles to parallelize jobs that have nothing to do with building software. Then throw a -j16 or something at it and watch the magic happen.
I was stuck on an old DoD redhat box and it didn't have gnu parallel or other such things and co-worker suggested make. It was available and it did the job nicely.
Seconded. Really good for preprocessing of data files, or repetitive data pipelines. Just set up a naming convention for the intermediate files and define transformation rules. Then you can drop one new file in somewhere, type "make", and the minimal set of processing will just run.
If you need more power, use the wildcard expansion and "patsubst" type rules to define rules at runtime.
I forgot why I couldn't use xargs -P there. If I had to guess ancient DoD version of RHEL, like say RHEL 5. Note it was released in 2007 and xargs as you mentioned had that since 2009.
Sometimes Redhat backport things but I just remember not finding anything there and then being very happy to have discovered make. I think even with xargs -P I would have gone with make anyway as it involved a few dependencies and checking of file times.
The closest thing I've seen that seem likes a real replacement is Bazel, but I haven't tried using it for anything but a couple toy projects yet. The concepts seem solid though, as expected from Google.
My main complaint about make is that relying on timestamps, while fast, is error-prone and doesn't work with idempotent commands that might refresh the timestamp without altering content.
Make is fine for simple cases, but I'm working on a project that is based on buildroot right now, and it is kind of a nightmare: make just does not provide any good way at this scale to keep track of what's going on and inspect / understand what goes wrong. Especially in the context of a highly parallel build with some dependencies are gonna get missing.
In general also all the implicit it has makes it hard to predict what can happen. Again when you scale to support a project that would be 1) large and 2) wouldn't have a regular structure.
On another smaller scale: doing an incremental build of LLVM is a lot faster with Ninja compared to Make (crake-generated).
Make is great: just don't use it where it is not the best fit.
Using Cmake is so much nicer than make, and it's deeply cross-platform. Cmake makes cross-compiling really easy, while with make you have to be careful and preserve flags correctly. Much nicer to just include a cmake module that sets up everything for you. Plus it can generate xcode and visual studio configs for you. Doing make by hand just seems unncessary.
I always felt that for small setups, makefiles are much more concise and easy to understand. And more generic.
Cmakefile.txt seems always really messy and introduce concepts such as project and exes and dlls. But of course Cmake is super-portable I know.
Every time I think this, I end up going back later and wishing I had just done Cmake. If you just want to compile all the .c files it's easy enough to GLOB and make a couple targets, then come back and make it more sophisticated later
I'm not sure why people compare cmake and make - cmake is essentially a makefile generator to handle automatic dependency tracking. You can even generate other types other types of build files (ninja, visual studio, etc). Not to just point your comment out because I see this all the time, but I don't understand where this confusion is coming from.
You can declare your project structure, metadata, and other build/install/test logic in either. They're thinking about their use cases, not the language and workflow used to achieve them.
Well, if anything, I think it demonstrates that Makefile is something that should be compiled, not written by hand
edit: and for context, this is usually what happens anyway, since autoconf does it too. it's hard to think of a major project with a manually edited Makefile
May I kindly plug my upcoming book [1] for writing CMake in an effective and straightforward manner :-)
I was just porting a large Makefile based build system over to CMake and had the 'pleasure' to find out why recursive Make really is the root of all evil and just doesn't scale well enough for large software systems. At least to my mind.
282 comments
[ 3.0 ms ] story [ 290 ms ] threadMost attempts to improve build tools completely replace make rather than adding features. I like the basic simplicity and the syntax, (the tab thing is a bit annoying but easy enough to adapt to).
It'd be interesting to hear everyone's go to build tools.
A script written in whatever the primary language of the project is (usually with some library support), ideally; to reduce the minimum required knowledge to be a first time contributor to the project. Some kind of js build tool for js projects, fake for f#, and so on. I don't want people to need to learn "the one build tool DSL to rule them all" (be that makefile syntax, bazel rules, or whatever) to contribute to a project's infrastructure, on top of the project's primary language.
I've used make quite a bit, and it is doable for individual projects. Where I start having trouble is managing libraries that may be used across multiple projects. When a library needs to supply its own configuration, and be subject to configuration specified of the project using that library, things get rather complicated, which is why I turned to scons.
From memory here's a Makefile that serves most of my needs (use tabs):
Plus I was thinking maybe less magic for someone trying to make sense of it.
Is there no command needed for name_of_bin to be linked? Does it automatically figure it out based on *.o dependencies?
Personally, I would just write "rm -f" explicitly. I don't see any advantages of using $(RM).
Replace your own toolchain for the CC and related commands... really anything that takes multiple files as input and emits one file as output should fit the paradigm.
Which parts are "most of the material" though? ;)
> Replace your own toolchain for the CC and related commands... really anything that takes multiple files as input and emits one file as output should fit the paradigm.
I tried to. It's messy and undebuggable if you run into problems. Especially with tools that already look at the whole project (such as Typescript).
We ended up using Makefiles as just "command launchers" with targets that basically look like
* Clovis L. Tondo, Andrew Nathanson, and Eden Yount (1994). Mastering MAKE: a guide to building programs on DOS, OS/2, and UNIX systems. Prentice Hall.
* Robert Mecklenburg (2004). Managing Projects with GNU Make. Nutshell Handbooks. O'Reilly Media. ISBN 9780596552541.
* John Graham-Cumming (2015). The GNU Make Book. No Starch Press. ISBN 9781593276492.
The only "unusual" things I'm seeing there are the $@ (target) and $^ (dependencies) automatic variables, the wildcard and patsubst functions, which have the description right in the title, and the general ``target: dependencies \n\t command-list`` format of Makefile rules.
And then proceeds to list almost the entirety of the Makefile
Disclosure: I wrote it so I apologise for any mistakes
That's probably in the ballpark, anyways.
The good (and horrible) stuff:
- implicit rules
- target specific variables
- functions
- includes
I find that with implicit rules and includes I can make really sane, 20-25 line makefiles that are not a nightmare to comprehend.
For a serious project of any scope, it's rare to use bare makefiles, though. recursive make, autotools/m4, cmake, etc all rear their beautiful/ugly heads soon enough.
But make is my go-to for a simple example/reproducible/portable test case.
If I didn't have to provide the option to build under Xcode or VS, I wouldn't have to live in the hell that is Cmake.
I'd just use make.
Personally, I'm glad we use cmake at work simply because we have a diversity of preferred build tools: some people like to work in XCode or Eclipse, others like to use a text editor and make, others like to use a text editor and ninja. While we all suffer a bit with cmake, we all benefit from its position as a "metabuild" tool.
[1]: https://snakemake.readthedocs.io/en/stable/
Then we started using maven, and man, maven is ridiculously complex, especially adding custom tasks, but at least it was declarative. After getting into Rust, I have to say, Cargo got the declarative build just right.
But then, for some basic scripts I decided to pick Make back up. And I wondered, why did we move away from this? It's so simple and straightforward. My suggestion, like others are saying, is keep it simple. Try and make declarative files, without needing to customize to projects.
I do wish Make had a platform independent strict mode, because this is still an issue if you want to support different Unixes and Windows.
p.s. I just thought of an interesting project. Something like oh-my-zsh for common configs.
https://www.gnu.org/software/make/manual/html_node/Guile-Int...
https://jaxenter.com/gnu-make-4-0-adds-guile-output-sync-bre...
FWIW I wrote about how Make, shell, and awk heavily overlap in functionality here:
http://www.oilshell.org/blog/2016/11/14.html
This probably won't happen for quite a long time, but I'd like to expand my Oil shell to have the functionality of Make.
On the one hand, it's kind of ironic that you're asking for Tcl, when shell is Turing complete and already such an integral part of Make (every line of the Makefile spawns /bin/sh).
On the other hand, I understand that shell is a language with many sharp edges and most people dislike it. The point of Oil is to get rid of the sharp edges, and then maybe shell can take the place of Tcl/Guile.
It seems a little ridiculous to write build scripts in make, shell, and guile, all of which are Turing-complete (not to mention Awk or Perl, which often show up). And I don't actually think Lisp/Scheme are very good languages for Unix-like text processing and syscalls/OS integration.
Absolutely love your oilshell posts. Keep it up. I think a concatenative language would be great as the embedded logic for Make, but a Lisp would also work.
Combining make and shell would mitigate this problem. Then you would need one binary in sync instead of two. Well, I suppose you also need "busybox", e.g. cp, mv, mkdir -p, etc. But in practice I think that is less of an issue than shell and make colliding.
Thanks for the encouragement! (If you didn't catch it I linked these build system observations in a sibling comment: http://www.oilshell.org/blog/2017/05/31.html)
I've played with concatenative languages, and read a lot about Forth. I'm not sure I see them as great for "logic". They are very elegant for certain problems, but fall down for others (IIRC the quadratic formula was a popular example.)
The main kind of logic you need in Makefiles is expressing build variants -- e.g. for debug/release, internationalized builds, coverage builds, profile-directed feedback, running parameterized test suites, etc. I think that can be done fairly well with a Python-like imperative language with dicts and lists. (Some people have mentioned Bazel, and it is derived from Python and works fairly well for that.)
Write portable shell scripts (or lines, in the case of a Makefile).
There's some tips here: http://people.fas.harvard.edu/~lib113/reference/unix/portabl... and of course you can refer to http://pubs.opengroup.org/onlinepubs/009695399/utilities/con...
There's no way to test that the commands you're running will actually work on someone else's machine, other than by "memorizing the manual".
I have an entire book on this:
https://www.amazon.com/Beginning-Portable-Shell-Scripting-Pr...
It goes through all the commands and common versions of Unix and which flags are likely to work, etc.
Even that book admits there's a lot of folklore, because nobody has actually gone and tested things recently. Something as simple as safely writing with "echo" is a problem. You can argue that any script that uses echo $foo is incorrect (because $foo might be a flag). Conversely 'echo --' is supposed to print -- by POSIX.
The only people who are likely to even attempt this are people whose full-time job it is to write shell scripts, or the authors of tools like autoconf, which must generate portable shell. autoconf shell is a good example of the anachronisms and hoops you need to jump through to support this style.
Nobody else has time for that, because they have to spend their mental energy writing C and C++, not shell and make. So that's why we have pretty low standards for the quality of build systems. The tools aren't there to support writing a robust and easy-to-use build system.
There is way too much incidental complexity in /bin/sh and almost all build systems. Unix is awesome, but I also hate Unix for being "good enough". Unix Hater's Handbook and all. I hate it from above, not below.
We should always be seeking to reduce the cognitive burden in the tasks we do. One level of fail for /bin/sh is the level of semantic density and the lack of discoverability. It violates the principle of least surprise like nothing else I have used. Look at variable assignment!
These are semantically different. One evaluates the string, the other does not. And by being so obtuse, the majority of folks randomly mutate their sh scripts until they appear to work. No knowledge gained and nothing that would qualify as engineering.The biggest problems with sh and Make are that lack of debugging and introspection. Any follow on tool that would displace them should make developer ergonomics the highest priority.
When Bash is invoked as "sh", it runs in posix compatibility mode.
There is a reason some of us keep harping on about "don't write bash-scripts, write portable shell scripts".
> I despise /bin/sh, it is basically PHP.
Similarly, I despise chickens. They're basically the sub-prime mortgage crisis.
As a learning experience, I wanted to implement parts of the GMSL[3] on top of guile. It looked like it could be useful to others, so I made it into its own project: the GNU Make Toolkit[4] (it's still very much a work in progress - I've put more effort into the test suite and the documentation generator than into actual functionality).
[1]: http://git.savannah.gnu.org/cgit/make.git/tree/guile.c
[2]: http://git.savannah.gnu.org/cgit/make.git/tree/gmk-default.s...
[3]: http://gmsl.sourceforge.net/
[4]: https://github.com/gvalkov/gnu-make-toolkit
[0] https://www.kitware.com/platforms/#itk
The CMake scripting language [0] .
[0] https://cmake.org/Wiki/CMake/Language_Syntax
But then again maybe that's just an argument against C and C++'s compilation model. :)
Yes, that's annoying and definitely a problem with the inclusion-based dependency model of C. But really it's not such a big deal.
This is not as simple as it sounds. Make has a fundamental limitation that dependencies cannot be generated on the fly, and current workarounds for dependency-generating dependencies are very messy [1].
[1] http://make.mad-scientist.net/papers/advanced-auto-dependenc...
Not so fundamental, if you quickly make depend after changing includes. Why is that messy? I never got that "buy magic automation, sell simplicity" trading. Deps changed, deps have to be updated. Once updated, they just work.
* http://jdebp.eu./FGA/introduction-to-redo.html#CompilerDefic...
* http://cr.yp.to/redo/honest-nonfile.html
Almost all other systems have this flaw in some way as well. You can fool everything, because a system capable of not being fooled would literally have to reparse and rebuild everything from scratch.
http://news.dieweltistgarnichtso.net/posts/redo-gcc-automati...
Also, have you looked at the redo tool redo-ifcreate? http://news.dieweltistgarnichtso.net/bin/redo-sh.html
You've switched tack between "not such a big deal" and "can't be solved" in the space of two messages. Both of your extremes are wrong. The truth is that this is simply difficult and needs either improvements to compilers or, as in my case, add-on tools that replicate the compiler's pre-processing and emit both the redo-ifchange and the redo-ifcreate information.
> In 2014, Nils Dagsson Moskopp re-implemented Pennarun redo, retargetting it at the Bourne Again shell and BusyBox.
I targeted the Bourne Shell (sh), not the Bourne Again Shell (bash). Also, my redo implementation contains redo-dot that paints a dependency-tree – I have not seen this otherwere.
These include amongst others local, $(), >&-, the -a and -o operators to test, and command. (You also failed to guard against variables expanding to operators in the test command, but that is a common general error rather than a Bashism.)
Beware of testing against /bin/sh, even the BusyBox one, and assuming that that means POSIX compliance, let alone Bourne shell compatibility. Even the OpenBSD Korn shell running as /bin/sh or the Debian Almquist shell running as /bin/sh will silently admit some non-POSIX constructs.
I plan to work on POSIX compatibility for my redo implementation.
The real problem I have is that make generally relies on shell commands like cp, so there's no truly cross-platform (e.g., Windows and POSIX) way to copy files around in a makefile. I guess you could build your own utility, use it, and delete it when you're finished building. I've actually resorted to using ExtUtils::Command ( http://perldoc.perl.org/ExtUtils/Command.html ).
But because it is Make written in Java with a XML syntax, it has inherently the same problems.
Fun fact: They had for years a makefile rant on their home page which boilt down to "Ant was invented because its developer couldn't make Makefiles work as his editor didn't properly show tabs vs spaces": https://web.archive.org/web/20100203102803/http://ant.apache...
Still ugly and terrible and limiting.
You declare targets and dependencies and the steps within are target a executed procedurally in sequence.
This sentence is 100% true for both Ant and Make.
Ugly, yes. It was developed during the times when everything had to be XML. But also Makefiles don't win a beauty contest.
Terrible. Maybe, but not worse than trying to write portable Makefile for something non-trivial (e.g. libraries).
Limiting? In what way? You are encouraged to use the supplied tasks which cover a lot. Creating new tasks would be done in the language you programming. You can still execute any arbitrary command if you really want to.
Ant is declarative in that you do not directly refer to files except in tasks that directly handle files. You cannot say that phony task x depends on file y.
Now the blog itself is built with make: http://flukus.github.io/building-a-blog-engine.html
1. Syntax higlighting doesn't exist in reader.
2. The index page (http://flukus.github.io/) can't render in reader mode, it didn't hit the "must have a paragraph with at least 68 characters" or whatever the arbitrary limitation is.
For the later I'm hoping the add some sort of meta tag that allows me to enable it in future.
Is there any reason why you didn't include the date of publishing in the pages. Or is it just me who looks for the dates on all the blogs that I read.
[0] http://aegis.sourceforge.net/auug97.pdf
In seriousness, the linked paper describes Shake, a Make replacement with two party tricks: one, it is implemented as a DSL embedded in Haskell, thus giving a nice way of programmatic rule generation; and two, it supports monadic dependencies, unlike Make's applicative-only ones.
A properly hand-tailored Makefile is a thing of beauty, and it is not difficult.
https://github.com/devkitPro/nds-examples
They're very clearly hand made, do just enough to be immensely useful, and are well commented enough to be easy to modify for a Make beginner, just like I was back in the day. I'm sure there are other great examples floating around the 'net, but I cut my teeth on GBA and NDS development, and taught myself make by following in their footsteps.
Meson seems to be the latest fashion in Linux circles, it seems...
You are trying to solve a Turing complete problem in a simplest low impact way as possible.
There is no good solution. Everyone will fall short in one way or another. Hence a new build system. The wheel never stops.
Most build tools do have some kind of Turing-complete tooling built in with their DSL. But I don't think that's an absolute necessity.
These symbols are artifacts/files. Elementary recursive Build-Scripts can be Turing complete even if the overall language isn't.
I've seen failures of such schemes to rebuild stuff when command line parameters (defines, environment) changed.
It also brings all sorts of trouble in parallel builds as Make is weak at handling dependencies that are not generated in the exact Makefile you run. (Thus non-recursive Makefile which still fail and have other warts.)
Even cmake and autoconf generated Makefile have trouble with complex projects. (Part of the reason why cmake can now generate Ninja files instead.)
Recursive Make Considered Harmful
http://aegis.sourceforge.net/auug97.pdf
I have a recursive-make project at the moment that suffers from none of the problems described in the paper. We will likely move to an include-based scheme before long, which will take some minor tweaking of targets in the leaf Makefiles.
Also - Considered harmful essays considered harmful - http://meyerweb.com/eric/comment/chech.html
I'll write Makefiles by hand for small C/C++ projects, but for anything serious I'll use cmake/etc.
Source: We made a "properly hand-tailored Makefile" for Rust. It started out short and elegant, but it quickly grew into a nightmare lasting 4 or 5 long years. Only about one or two people (Alex Crichton and Brian Anderson IIRC) had any clue how it operated. Proper cross-compiling support involved multiple levels of nested variable expansion all over the place to find the right build/host/target compilers so that Canadian cross builds worked. Alex ended up doing some heroic effort to throw away all the Makefiles and going to (effectively) a custom build system using Cargo, which was a massive simplification.
Then again, as a ROS user, I'm also a defacto CMake expert, so perhaps I underestimate how difficult CMake is.
I only came into KDE when the switch to CMake had already happened, and remember it as reasonably approachable (even if quirky). Most developers were familiar with it and actively authoring the CMakeLists.txt files for their own projects. (That doesn't mean that there weren't some experts again, but they focused on implementing reusable modules that the others could easily integrate into their own build system.)
The other case is builtin macros that expect to be passed the name of a variable that they themselves are manipulating. For example the list and string operations. See: https://cmake.org/cmake/help/v3.0/command/list.html
Most hand-tailored Makefiles have seen aside from well known OSS projects (Linux Kernel, suckless stuff) generally had major issues.
Here is a sample of what I've seen:
* not supporting common variables, specially DESTDIR in install target, but also PREFIX, BINDIR, etc, or CC and CFLAGS for compilation. (https://www.gnu.org/prep/standards/html_node/Directory-Varia...)
* bad ordering, at least preventing parallel build (-j N)
* bad error handling which result in silent failures, I've seen it in Makefiles using for/while loops, if/then/else, or successions of commands (cmd1;cmd2) inside targets for example.
Makefiles are also a little too crude for somewhat complex projects. Doing things like searching header pathes, detecting which OS you're on, what is the CPU endianness, searching the required dependencies and recovering information such as version about said dependencies would be extremely painful to do with only plain Makefile.
CMake contains a lot of ready to use modules to tackle the most common issues, and you can easily write your own modules if you need to.
Aside from very simple project, I would not recommend using plain Makefiles, there are just to easy to do wrong.
...but scaling from the small to the large elegantly is what make doesn't do well; and there's an astonishing number of tools designed to try to work around the problem.
A lot of people hate CMake for its strange language and (arguably) questionable design choices; but it scales to large projects without significant problems; you only have to look as far as the android native client makefiles to see how the truely heroic efforts to use make have resulted in makefiles that are only... moderately terrible, instead of absolutely aweful.
I feel like there's this nostalgic desire from many programmers to 'embrace simplicity' and avoid the complexity and annoyance of learning and using complex 3rd party tools when 'simple' solutions are good enough.
...but often those simple solutions are ever any good at a trivial scale.
In fact, I realize that the whole idea of using make is probably outright foolish owing to the intertwined nature of the classpath (which expresses runtime dependencies) and compile-time dependencies (which may not be available in compiled form on the classpath) in Java. I'm merely curious if it can be done.
The most glaring example is .PHONY targets, which are a hack and should just be shell functions. In 'make <foo>', <foo> should be data, not code. 'make test' doesn't make sense, but 'make bin/myprog' does.
I posted this link in another comment showing how Make, Shell, and Awk overlap:
http://www.oilshell.org/blog/2016/11/14.html
Here are some more comments on Make's confused execution model. It's sort of functional/dataflow and sort of not. In practice you end up "stepping through" an imperative program rather than reasoning about inputs and outputs like in a functional program.
https://news.ycombinator.com/item?id=14840696
And at the end of this post, I talk a bit more about the functional model:
http://www.oilshell.org/blog/2017/05/31.html
So instead of 'make test', I just use './test.sh all', or './test.sh foo'. The test script can invoke make.
The idea is that 'dataflow' parts are done in Make, and the imperative parts are done in shell. This works out fairly well if you're disciplined about it. The only point of Makefiles is to support quick incremental builds. If there's no incrementality, then you might as well use shell. (Note: whenever you use Make, you're using shell by definition. There's no possibility of only using Make. So I take Make out of the picture where it's not necessary.)
For example, all the instructions here are of the form 'foo.sh ACTION':
https://github.com/oilshell/oil/wiki/Contributing
Pretty much every shell script in the repo uses "the argv dispatch pattern"... I've been meaning to write a blog post about that pattern, i.e. using functions with the last line as "$@".
https://github.com/oilshell/oil
I've work in embedded software for over a decade, and all projects have used Make.
I have a love-hate relationship with Make. It's powerful and effective at what it does, but its syntax is bad and it lacks good datastructures and some basic functions that are useful when your project reaches several hundred files and multiple outputs. In other words, it does not scale well.
Worth noting that JGC's Gnu Make Standard Library (GMSL) [1] appears to be a solution for some of that, though I haven't applied it to our current project yet.
Everyone ends up adding their own half-broken hacks to work around some of Make's limitations. Most commonly, extracting header file dependency from C files and integrating that into Make's dependency tree.
I've looked at alternative build systems. For blank-slate candidates, tup [2] seemed like the most interesting for doing native dependency extraction and leveraging Lua for its datastructures and functions (though I initially rejected it due the the silliness of its front page.) djb's redo [3] (implemented by apenwarr [4]) looked like another interesting concept, until you realize that punting on Make's macro syntax to the shell means the tool is only doing half the job: having a good language to specify your targets and dependency is actually most of the problem.
Oh, and while I'm around I'll reiterate my biggest gripe with Make: it has two mechanisms to keep "intermediate" files, .INTERMEDIATE and .PRECIOUS. The first does not take wildcard arguments, the second does but it also keeps any half-generated broken artifact if the build is interrupted, which is a great way to break your build. Please can someone better than me add wildcard support to .INTERMEDIATE.
[1] http://gmsl.sourceforge.net
[2] http://gittup.org/tup/ Also its creator, Mike Shal, now works at Mozilla on their build system
[3] http://cr.yp.to/redo.html
[4] https://github.com/apenwarr/redo
From my point of view it does more than Make with much less. If all I want is a better make, redo is what I use.
Tup is not a make replacement, because it can do strictly less than what make does (e.g. implementing "make install" is impossible because tup only is for building outputs that are local to the project). However, generating correct build files is so much easier because of the guarantees it enforces. This is despite the fact that I after using it my feelings about its syntax have gone from "hate with fury" to "still don't like it, but with the docs open I can figure out the right syntax for what I want"
Make is scary because it's arcane and contains a lot of gotcha rules. I avoided learning Make for a long time. I'm glad I did learn it in the end, though I wouldn't call myself properly fluent in it yet. But there are a ton of gotchas and historical artifacts in Make.
I was stuck on an old DoD redhat box and it didn't have gnu parallel or other such things and co-worker suggested make. It was available and it did the job nicely.
If you need more power, use the wildcard expansion and "patsubst" type rules to define rules at runtime.
Sometimes Redhat backport things but I just remember not finding anything there and then being very happy to have discovered make. I think even with xargs -P I would have gone with make anyway as it involved a few dependencies and checking of file times.
I use a build system, redo rather than make, to rebuild my GOPHER site and Debian/FreeBSD package repositories. (https://news.ycombinator.com/item?id=14837740 https://news.ycombinator.com/item?id=14928216)
Also see this if you have a system without GNU Parallel: http://oletange.blogspot.dk/2013/04/why-not-install-gnu-para...
My main complaint about make is that relying on timestamps, while fast, is error-prone and doesn't work with idempotent commands that might refresh the timestamp without altering content.
In general also all the implicit it has makes it hard to predict what can happen. Again when you scale to support a project that would be 1) large and 2) wouldn't have a regular structure.
On another smaller scale: doing an incremental build of LLVM is a lot faster with Ninja compared to Make (crake-generated).
Make is great: just don't use it where it is not the best fit.
* this can generate ninja files instead of make
* it adds the common targets such as make help, make clean, make install
* dependency graphs can be generated with cmake --graphviz
* solutions for various IDEs
* etc...
You can declare your project structure, metadata, and other build/install/test logic in either. They're thinking about their use cases, not the language and workflow used to achieve them.
edit: and for context, this is usually what happens anyway, since autoconf does it too. it's hard to think of a major project with a manually edited Makefile
[1] https://leanpub.com/effective-cmake