94 comments

[ 1.9 ms ] story [ 158 ms ] thread
Make is very versatile but unfortunately underappreciated today. But just look at buildroot, it’s a ginormous system made with Makefiles.

I only wish for something which works like Make but has a more ergonomic syntax and an ability to define your own functions to determine if a target is up-to-date.

And for the love of everything that’s good, stop pigeonholing make into being a tool for compiling C-shaped programs. It’s useful for more than that.

And for the love of everything that’s good, stop pigeonholing make into being a tool for compiling C-shaped programs. It’s useful for more than that.

Absolutely. When you need to transform one set of files into another and have the parallelism taken care of and the cleanup able to be expressed as a target, it's a great tool. One of many one uses on the jobsite every day.

>ability to define your own functions to determine if a target is up-to-date.

Yes! A modular make-like would allow the tool to fit any scale. Imagine not only being able to define the up-to-date function, but being able to plug into remote build caches, and perform remote execution of build rules. There are already tools like this, such as Bazel, but they are less appropriate for the small scale than Make. The closest that I know of is BuildKit and related projects.

I think the closest we have today to "make with better syntax" is ninja. It lacks pattern rules and macros and other fancy features because it is expected that ninja files are built with a code generator, but for a tool that just handles specifying targets, rules, and dependencies, it works great (and as a bonus it is faster than make).
Which makes ninja not that good enough a replacement, as make has the same problem: it’s so layered in arcane that in most cases you rely on tools to generate Makefiles for you anyway.

There are “alternatives” like some Python frameworks, but they are more “command runners” rather than graph resolvers, they lack inbuilt or simple ways to determine that a target is up to date, or they promise paralellism* (with fine print that maybe someday in a future version).

One thing make is lacking (arcane stuff aside) is dynamic dependency graph, which could have made it viable for things like javascript transpiling and bundling. Even for C projects which are sorta kinda match made in heaven, you need a “makedepend” tool to traverse your sources.

Make can be a great tool, but it requires extreme amounts of diligence and good taste in order to scale well.
Man I love Make. But recently started a new job and we decided to use Just* and it's been fantastic. I doubt I would use Make again unless I was planning to use it as a real build system (which has been the minority use case of my use cases in the last 5 or so years).

* https://github.com/casey/just

Same, except I do not love make. Love did not grow in the three years I have had to use it for C and C++ building and as a command file. I don't understand the love. There are so many footguns. Nothing you want from it is easy. `make` experts always tell you that your makefile is wrong even when it does what it is intended to and when you ask to see a good makefile, you're given a document you can barely understand even though you thought you had good `make` knowledge. Correct handling of header files needs files generated by the compiler the alternative being recompiling everything when you change one header file.

Run

I now use `xmake` for building (found love at last) and basic scripts or `just` for command files

I suspect people love the concept of Make, not its implementation, that’s just Stockholm Syndrome after using it for too long. It does fit beautifully into the „everything is a file“ paradigm tho!

Never used Xmake, looks nice and clean - the Zig way seems promising as well…

I love make, personally, because no other system gives me the level of flexibility and power. I use it for all sorts of things that aren't even building code.

> There are so many footguns.

That's true. That sorta comes with flexibility and power.

> `make` experts always tell you that your makefile is wrong even when it does what it is intended to

I wouldn't do this. "If it's stupid and works, it's not stupid."

> Correct handling of header files needs files generated by the compiler the alternative being recompiling everything when you change one header file.

I'm not sure I understand what you mean here.

Huge vote for Just from me as well.

It has revolutionised how I build my projects, particularly with monorepos and is a massive productivity booster when compared to manual task running.

I dont get it. Isn't "just" simply a "friendlier make" but only for linear pipelines that could have been simple bash scripts in the first place?

(in which case, why not just use a bash script to begin with?)

I picked up a project with make file and it’s insides were so horrible I switched the project to cmake.

I asked ChatGPT to write the CmakeLists.txt for me and after some debugging it worked.

It's kind of infuriating that CMakeLists even has to be manually made, because the compiler can clearly tell you what's wrong about almost every line in it if you mess it up. It should be completely doable to generate at least like 95% of it at compile time. Just so much repetitive boilerplate, like it was made by Oracle or something.
I don’t understand what you’re unhappy about.
About having to specify all deps manually, all executable files, in which directories they are, which libs to link, etc. I mainly use it with ROS, where there's even more custom garbage to specify, like message building, package definitions and more. Don't forget to get it all in the correct order too and list every dep in the correct group depending on how it's used, or it won't compile. Fml.

Things every modern language with a deps manager infers completely automatically without having to ever think about it. Sure it's understandable that C++ is still so far behind the times for legacy compatibility reasons, but I don't see why we haven't yet gradually moved towards something less tedious.

Well that's an interesting new level on the teetering tower of build systems. Perhaps, in the future, projects will simply include ChatGPT prompts instead of CmakeLists.txt directly. Or a script to generate the correct ChatGPT prompt...
(A prompt (to generate a script to generate a prompt)^N) to generate CMakeLists.txt

A prayer that the result is correct is not included.

I miss makefiles. I had a side-project that included a program that converted a text file into a C# file and I wanted that integrated into the build process so when I hit F5, the system would - if needed - invoke the code-gen program and then recompile the .cs file that got generated. Alas, I couldn't find a good way to do it so I ended up adding pre-build and post-build actions. They sorta work but I have to manually invoke builds occasionally.

I asked on stack-overflow if there was a way to do this. https://stackoverflow.com/q/73816110/3568

(Short version of answers: No.)

You certainly can have tool-generated C# files, because the GRPC protobuf compiler does that and integrates it nicely into VS projects. It's not exactly what I'd call simple, though: https://github.com/grpc/grpc/blob/master/src/csharp/Grpc.Too... so I'm not surprised that people don't try to spend days learning it and making it work.

One could probably write a "how to speak MSBuild for people who understand Make" page based on https://learn.microsoft.com/en-us/visualstudio/msbuild/msbui... ; basically

Make -> MSBuild

Item : File (e.g. foo.c)

Target : Target (e.g. %.o:%.c)

(environment) variable : Property

tab-indented shell : Task (usually a C# assembly, although there's a generic "run command" one as well)

Free tip for anyone trying to debug MSBuild: https://msbuildlog.com/ . AFAIK there isn't anything quite as good for Make.

(comment deleted)
Reads a lot like Stockholm Syndrome.

Make is terrible. It's just widespread, not very hard to start with and better than all the alternatives (which often are better, but not as easily found or preinstalled)

But it is obtuse for no good reason other than being a piece of software born in a long-gone era of computing. Whoever loves Makefile has never had to debug the compilation process of any non-trivial build system, with three dozen ways to define variables, rules and targets, spread over multiple files.

When in doubt, I'll create a Makefile (actually, I use `just` for my personal projects because 99% of the time in 2023 one just needs a task runner). There is much better software to write love letters about.

I find a bash script with one big case statement and a hard limit on file length is pretty underrated.
Even if you’re always rebuilding everything, where makefiles win over a flat script is being able to group things, e.g.

  CFLAGS = -g -O2

  all: foo
  foo: ; # how to build foo
  install: install-foo
  install-foo: ; # how to install foo
  clean: clean-foo
  clean-foo: ; # how to clean foo

  all: bar
  # etc.
This is somewhat similar to AWK’s “pattern-action” structure and perhaps literate programming, but otherwise I struggle to describe how it fits into the world of programming languages.

The ability to override variables from the command line is also pretty sweet: e.g. with the above you can get a non-optimized build with `make CFLAGS=-O0 clean all`. But I guess by using ${CFLAGS=...} you could get `CFLAGS=-O0 ./build.sh` to work as well.

Personally I use this pattern to make subcommands:

    main() { 
        if [[ $1 ]]; then 
            cmd_"$@" 
            return 
        fi 
 
        cmd_all "$@" 
    } 
 
    cmd_all() { 
        cmd_something "$@" 
        cmd_some-other-thing 
    } 
 
    cmd_something() { 
        ... 
    } 
 
    cmd_some-other-thing() { 
        ... 
    }

    main "$@"
I find it much easier to understand when you execute things in a predictable order and you can have a `set -x` to follow everything through. If it's too slow, then only the show bits should be parallelized.

At work, on the other hand, every project uses something like this in the Makefile:

    -include $(shell 'pwd')/makefile-modules/*/*Makefile*
That, and wildcard job names makes it quite difficult to follow the flow when things break.
Obviously not terrible for everyone. There are clouds of people who like it, even love it.
I'll prefer debugging Makefiles any day than cmake. But I yield to the de-facto standard.
> > compilation process of any non-trivial build system

I kinda get the feeling they're mainly complaining about autogenerated Makefiles from the output of cmake and similar...

OOhhhh!

If that's the case, then I'll join the chorus complaining about it. Autogenerated makefiles tend to be nasty.

I've recently had to debug LLVM cmake on Windows, with a good dose of .bat scripting. That was truly hell on Earth.
> Whoever loves Makefile has never had to debug the compilation process of any non-trivial build system

Whoever hates Makefile has never been forced to use ant.

Agreed.

I personally reach for Makefiles in exactly one situation: in research, when I’d like to collect lots of results from many computational experiments to make some plots or tables. I might have the whole process of re-running the experiments and rebuilding the final manuscript driven by Make. But even for this task, it feels pretty awkward and painful. I did try using Scons for the same purpose but didn’t like it as much.

Who said anything about compilation?

Compilation is the use-case that helped make to gain traction. Make has long become outdated for the specific use-case, but it otherwise remains a fantastic general-purpose automation tool.

> but it otherwise remains a fantastic general-purpose automation tool.

...can you recommend a static analyzer and linter for Makefiles?

`make`. Like half of all its options are for tracing, debugging and outputting internal information.
> fantastic general-purpose automation tool

It is not though. It's a barely passable automation tool that's just slightly better than bash as a tool runner.

Hey killjoy, your reply reads a lot like the conservative father of a love-smitten youngster.
I don't understand what you are unhappy about.

When maintaining recipes for building packages, i'd always prefer to debug a Makefile where i can insert a `sh -x` than some magic tools that attempts to do everything for me and then fails without observable reason.

> than some magic tools that attempts to do everything for me and then fails without observable reason.

Ok, but then what about autoconf/autoreconf, aclocal, and automake used to generate makefiles for what seems like the entire GNU ecosystem?

At least the "magic" tools that are more modern have documentation. You can print debug a cmake build pretty dang easily, and it at least doesn't pull in the entire user environment implicitly.

Please back up the implicit claim of missing documentation.
> Ok, but then what about autoconf/autoreconf, aclocal, and automake used to generate makefiles for what seems like the entire GNU ecosystem?

I thought we were talking about make, though. Autoconf and the ilk is an entirely different conversation.

It's really not, since they're used to generate or manage so many make builds.
But make is most often used without those other tools. Autoconf serves a very different purpose than make.
(comment deleted)
> Whoever loves Makefile has never had to debug the compilation process of any non-trivial build system, with three dozen ways to define variables, rules and targets, spread over multiple files.

I dunno. I've done that many times over the years, and I still love make. It certainly has its flaws, but it also has benefits. For me, the cost/benefit of make is better than any other such system that I've tried.

If all I want to do is run a compiler over some C files, sure, make is fine. You can love it for that.

That's almost never exclusively what I want to do.

I want to chainload a dependency manager, I want to download, compile, and link those dependencies. I want to run a code generator and integrate that generated code into the IDE's intellisense without ever letting that generated code out of the build tree. I want to generate a compile_commands.json to communicate to my IDE the relationships between the in-source libraries, executables, dependencies, and generated code.

For most build systems it is sufficient to describe the relationships between these components and the rest you get for free. make would require manually specifying every single command, every single tool, every argument. Miserable.

Our team uses it to execute node programs. Starts so much faster than npx.
I am not following how Make is a replacement for npx.

Npx is just for us lazy command-line users who can't be bothered to make sure a Node package is installed before trying to run a command, right? Make would either have to run npx itself, or you'd have to make sure the package you want to run was installed before running its command, in which case you can skip npx... but that's the same with or without make.

I don't know about npx specifically, but you can have the command recipe depend on the installed package, then have a first recipe to do the install only if it's missing. I do something like this with python and virtualenvs.
I could never get my head around Make except for very simple commands.

Was very recently looking for an alternative for automating frequently used commands for our Elixir app – especially for local dev (without using docker) and came across Taskfile (https://taskfile.dev/) and have been liking it quite a bit.

I am guessing this was written by ChatGPT. As far as I know, No one likes make. Imagine having a choice between make and cargo and still picking make. ridiculous.
I like make. So now you can say that as far as you know, no one likes make except one oddball who replied to you on HN.
Two.

Make works better than anything else for what it does.

Once your project grows, then sure, other tools is less work, but they still require arcane and obscure knowledge to make thos big things work.

There's a parallel universe where the designer of make decided that lines starting with any whitespace indicated the commands to run, instead of requiring actual tab characters.
And in that universe Python uses braces.
I always see others complain about Makefile syntax, especially the whitespace, but that is, to me, one of the smallest problems with make. For myself, the top problems are:

- No tracking of changes to build rules. The build rule of a target should be one of its dependencies.

- No protection from dirty builds. Your sources root is also your output root, and make does not help you keep it clean.

- No options for extending the build to use alternate up-to-date checks (e.g. not mtimes), caches, execution mechanisms, etc.

These all have workarounds, but they require the developer to manually include the workarounds in every build rule.

> Your sources root is also your output root

Only if you learnt to program by reading Microsoft examples. Unix developers do not mix compiled files with source files.

Laughs in openssl

Seriously, it’s very project-dependent, but you’re lucky if your thing of choice that you need to build often supports a separate build directory, let alone enforces it.

Yes, it is one of those things that many experienced makefile writers know to avoid, but the tool does not guide new users to this wiser approach. Contrast with Bazel, which does not pollute the workspace with outputs.
But Make isn't really a build tool, even if it was/is intended as such. This kind of thing is exactly what more sophisticated build tools are good for.
One of the foundational concepts in an autotools builds is mixing generated files with source files.
Huh? Generated files are written to the build directory which is only the source directory if you choose to make it so...

    $ cd /tmp/build
    
    $ ../bash-5.2.15/configure
    checking build system type... x86_64-pc-linux-gnu
    checking host system type... x86_64-pc-linux-gnu
    
    Beginning configuration for bash-5.2-release for x86_64-pc-linux-gnu
    
    checking for gcc... gcc
    ... etc ...
    
    $ tree
    .
    ├── builtins
    │   └── Makefile
    ├── config.h
    ├── config.log
    ├── config.status
    ├── doc
    │   └── Makefile
    ├── examples
    │   └── loadables
    │       ├── Makefile
    │       ├── Makefile.inc
    │       ├── Makefile.sample
    │       └── perl
    │           └── Makefile
    ├── lib
    │   ├── glob
    │   │   └── Makefile
    │   ├── intl
    │   │   └── Makefile
    │   ├── malloc
    │   │   └── Makefile
    │   ├── readline
    │   │   └── Makefile
    │   ├── sh
    │   │   └── Makefile
    │   ├── termcap
    │   │   └── Makefile
    │   └── tilde
    │       └── Makefile
    ├── Makefile
    ├── po
    │   ├── Makefile
    │   ├── Makefile.in
    │   └── POTFILES
    ├── stamp-h
    ├── support
    │   ├── bashbug.sh
    │   ├── bash.pc
    │   └── Makefile
    └── tests
    
    17 directories, 24 files
The makefiles and any files they generate are in /tmp/build, the source directory remains unmodified.

(Recursive make, eww!) ;)

We use Make a lot at $dayjob, mostly for running tasks on AWS CodeBuild machines and the like.

I used to curse having to trawl through docs older than myself just to get something working but now I just send the recipe I need to write/edit to ChatGPT and it does it for me.

> with your insistence on using tabs instead of spaces,

You can tell GNU Make to use a leading character other than tab for recipe lines, like the > symbol or whatever:

  .RECIPEPREFIX = >
This is a variable with a leading dot its name, not a directive like .SUFFIXES: or .PHONY:.
I don't really like make, but I use it in all my projects as a bootstrap. You can enter any of my project and do make help and get some help. Also make server will launch a server (static server, phoenix, rails, webpack, esbuild, whatever) and print the port. I just use it to wrap language specific tools. This way I can type the 3 same commands to get started in any of my projects as I change tooling and language 3 times a day :P
My biggest disappointment with make, and almost all it’s clones, is that I have to manually write out pre-reqs and targets.

My computer knows what files I read and write. Use that to calculate what you should do.

Some systems head in this direction — there is “tup”, and a Python program I love called fabricate ( https://github.com/brushtechnology/fabricate ), that records what a program reads, then doesn’t rerun it if no input has changed.

for gcc there is the -M flag to generate dependencies.

similarly, there's nothing stopping you from autogenerating them in some way that you prefer.

I used GNU make to manage a data pipeline with a large number of (mostly Python) scripts written by a team. It had to process lots of data using many steps.

Make worked very well for this use case. It easily auto-generated dependencies, ran tests, and supported parallel execution. In a tiny fraction of a second it determined what needed to be done, and only did that.

We created a small collection that we released as open source software. You can get it here: https://github.com/david-a-wheeler/make-booster

I think a lot of hate on make is due to poor use. If your makefile is complex, refactor it. Auto-generate dependencies (it only takes a few lines in GNU make). And don't use recursive make, that way lies madness. I also think GNU make is the wiser tool; POSIX make lacks too much in many cases.

I've contributed patches to Make, such as the support for grouped targets in a regular direct rule (not pattern). I receive the mailing list and sometimes answer questions.

Many years ago, I transplanted the process image saving code from GNU Emacs into GNU Make. I keyed that to a new option in make, --dump, which would save the executable image and quit after reading all the makefile material, instead of executing them. The resulting executable would come up with all the Makefiles already in memory, and so dispatch the build very quickly.

This was on a project where reading the tree of Makefiles took 30 seconds. If you touched some C++ file in the tree, it would take half a minute to begin to dispatch the command to rebuild it and then link. With the dumped image, it was nearly instant.

---

OMG, I just found the code for this. I thought it had been lost for 13 years. Turns out, I have the whole damn SVN database from that folded startup, of the Linux distro and toolchains stuff.

This would be fabulous to share, if you are able to.
I just wanted to say: thanks for the grouped-targets! I hated having to hack pattern-based workarounds.
The original mental model behind Make is pretty. At its best it feels like prolog for software construction. Like Horne clauses for compiler invocations on source units.

The problem is that the dirty real world of software construction doesn't match this ideal because the tooling is full of OS and compiler and library oddities that require explicit non-declarative interventions.

And that's how we ended up with source trees littered with shell scripts and M4 and autoconf and automake and it was a disaster.

One of the joys of operating in a net-new ecosystem like Rust or Go is having modern build tools built explicitly to avoid all this.

Make filled the "batteries not included" niche for C compilers in the 80s. But the world was too heterogeneous for it to scale nicely for larger, more complicated projects.

You may love make, but make doesn't love you back.
In 2029, make will love you back. MakeGPT!
The Makefile structure is the real king - the simple idea of targets, dependencies and using the plain old shell is very powerful and quite easy to understand (until you get into some of the recursive, iterative bits at least). Very useful if you have custom CLIs and multiple languages in your repo too.
> the simple idea of targets

That works very well for compiling C. But make is often abused for things that don't match that pattern at all, like a Python project when there are no intermediate targets and they are all .PHONY.

Python is actually one of the things I've found it to be great with, you can for example have all your "command" recipes depend on the virtualenv directory, and the virtualenv directory depend on your requirements / lock file. That way you don't have to worry about keeping the virtualenv up-to-date, it'll automatically do so any time the requirements file changes next time you try to do something, and it'll skip that step entirely if nothing changed.

The only downside is that the last part requires a small addition, the install doesn't change the virtualenv dir's last-modified, so that recipe has to end with "touch $@".

(comment deleted)
> And what really sets you apart, is your reliability. You never let me down or give me any trouble. You always do exactly what I tell you to do, without any fuss or drama (...)

I've had so many issues with builds failing when the number of jobs increases, random shit breaking between versions of make, issues with timestamps causing false positives for reconfiguration, and so on - it's really not reliable at all when doing big builds on modern systems where you care about caching, distribution, and parallelism.

Having your other comment in mind, is your grievance with Make in general or some specific Makefiles you have to work with?
It's with make - a modern build system should be content addressed and not rely on mtimes to determine if anything is out of date. Add on that many make builds are nondeterministic and you have a recipe for sadness.
I love make.

I've used dozens of different build systems over the decades, and while each has their own strengths and weaknesses, make remains my favorite.

Same. I honestly don't get the hate.

I'm inclined to think that people who shiver at its name, either haven't bothered to read the manual and go beyond basic features, or have encountered some pretty shoddy makefiles (made possible by make's power and flexibility in the first place). Or, more likely, both.

Random: did you know that you can write your recipes in python? Or any language for that matter? This is as simple as

  SHELL = python
(technically one also has to set shellflags, but python happens to share the same "execute command" flags as bash)
If you're actually going to do this, you'll probably also want:

  .ONESHELL:
This puts the entire recipe into one script, instead of running each line separately, so you can use (for example) variables across lines. It does mean you lose the "abort the recipe immediately if any individual line fails" functionality in make, but for python that's just how it works anyway when an exception is raised.
Oneshell is indeed a useful thing to keep in mind, but it's mostly when you want to retain state between commands; why would it be more relevant to python than other languages?
...you used python as the example.
Yes, and your comment seemed to imply that state between commands in make recipes (and thus use of oneshell) is somehow more important with a python shell than with the default bash shell.

And I'm curious why you say this, as I wouldn't have thought it to necessarily be the case, so I thought perhaps you had some nice insight about the use of python as a make shell that I hadn't considered.

Just because with default (ba)sh recipes it's usually unnecessary, to the point where people are surprised when learn make does this. You're working directly on files so you don't need to carry any state in memory across lines. With python, this isn't really possible - even the most basic "open a file and write a value" spans two lines with state, so you need to write it in an unusual way, use backslashes to escape the newline, or use ONESHELL.