Show HN: Making GNU Make a better Task Runner (github.com)

Thanks, that is a good idea. I also prefer GNU Make for automatization if the number, diversity or dependencies of the tasks would exceed the benefits of a simple bash script.

I stumbled over this statement in your README:

> It is recommended to use .PHONY for targets that are not actual files. In the example below I am not doing that though, but it is wise to follow that rule.

Please also give the example as you think it would be correct. Or write the understandable reason why you don't do it here. I think many people don't read the text between the code snippets but only scan the code examples. It would be helpful to find the ‘best practices’ right there.

remake is a more powerful tool, a fork of GNU Make that adds a debugger and profiler, among other extensions.

https://remake.readthedocs.io/en/latest/

> I know this could be considered blasphemous, but I constantly find myself using Make as a task runner

It's not, don't apologize, reusing battle tested tech is the way.

I have a quite close "template Makefile", though I prefer to store it as a cookiecutter (or more recently "copier") template for easier per-project bootstrap.

You could add a timestamped log function. I use that in mine so that targets have timestamped messages pre/post.

A while back someone linked to this relevant comment on HN, about using make as orchestration: https://news.ycombinator.com/item?id=32441602

I also believe I've seen someone setup tools that can draw the dependency graph for you, so you can view (not edit) it airflow style.

Very nice.

Whenever I see comments about people trying to reinvent make, my conclusion is that make "as a tool" is inevitably far superior, but people are mostly familiar with a small subset of its functionality, which makes it appear clunky. That, and the syntax is very flexible, which means it can easily be abused to write ugly/convoluted makefiles.

If I had one thing I'd want to improve about make, it wouldn't be the tool itself, but its documentation, and a set of reasonable defaults/templates, like you have here.

The documentation is amazing in one sense, in that it's very comprehensive, but very clunky and hard to master in another, in that it's organised in a very high-level manner that doesn't allow it to be used as a quick reference / point of truth, if you don't know what you're looking for. I found myself making anki notes for it, as it's not organised in the kind of way that would simply allow me to refer to the docs for your usecase and get things done. You really either know some functionality exists and can look it up or you don't, and unless you're someone who uses make in expert mode on a daily basis, it's hard to know all the specialised components and be able to combine them all together seamlessly. Hence my anki notes, hahah.

But make really is an amazing tool. I wish people focused on improving make workflows instead of reinventing the wheel all the time with other tools (which are then forced on the user).

  fetch PACKAGE VER:
    @echo fetching {{PACKAGE}} at version {{VER}}

  $ just fetch foo 
  error: Recipe `fetch` got 1 argument but takes 2
  usage:
    just fetch PACKAGE VER

  $ just fetch foo 1.2.3
  fetching foo at version 1.2.3

    GOOS=darwin GOARCH=arm64 go build -ldflags="-X 'main.Version=v1.2.3'"

    GOOS=darwin GOARCH=arm64 go build -ldflags="-X 'main.Version=v1.2.3'"

    $ cat Makefile
    goos := FOO
    goarch := BAR
    mainversion := BAZ

    .PHONY: build
    build: ; @GOOS=$(goos) GOARCH=$(goarch) go build -ldflags="-X 'main.Version=v$(mainversion)'"*

    $ make -n
    GOOS=FOO GOARCH=BAR go build -ldflags="-X 'main.Version=vBAZ'"

    default: your_task
    
    VENV_DIR := venv
    
    $(VENV_DIR)/bin/python: requirements.txt
        python -m venv $(VENV_DIR)
        . $(VENV_DIR)/bin/activate && pip install -r requirements.txt
    
    your_task: $(VENV_DIR)/bin/python
        do_your_thing
    
    clean:
        rm -rf $(VENV_DIR)

    export GOOS ?= darwin
    export GOARCH ?= arm64 
    VERSION ?= 0.0.0+$(shell git describe --dirty --always)

    build:
        $(if $(VERSION),,$(error VERSION was not set))
        go build -ldflags="-X 'main.Version=v$(VERSION)'"

    $ make build (uses darwin/arm64/0.0.0+<git ref>)
    $ make build GOOS=linux VERSION=1.0.0 (uses linux/arm64/1.0.0)
    $ GOOS=templeos GOARCH=sparc make build VERSION=1.1.0 (uses templeos/sparc/1.1.0)

    ---- 
    fetch PACKAGE VER:
        @echo fetching {{PACKAGE}} at version {{VER}}
    ---- 
    $ just fetch foo 1.2.3

    ---- 
    fetch:
        $(if $(PACKAGE),,$(error PACKAGE was not set))
        $(if $(VERSION),,$(error VERSION was not set))
        @echo fetching $(PACKAGE) at $(VERSION)
    ----
    $ make fetch PACKAGE=foo VERSION=1.2.3

    1. Use sshfs to mount your remote target locally on your dev machine, or:
    2. Use vscode's remote-ssh mode (if the agent can run on your target).

May want to look at Just. It is heavily inspired by Make and shares much of the same syntax, but removes a lot of the workarounds necessary to use Make as a task runner and adds a few other features.

https://github.com/casey/just

Nice, I will definitely use these techniques.

Here are a few settings that make GNU make a bit faster, and enable multi-line, strict-mode bash scripts as recipes (make recipes are normally sequences of single-line sh invocations)--and enable quietude & tracing: https://github.com/barries/polling_state_machine_cpp/blob/ma...

people often turn to make as a taskrunner, but i've never understood why. i've heard people say "i don't want to add another tool/language/dependency to my project, so i'll just use make" but, usually, `make` is a new dependency for the project. `make` (realistically) excludes Windows users, most Linux distros don't ship it by default, it is a unique syntax (no, it's not shell). even in a C project, it's not a given that Make is already a dependency. in languages like python or javascript, with rich package ecosystem, `make` makes even less sense; why limit your users to one OS, and why not write your tasks in your project's native language? as a taskrunner, `make` has so many limitations -- because that's not what it's designed for. `make` is good for tracking dependencies between files that are generated from other files, and that's about it.

a good taskrunner makes it easy to run a task while still exposing the tool's underlying flexibility. a good taskrunner lets me invoke the entire test suite with a short command, but also allows me to add custom options and arguments to, say, run a specific test case in an alternate environment.

`make` fails to expose the tools' underlying flexibility. sure, you can write a .PHONY target to run the full test suite, but `make` can't handle passing options or arguments (besides cumbersome Makefile variables).

a makefile tends to obscure the underlying tool, enshrining its launch arguments. anyone who has tried to cross-compile a makefile codebase authored by someone who didn't consider cross-compilation understands what i mean (you'll end up re-writing the Makefile 90% of the time).

  [tool.taskipy.tasks]
  fmt = "nox -s fmt"
  lock = "nox -s lock"
  install = "if [ $VIRTUAL_ENV ]; then pip install -r local-requirements.txt; else printf '%s\n' 'Please activate a venv first'; return 1; fi"
  test = "nox -s test test_without_toml typecheck -p 3.12"
  docs = "nox -s render_readme render_api_docs"

I saw MEX_LICENSE and thought: What, this requires a Matlab licence?!?

I wonder, why GNU make is much more popular than BSD make/pmake.

Both are extensions to classic make (makefiles without advanced features works with both implementations), but, to my eye, pmake extensions is much more simpler and nicer to use, than GNU make extensions.

Simple ".if/.else/.endif" and true ".for" loop, like in sh is much more comprehensible than GNU make functional extensions to me.

Is it only me?

And BSD systems have extensive library for pmake already, which allows to build programs, libraries (both shared and static) and more in declarative way, without any custom rules. It is like these extensions, but battle-tested for 20+ years. With nice license, too.

I've never found the need for a task runner since learning shell scripting.

A lot of people recommend just; here's their sample file:

  alias b := build
  
  host := `uname -a`
  
  build:
      cc *.c -o main
  
  test-all: build
      ./test --all
  
  test TEST: build
      ./test --test {{TEST}}

Here's how I would do that with shell:

  #!/usr/bin/env sh
  set -eu
  
  b="build"
  
  host=`uname -a`

  _cmd_build () {  # build:       Build main program
      cc *.c -o main
  }
  
  _cmd_test_all () {  # test_all:    Run all tests
       _cmd_build
      ./test --all
  }
  
  _cmd_test () {  # test TEST:   Run a test TEST
      _cmd_build
      ./test --test "$1"
  }
  
  _cmd_help () {
      echo "Available targets:"
      grep -E "^_cmd_[^[:space:]]+ \(\) {.+" "$0" | sed -E 's/.*#/  /'
  }

  [ $# -gt 0 ] || _cmd_help
  set -x
  "_cmd_$1" "$@"


  $ chmod 0755 run.sh
  $ ./run.sh
  Available targets:
     build:       Build main program
     test_all:    Run all tests
     test TEST:   Run a test TEST
  ./foo.sh: line 23: $1: unbound variable
  $ ./run.sh build
  $ ./run.sh test_all
  $ ./run.sh test foo.t

But this example (from just's homepage) is clearly a build process, so I would use Make for it instead. When I need to be able to run individual sets of commands with arguments and options, I make a shell script. Many more features available, more flexibility, I can tailor it to my use case.

Thanks! Typically I don’t like solutions that involve extra environment variables as things inevitably get messy over time, but your use cases are clean, simple, and common enough that it’s nice to have them documented in one place in this extra makefile.

Biggest showstopper for me personally is Make's inability to properly pass through arguments from `make run <arbitrary number and type of arguments>` to the underlying program.

Some of the issues could be avoided by requiring a -- and using the .SILENT modifier, but some are extremely difficult: e.g. it word-splits strings and you can't just pass "some string" as a single argument, everything will be separated by spaces.

In case the underlying program is a script-language wrapper that you also control, it is possible to hack around and let Make pass through its own $$PPID so that the underlying script could read the nul-terminated /proc/<Make's pid>/cmdline and pass it on to the actual program.

This, of course, only works where procfs is a thing, so e.g. on macOS (if it's among your target platforms) you'd have to reinvent the wheel and learn a whole lot of Darwin sysctl dark magic to read arbitrary process' arguments, at which point you'd ask yourself if this is even something a sane person would ever do...

I used make as a task runner for probably 30 years; using all the tricks to do most of the things you want in that use case (arguments, help, portability, reliability..). I worked around all the idiosyncracies continually. That's enough time sunk into wasteful friction for me. Now just is here, and the people pushing it are right. For this job (task/script manager for more than trivial commands) it's better enough and reduces cognitive load enough that it's very often worth the install requirement and new learning curve. In time it will be more installed-by-default and I'm certain it'll acquire some form of make-like dependent building as well.

[Apologies for contributing to the slightly off topic discussion. makext sounds great for people still using make for this and I totally would have used it in the past.]