One thing I've found useful about gulp and webpack is the fact that it's multiplatform.
What is the best approach to make sure your Makefile will work as expected on every platform, meaning, windows included?
For example: Can we write file paths using forward-slashes, or is it still an issue? I imagine running it via git bash (bash provided by the git installer on windows).
Good point - cross platform issues are a concern. I've noticed in the past that OS X machines tend to run older versions of Make (unless the owner has installed a newer version with Homebrew). It is a good idea to pay attention to the Make features that you use, understand which features were added in recent versions, and be aware of which features are specific to GNU Make. In my projects I recommend that users make sure that they are in fact using GNU Make.
Some systems might not support the `-p` flag in `mkdir -p`, or the `-rf` flags in `rm -rf`. There are scripts distributed on NPM called `mkdirp` and `rimraf` to work around such issues.
Yes! I recently used Make on some JS projects and my coworkers looked at me like I was insane. Even if you don't know advanced Make-fu it's a really good way to run all of your build steps in the right order without some crazy JSON config format.
After going through a few build systems for Javascript, I realized they were all reinventing the wheel in one way or the other, and pulled out venerable Make from the closet. It turned out to be way more expressive and easy to read.
One target to build (prod), another to run with fsevents doing auto-rebuild when a file is saved (instant gratification during dev), then a few targets for cleaup & housekeeping. All said, the file is 1/4 the size of any other JS-based build systems.
Something about the Makefile in your post is a bit weird though: You only declare "test" as a PHONY target, but every other target in the file is just as PHONY. And since you don't even explain that line, it could be a bit confusing.
Vim does handle this. Here is Vim (and I've removed my .vimrc file, so this is not a setting I have set personally), on a Makefile where the third line erroneously uses spaces:
> I just don't see how insisting on tab and not space is remotely defensible.
This is like insisting that people haven't been having religious wars over this sort topic for years. You can see the Wiki[1] page on it (which starts "This is one of the eternal Holy Wars") for arguments as to why someone might prefer tabs. I personally like them because they allow the reader of the code to choose the visual layout of the indentation.
Now, while I love tabs, I work in Python all day. But good editors, like Vim, allow you to customize the indentation settings sufficiently to handle both cases like Python (spaces for indent) and Makefile (tabs) gracefully. So "Indent" and "Dedent" just do The Right Thing™ and it mostly never matters.
If someone is really paranoid about white space issues in a file you’re editing on vim, you can can always use `:set list` to display those characters.
FWIW, the more experienced a programmer is, the more likely they are to prefer spaces over tabs.
My pet theory is that the more experience you have, the more likely you are to have worked with Python, which is a lot easier to work with if you just always s/\t/ / (replace tabs with spaces).
Maybe this isn't "real-world" but I've made this cookiecutter template which generates projects with Makefiles less than 100 lines. It works decently well for my personal projects and for playing around. https://github.com/MatanSilver/cookiecutter-cproj
It's certainly better than the 200+ lines of JavaScript something like Webpack spits out when you statr a new project. I had to debug some of our project's build files and realized halfway through that my teammates didn't write this Webpack config monstrosity, Webpack generated it by /default/.
Sure, I can sympathize. That said, I've seen a bunch of "here's my results after upgrading to Webpack 4" tweets going around, and they all indicate much faster builds and somewhat smaller bundle sizes, with very minimal config changes needed.
The reason I don't like doing this is portability. Since the steps within the makefile are going to be run through a shell, it is going to behave differently on different systems.
If your makefile fixes up a file using sed and your system has gnu sed, your makefile may fail on a system with BSD sed (e.g., a mac). If you rely on bash-isms, your makefile may not work on a debian system where it will be run with dash instead of bash. And so on.
If you look in your package.json, you'll surely see a dozen or so "scripts" lines that run through the same shell that make does and have all the problems you just mentioned.
I'd also like to point out that Linux and almost certainly your production environment (because it's most likely *ix) will be case sensitive. Your macOS or Windows file system? Not so much. Point is, you're already up to your neck in portability issues. My macOS coworkers often forget this detail.
If an external package has scripts in it, those scripts have very likely been run by somebody on a mac and somebody on linux and worked in both cases. That is totally different than writing a line of script that only you have ever run and assuming that because it works on your laptop, it will run everywhere.
> That is totally different than writing a line of script that only you have ever run and assuming that because it works on your laptop, it will run everywhere.
huh?? No it's not. That's the whole discussion we are having now. I even specifically mentioned my macOS coworkers who develop on their Mac and are oblivious to case-sensitivity issues. Because "it worked for me."
> If an external package has scripts in it, those scripts have very likely been run by somebody on a mac and somebody on linux and worked in both cases.
Oh, I'd love for that to be true. But also not what I was referring to. Most large projects have a "scripts" section and those are no different than Makefile commands. If you're paranoid about your own Makefiles then you're going to need to be paranoid about your own package.json.
Not to mention that gulp, grunt (do people still use this?) and webpack configs are written in JavaScript, so they're likely to have very hard to debug errors in them on the first 5 revisions.
Because `require` and `import` look at your package's `main`, which (if it's a directory) implies `dirname/index.js`. Building to an `index.js` entry file is, then, the most standard name that allows `require 'modulename'` to work.
The capitalized rending-of-garments is silly. This is transpiling; file names should remain the same from input to output.
Why not? They're both entry points (hence 'index'), both JavaScript (hence '.js') but in different directories. Do you have difficulty distinguishing between /home/me/.vimrc and /home/somebodyelse/.vimrc too?
It's because there is always lag between the latest .js spec and what is supported by browsers. So transpilers (like babel) have to dumb it down from fancyNew.js to somethingIE6MightRun.js
It is not realistic/optimal to try to write your javascript to be supported by all browsers or runtime clients. It's better to write using the latest spec, and just have it dumbed down for you by transpilers at build time
I think it’s useful to distinguish compilation whose target is another language that was originally intended to be human-readable (transpilation) from compilation to a form intended exclusively for machines (compilation to machine code or bytecode).
Personally, I think of transpilation as a subset of compilation. So a transpiler is also a compiler, and a compiler that has a readable language backend can do transpilation, which is a type of compilation.
u/masklinn still asks a good question though: there are tools that act on the source code for both "compilation" and "transpilation" (eg Kotlin), depending on the target platform. They do not distinguish, why should we?
For the same reasons you might ever want a more specific term?
What about assemblers or disassemblers? Given a suitably broad definition of compiler that includes transpilers, are they not also included? Wouldn't the same arguments apply?
> For the same reasons you might ever want a more specific term?
I asked why you might want this here and there's been no answer yet. Having a term for something you don't need or want to know isn't actually useful.
> What about assemblers or disassemblers? Given a suitably broad definition of compiler that includes transpilers, are they not also included? Wouldn't the same arguments apply?
Because these are actually useful qualifier, in the same way that "a C compiler" is a useful qualifier.
Compilation is the act of going from one language to another, machine or not. This is what I was taught at least. "Machine code" or "bytecode" holds no special distinction.
In this article's case, there a reasonable exception to made when JavaScript is being "compiled" into JavaScript itself. For that, I don't know what the term is (or if there even is one).
"Transcompiler" was the older term for assembly -> assembly.
I don't think it's too much of a stretch to shorten that to "transpiler". I think people hate the term because they associate it with JavaScript hipsters who have no sense of CS history and presume they've invented something new. The same as how you cringe when someone refers to the "#" character as "hashtag".
But "transpiler" is a useful term. We're in a world now where source-to-source compilers are much more prevalent than they were ten years ago, and there are real workflow differences between working with a compiler that targets a low-level language versus a high-level one. Things like how you debug the output are very different.
> there are real workflow differences between working with a compiler that targets a low-level language versus a high-level one. Things like how you debug the output are very different.
Like what? When I've had to debug GCC, I just dumped out the GIMPLE et al.; what do you do differently with a 'transpiler'?
Sorry, "debug the output" was a really confusing way to say what I meant. What I meant was users will want to debug their code, and the way they do that is often affected by what their code is compiled to.
With a transpiler, it's fairly common to actually debug using the generated output. That pushes many transpilers to place a premium on readable output that structurally resembles the original source code, sometimes at the expense of performance or code size.
Other times, users will rely on things like source maps in browsers. That gives more flexibility to the transpiler author but can make user's debugging experience weirder because things like single-stepping can get fuzzy when a "step" in their source program may correspond to something very different in the transpiled output.
In compilers to lower-level languages (machine code, bytecode, etc.) there is a strong implication that most users can't debug the output, so the compilation pipeline and tools are obligated to have pretty sophisticated debugging support.
In other words, different tools, priorities, constraints, etc. Having a separate word to help tease these apart seems useful to me.
All of that I view more as examples of how immature the tooling is on the frontend, rather than an inherent distinction between source->source and source->machine compilation.
I remember crappy C toolchains that only gave you a .map file for debugging. It was just table of where the linker put the symbols from the intermediate objects. Hell, there were a lot of really crappy C compilers then that looked way more like babel than gcc. They basically just parsed into an AST and walked it to spit out asm, hardly doing any optimizations.
Admittedly translator and transcompiler where probably the more popular terms, but the term transpiler can definitely be found in the literature of the 80's. I know last time this came up someone posted a link to an article from the 70s using the term in that context.
> [...] there a reasonable exception to made when JavaScript is being "compiled" into JavaScript itself. For that, I don't know what the term is (or if there even is one).
Well, given
trans : (+ acc.) across.
and
ipse ipsa ipsum : himself, herself, itself.
I propose we call that ipsumpilation ;)
More seriously though if the input is js and the output is js, and it’s not different versions of the language — e.g. es6 -> es5 (because imo that qualifies as different languages) — I’d call it an optimizer or a packer depending on what it does.
Compiler: collects sources from various places, assembles the result into machine code.
Transpiler: collects sources from various places, converts the source to another language.
Personally I cringe at the newspeak you seem to imply we should all be applying to our lives. These words mean things - quite different things, it turns out. There's a difference between human-readable source code language, and machine-executable binary code. These tools function in different ways entirely; your optimization to the language is not only un-warranted, but leads to a desultory effect: programmers get stupider when they don't know what their tools are actually doing.
foldoc.org is the product of a single individuals' construction of definitions of computing terms and is by no means a complete and authoritative source of technical terms. It says so right there on the site...
4 refs in my post, check it. You won't find "full stack development" in the roots of computing history either; language is malleable and useful. Get over it.
I'm not so angry about the word transpiler; perhaps it might improve things.. but to imply that using compiler as a common term for all things here is a regression/neologism is patently wrong - transpiler is the newcomer.
I'm OK with using the word transpiler, but don't try to modify existing words. A compiler translates from language to language. If you want to say that the compilers whose target is a high level language should be called transpilers, that's ok, but that makes them a subset of compilers, not mutually exclusive.
A compiler converts code in one language to another language. That’s the literal definition. In the case of machine code, the “collection” step would be performed by the linker.
gcc compiling some C and Babel compiling some JavaScript are both performing the same role. So much so the similarities in how they carry it out are striking.
Just look at all those magic things. The percent signs! $<! $@!
Well, I know they are not magic ;). But why would I want them when I can actually use normal names like "deps"/"entries" and "target"?
It gets substantially worse as we go down the rabbit whole. Where webpack can easily walk the entire dependency tree by itself, we have to invoke
src_files := $(shell find src/ -name '*.js')
Where we can use same webpack to seamlessly output the resulting file into an output directory, we need to do the (very unintuitive) pattern substitution:
As a non-programer the only experience I have with makefiles are unpleasant ones of running ./configure then ./make only to go down an endless rabbit hole of missing dependencies. I am very grateful that yum/apt-get, etc. have come such a long way, they grab all your dependencies, you don't have to wait long periods of time for the code to compile.
I am sure Make still has it's uses, but I am sure glad package managers of have made makefiles irrelevant to me.
This is not really a fair comparison though. Makefiles and package managers are orthogonal and the latter has not really replaced the former. Makefiles are like a recipe, package managers are food delivery. Somebody has still to cook the food.
I'd argue it's entirely fair with other examples (e.g. compare and contrast to, say, Rust's cargo build.) You can write a Makefile which fetches and builds your dependencies. Makefiles might be like a recipe - but these days we're building recipes for building entire OS images, including grabbing the dependencies in the first place. A single software project by comparison is trivial.
The problem is Makefiles are a jack of all trades, and master of none. You have to write or copy a distressing number of rules yourself, per project, per subproject, and know the underlying build chain in relative depth to do even some rather basic things. I'd argue makefile authors not automating their dependency fetching is a symptom of this.
As a programmer, I avoid them, because I'll end up stuck maintaining them and end up lowering the build system's bus factor. With perhaps the exception of a couple of simple rules that forward to a "proper" build system, because I've written enough of them in the past that "make" still feels like the right default action that should generally work. But if it all possible, never for the meat of the build system.
Of course. But I would not compare cargo build or CMake to package managers because again, they are developer tools and unless you are a developer you should not need them.
I definitely think that we can do better than Makefiles, but again they have the benefit to be extremely versatile so they can be bent to many uses. (personally I do not use them, I use CMake and generate ninja files)
Autotools (the ./configure) is generally a pain to work with. It essentially spits out a massive script and incomprehensible makefile that allows your program to be used by the three people who still run HP-UX or AIX.
Good makefiles are simple, and should Just Work™. The dependencies should (theoretically) be listed in the project documentation, though it often isn't.
Package managers are definitely better for non-developers, though. Build systems generally aren't geared toward regular users, so users will find them confusing.
Working somewhere that really embraced Makefiles is actually very nice. If you keep your dependencies simple, the Makefiles to build stuff are pretty simple too, and it works for all the languages you might write software in. You can use it to deploy to servers too.
It lets you generate makefiles in a less verbose way. Like if you have a lot of files in a project or a bunch of targets Makefiles can get a bit unwieldy by themselves.
I've only ever used it for C++, but I don't see why it (or something like it) couldn't be used for other languages too.
CMake is mostly dedicated to C/C++ projects, but also has built-in support for assembly, Fortran, and Java. CMake has templates for adding a new language as well. The benefits are: cross-platform "shell" commands (you can copy files without using an explicit cp), standard lookup for programs, and easy target generation from a collection of files. Most languages are pretty similar to C/C++ when it comes to the build cycle. Portability (i.e. testing for existence of headers) is less of a concern for something like Javascript, but everything else still applies.
I'll second this. I had to use CMake for the first time recently and was pleasantly surprised by the experience.
This tutorial was making the rounds (on HN and elsewhere) last week, but in case anyone missed it, here's a link: https://github.com/pyk/cmake-tutorial I found it to be a great primer/refresher.
cmake is good but it is quite heavily geared towards C/C++ development. I would love to have some modern makefile with a bit nicer syntax. Fitting into 80 columns is no longer a must.
Interesting. How why the size of the card affect the size of the screen? I have assumed that it was kind of a good number for size of screens at that time.
It most probably wasn't a limitation so much as a convention carried over from one generation of technology to the next in order to ensure the adoption of the latter.
Make may seem crude these days but has manageable complexity, I've never run across a build issue I could not debug. Its model is so simple, I can have it fully in my mind and be confident this is how it works. Moreover, this model simplicity allows me to build reliable systems on top of it.
CMake is a nightmare of implied complexity. I've run into multiple situations (usually but not always involving cross-compilation) where it was simply incomprehensible. No amount of time invested would let me figure out why CMake blew up.
There is no way I can build something a little out of the ordinary on top of CMake and be confident that it's solid. It's just too complicated.
This sort of balancing, upfront ease of use vs hidden complexity comes up often and I have been burned enough in the past that experience dictates to pretty much always go for model simplicity and pay the upfront costs. I don't see this often however. A lot of the time people will go for what feels or looks right, superficially, without bothering to look beneath the surface or think about model complexity costs. It's an attitude that has led to many disasters in this space.
Fair enough but it could be that CMake isn't the right tool across the board. It works well for the projects I've used it for. Maybe one day I'll come across a case that makes me hate CMake though who knows ;)
I do embedded work. I agree CMake is horrible when cross-compiling and runs into trouble. The errors make no sense. It took me days to get OSG cross-compiling. It was a horrible experience.
I'd much rather a Makefile where I can override the compiler and flags for my configuration. I can easily figure out what I need to do based on compiler and linker errors for missing headers and libraries.
As a distro packager, Autotools may be a mess, but debugging a package that won't build is fairly straight-forward. If the package uses CMake though, and there's a weird build issue, debugging that can be a nightmare! Everything is so opaque, with tons of implied complexity! CMake may be nice to write, but it's hell to debug.
I like this idea a lot. I personally don't like writing makefiles and so having nice build tools like cmake generate them for me is about as close to writing them as I want to get. Letting make stay as a low level building block for high level build tools seems like the right way to go.
We use make for standardizing the way docker containers are built, pushed, tested, and run (debug and production modes). I even prefer it to docker-compose at this point, because it is more programmable.
Not the OP, doing a bit less that it sounds like the OP is doing, but we built out a relatively handy make-based build system for building a set of images in correct dependency order.
Another member of the team subsequently taught make about the reverse dependencies so that you can split jobs across Travis nodes by the top-ish level image they depend on.
My favorite addition was the ability to generate a dependency graph using graphviz straight from the Dockerfiles.
N.B. Project is now moribund, the team was disbanded. May not build at all. Don't know if any of the forks are active.
Thanks for chiming in. That all sounds very interesting and this thread has got my gears turning.
I regret not considering Make for Docker administration months/years ago. I've taken to using bash scripts or, worse, tagged bash comments to recall commonly used complex commands.
That's pretty interesting about Travis. Make is great because you can do as much or as little with it as you want, but it generally always improves organization.
- I do major.minor.yy-mm-dd versioning, so this handles that automatically
- shell is for dropping you into a prompt with your present directory mapped to the working directory. I find this super helpful for debugging and testing containers
- run is for testing your CMD and entrypoint scripts
Do you handle dependencies (images, networks, volumes), and if you do - how?
I'm of an opinion that Makefiles with Docker are mostly a fancy way to write `case "$1" in build) ... run) ... esac` except that one needs to have make(1) installed to run them (in my experience, Bourne shell is much more commonly available than any make variant)
I've generally had to only do this for more-or-less stand alone webservices, but there's nothing preventing making extra targets for setting up networks. To date, I'm still using docker compose for that.
I started doing makefiles as a reaction to having a bunch of bash scripts cluttering up my directory. More or less just a collection of useful commands. But as I continue to do this, I find it encourages making consistent build routines/versioning. Also, being able to set up dependencies ( like "run relies on build") can also be helpful. Finally, variable substitution is useful for versioning.
I think what makes makefiles great are you can do as much or as little as you want with them. I put an example of a common one in another reply.
Make's interface is horrible. Significant tabs. Syntax which relies on bizarre punctuation... If only whoever authored Make 40 years ago had had the design acumen of a Ken Thompson or a Dennis Ritchie!
We're stuck with Make because of network effects. I wish that it could just become "lost" forever and a different dependency-based-programming build tool could replace it... but that's just wishful thinking. The pace of our progress is doomed to be held back by the legacy of those poor design decisions for a long time to come.
Make is such a horrifically awful thing to work with that I just end up using a regular scripting language for building. Why learn another language with all its eccentricities and footguns when I already know several others?
Another advantage of using a scripting language is that the hard work of portability will already have been done for you by the authors of the scripting language.
Make, in contrast, works within a shell and invokes programs which may be either wildly or subtly incompatible across platforms. Add in the lacking support for conditionals in POSIX make and portability is a nightmare.
it is possible that if you need to do that much in a makefile that you are running into shell incompatibilities then perhaps you are attempting to cram too much complexity into your build system and perhaps should be using something like Docker to decrease incidences of “works on my machine/os/distro/et c”.
(I am aware that this advice does not hold true in all cases; just that for many of them overly complicated build systems is a code smell.)
To be fair, at the core much of what you're going to be doing is invoking command-line tools. I mean, most compilers are invoked as command-line tools.
because then people new to your project can see a makefile and know that “make” and “make test” and “make install” probably work without having to learn your homebrew, one-off build system.
I disagree. It’s an argument for convention. This is the same reason we have package.json or Dockerfile or Pipfile or Rakefile - it tells us the standard interface for “install this”. It’s not specific to make.
Docker is new and didn’t have any network effect until recently.
Incremental builds. It's a pain in the ass to write this in a good, generic way yourself. If your build tools don't already understand it, then make (and similar tools) makes for a nice addition versus just a script that invokes everything every time.
EDIT: Oh, and parallel execution, but smart parallel execution. Independent tasks can be allowed to run simultaneously. Very useful when you have a lot of IO bound tasks. Like in compilation, or if you set it up to retrieve or transmit data over a network.
It's not too hard to do that in your custom script, but more care is required because until you custom script reaches make level internal complexity you will have to manually track dependencies or make sub-optimal assumptions.
I very much prefer Rake for orchestrating multiple build systems in a web project or dependency installation or just any sort of scripting. It comes with a simplified interface to shelling out that will print out the commands you are calling.
If for some reason the project is ruby allergic, I'll try to use Invoke [0].
Sometimes I feel like people's usage of Make in web projects is akin to someone taking an axe and hand saw out to break down a tree for firewood when there are multiple perfectly functioning chainsaws in the garage.
1) All dependencies are on only one output file (e.g. link stages that generate .map files, compile phases that generate separate .o and debug files). I just treat these as normal
2) I may need to depend on each of the files (I don't use yacc/bison, but it sounds like this would qualify). I select one file to be the main output and have the .do file for that ensure that the secondary outputs go to a mangled name; I then have the .do files for the secondary outputs rename the mangled file.
If the h output changes but the c doesn’t, this rule will miss it.
I once solved it by making a tar file or all the outputs and extracting it to both .c and .h but that’s incredibly kludgy, still looking for a better solution
As long as the timestamp changes on the C file, that's fine, right? At least with the version of redo I use timestamps are used by default, not file contents.
Because, like many other things in programming, you'll end up with a half-baked and buggy implementation of make anyways.
Incremental builds by looking for changed dependencies, a configuration file with its own significant identifiers (i.e. a build DSL shoehorned into JSON or YAML), generalized target rules, shelling out commands, sub-project builds, dry runs, dependencies for your own script, parallelization, and a unique tool with (making an generalization here) insufficient documentation.
If you're really unlucky, you'll even end up with the equivalent of a configure.sh to transpile one DSL and run environment into the DSL for your custom tool.
> Because, like many other things in programming, you'll end up with a half-baked and buggy implementation of make anyways.
I'd argue that make is a half-baked and buggy implementation of make - so that's not really a drawback so much as the status quo.
E.g. I have scripts that exist mainly to carefully select the "correct" version of make for a given project to deal with path normalization and bintools selection issues on windows - and none of these ~3 versions of make work on all our Makefiles. One of those versions appears to have some kind of IO bug - it'll invoke commands with random characters missing for sufficiently large Makefiles, which I've already gone over with a hex editor to make sure there wasn't some weird control characters or invisible whitespace that were to blame. So, buggy and brittle.
That is, unless the folks who maintain and champion make want me to use make. It's on them to court me, the cross-platform developer, not the other way around.
I kind of disagree, it's quite possible to use gnu make on Windows with msys (though requires some scripting discipline and probably not for everyone).
I'm currently doing this for a cross platform c++ project. Same makefile used for Linux, Mac, and windows+msys+cl.exe. (yes, fair amount of extra variables and ifdefs to support the last one...)
I'd argue that there are some major concerns with the makefiles if they require the use of 3 different versions of make to get it all working - a situation I've never personally seen before. I'd suggest prioritizing fixing that before attempting tracking down the cause of other issues. As it stands, there are too many points of interaction to attribute any bugs to any one program.
That said, Windows has never been a strong platform on which to run make (or git, gcc, or any other [u/li]nix originated CLI tools). When I hear of folks using make, I tend to make the assumption that they're running on a [u/li]nix or BSD derivative.
I've already had one upstream patch rejected on account of the additional complexity fixing it introduces, and would rather not indefinitely support my own fork of other people's build setups.
Or if I am going to indefinitely support my own fork, I might as well rewrite the build config to properly integrate with the rest of whatever build system I happen to be using - at least then I'll get unified build/compilation options etc.
Do you honestly think Make has no advantages over conventional scripting languages when it comes to building software? I suspect you know that it’s designed for that task and has been used for that task for several decades. Presumably you respect the community over those decades sufficiently to have a strong prior belief that there are good arguments for Make (as well as downsides), even if you can’t be bothered to research them.
I honestly think any advantages it has are significantly outweighed by all its disadvantages.
And no, I don't respect the community. The community very often makes "The Way Things Are Done" its personal religion and refuses to ever change anything for the better.
Before C was invented there were already other companies writing OSes in high level languages, but yeah thanks to its victory now it gets all the credits.
- Burroughs, now being sold as Unisys ClearCase, used ESPOL, later replaced by NEWP, which already used the concept of UNSAFE code blocks;
- IBM used PL/8 for their RISC research, before switching to C, when they decide to go commercial selling RISC hardware for UNIX workstations
- VAX/VMS used Bliss
- Xerox PARC started their research in BCPL, eventually moved to Mesa (later upgraded to Mesa/Cedar), these languages are the inspiration for Wirth's Modula-2 and Oberon languages
- OS/400, nowadays known as IBM i, was developed in PL/S. New code started to be replaced by C++. It was probably the first OS to use a kernel level JIT with a portable bytecode for its executables.
BitSavers and Archive web sites are full of scanned papers and manuals from these and other systems.
Everybody I know found it confusing at first, but its logical, and modular. I don't know a language with a better type declaration syntax. It gets impractical when you define function that return functions that return... because these expression grow on the left and right simultaneously. But realistically, you don't do that in C, and other than that, I find it easy to read the type of any expression...
const int *x[5];
*x[3]; // const int
x[3]; // const int *
x; // ok - there is first a decay of x[5] to *x, so: const int **
Is there any other syntax that has this modularity?
It is quite reasonably easy enough compared to any real practical programming task. For example:
// your ordinary function declarations
int plus2(int i) { return i + 2; }
int times2(int i) { return i * 2; }
typedef int (*modfun)(int);
// a very reasonable syntax altogether
modfun get_modfun(bool mul) { return (mul ? times2 : plus2); }
C is a systems programming language. It doesn't have closures or other features from functional programming. In other words, you can't "create functions at runtime". That is why you basically never see a function returning another function.
So, go is also a "systems" language, so the terms more or less meaningless now. Assuming you mean a language we can easily compile to an independent binary capable of being run directly on a microprocessor with no support, I offer you rust as a counter-example.
Also, functional programming and returning functions does not mean they are created at runtime.
It's a fact that C doesn't have closures. That is my point. I happen to like that fact, but you don't have to agree with me.
And "creating functions" means: closing over variables ("closures"), or partial application. I think it takes at least that to be able to return interesting functions.
(and whether go is a systems programming language is at least debatable. I think the creators have distanced themselves from that. It depends on your definition of "systems". You can't really write an OS in go).
Not unless your first two sentences have absolutely nothing to do with each other. Your point appears to be that because it's a low-level language it doesn't have these features, which is false.
> I happen to like that fact, but you don't have to agree with me.
Or I just think you don't have any experince using better languages. That isn't to say that other language could supplant C, but just that it's difficult for me to image actually liking the C type system (or lack thereof) and lack of first class functions. It's incredibly limiting and requires a lot of hoops to be jumped through to do anything interesting.
> And "creating functions" means: closing over variables ("closures"), or partial application.
Well, you said at runtime. Of course you can "create" functions at compile or programming time!
> Your point appears to be that because it's a low-level language it doesn't have these features, which is false.
I would think that first class closures do indeed _not_ belong in a low-level language. They hide complexity, and you want to avoid that in low-level programming. Not necessarily for performance reasons, but more from a standpoint of clarity (which in turn can critically affect performance, but in subtler ways).
> Or I just think you don't have any experince using better languages.
Nah, I have experience in many other languages, including Python, C++11, Java, Haskell, Javascript, Postscript. The self-containment, control, robustness and clarity I get from a cleanly designed C architecture is just a lot more appealing to me. The only other language I can stand is Python, but for complex things, it becomes actually more work. For example, because it's so goddamn hard to just copy data as values in most languages (thanks to crazy object graphs).
> It's incredibly limiting and requires a lot of hoops to be jumped through to do anything interesting.
It depends on what you are doing. It's a bad match for domains where you have to fight with short-lived objects and do a lot of uncontrolled allocations and string conversions. My experience in other domains (including some types of enterprise software) is more the opposite, though. Most software projects written in more advanced languages are so damn complicated, but do nothing impressive at all. They are mostly busy fighting the complexity that comes from using the many features of the language. But those features help only a bit (in the small), and when you scale up they come back and bite you!
> Well, you said at runtime. Of course you can "create" functions at compile or programming time!
Closures and partial application are done at runtime. The values bound are dynamic. So in that sense, the functions are indeed created at runtime. I sense that you were of the impression that a closure would actually have the argument "baked" in at compile time (resulting in a different code, at a low level) instead of the argument being applied at runtime. That's not the case, unless optimizations are possible. If that was really your impression, this makes my point regarding avoiding complexity and that closures do not belong in a low-level language. (Look up closure conversion / lambda lifting)
I'm really not sure what you mean by "modularity". There's a lot of languages with much more readable and composable type declarations than C. For example in OCaml:
let x : (int list ref, string) result option =
let x1 : int = 0 in
let x2 : int list = [ x1 ] in
let x3 : int list ref = ref x2 in
let x4 : (int list ref, string) result = Ok x3 in
Some x4
> Else why would there entire families of C-like languages ?
C became popular because Unix became popular, and when designing a language that intends to become popular one aims for a ratio of 10% novelty and 90% familiarity. Like, ever wonder why Javascript's date format numbers the months starting from zero and the days starting from one? It's because Eich was told to make JS as much like Java as he could, and java.util.Date numbers the months from zero and the days from one, which Java itself got from C's time.h. (Not coincidentally, Java and JS are both in the C-like language family.)
> You're writing real words
In C? Not compared to ALGOL, COBOL, Pascal, and Ada, you're not. :)
> the computers does the things you tell it to. To the letter.
As long as you're not using a modern compiler, whose optimizations will gleefully translate your code into whatever operations it pleases. And even if one were to bypass C and write assembly code manually, that still doesn't give you complete control over modern CPUs, who are free to do all sorts of opaque silliness in the background for the sake of performance.
To be fair with myself I was mostly snarking around op comment who dismissed the C language as if it was some ancient relic.
But even without accounting for compiler optimizations and cpu architecture, the C language just straight up lies to its user. You could code something entirely with void*.
PS: what I meant by "real words" is that you're naming functions and calling them by their name. Which in itself is very powerful.
I've got no issue with C syntax either, but to be clear, syntax != semantics.
To cut to the chase:
* syntax = structure
* semantics = meaning
C syntax would include things like curly braces and semicolons, whereas C semantics would include things like the functionality of the reserved keywords.
Most languages become popular because they have a reputation of necessity. Which is to say, they are popular due to marketing, not due to quality. (As most things are.)
Maybe I'm in the minority, but I've always found its syntax to be quite nice (though admittedly a departure from most modern languages). Then again, I find using JSON or not-quite-ruby to configure a build incredibly bizarre and confusing, so I guess I'm just set in my ways...
In all seriousness, what's wrong with it? Significant tabs aren't great, but I feel like that's a relatively minor wart. The simple things are _very_ simple and straightforward. The more complex things are more complex, but usually still manageable...
I've seen plenty of unmanageable Makefiles, but I haven't seen another system that would make them inherently cleaner. (I love CMake, but it's a beast, and even harder to debug than make. If it weren't for its nice cross-platform capabilities, I'm not sure it would see much use. It's also too specialized for a generic build tool. Then again, I definitely prefer it to raw Makefiles for a large C++ project.)
$< $> $* $^ ... Not particularly explicit. You also have the very useful substitution rules, like $(SRC:.c=.o) which are probably more arcane than they ought to be. You can make similar complaints about POSIX shell syntax but at least the shell has the excuse of being used interactively so it makes sense to save on the typing I suppose.
That's my major qualm with it however, the rest of the syntax is mostly straightforward in my opinion, at least for basic Makefiles.
give pmake a shot sometime.. the syntax/semantics are much more 'shell-like' imho and some things are just much more possible.. (e.g. looping rather than recursive calls to function definitions)
Isn't that a GNU extension? Here's an other problem right there, figure out which dialect of Make you're using and their various quirks and extensions.
I don't like the syntax much, but I love the programming model. I think people who are used to imperative languages are put off by the declarative programming model of make.
1. Claiming a rule makes a target, but then fails to make that target, ought to be a runtime fatal error in the makefile. I can hardly even guess at how much time this one change alone would have saved people.
2. String concatenation as the fundamental composition method is a cute hack for the 1970s... no sarcasm, it really is... but there's better known ways to make "templates" nowadays. It's hard to debug template-based code, it's hard to build a non-trivial system without templates.
3. Debugging makefiles is made much more difficult than necessary by make's default expansion of every target to about 30 different extensions for specific C-based tools (many of which nobody uses anymore), so make -d output is really hard to use. Technically once you learn to read the output it tends to have all the details you need to figure out what's going wrong, but it is simply buried in piles of files that have never and will never be found in my project.
4. The distinction between runtime variables and template-time variables is really difficult and annoying.
5. I have read the description of what INTERMEDIATE does at least a dozen times and I still don't really get it. I'm pretty sure it's basically a hack on the fact the underlying model isn't rich enough to do what people want.
6. Sort of related to 2, but the only datatype being strings makes a lot of things harder than it needs to be.
7. Make really needs a debugger so I can step through the build, see the final expansions of templates and commands, etc. It's a great example of a place where printf debugging can be very difficult to make work, but it's your only choice.
That said, I'd sort of like "a fixed-up make" myself, but there's an effect I wish I had a name for where new techs that are merely improvements on an old one almost never succeed, as if they are overshadowed by the original. Make++ is probably impossible to get anybody to buy in to, so if you don't want make you pretty much have to make something substantially different just to get people to look at you at all.
Also, obviously, many of the preceding comments still apply to a lot of other build tools, too.
I'm happy to see this kind of detailed criticism. I would be happy to use a new tool if it is similarly general, and has a declarative style. Other commenters brought up Bazel, which I am looking forward to learning about.
Concerning expansion, I guess the article is not claiming that it is good, it doesn't even mention this "feature". I guess it's just saying: "make can do everything your gulp can, it's better, faster and more readable", that's without using any variable expansion feature.
As soon as more JavaScript developers start writing very very simple Makefiles, tooling will improve and maybe someone will come up with a better make.
The other option is to let people keep using webpack and gulp until they come up with another JS-based build-system, webpack 5 or 6, and grulpt or whatever comes after grunt/gulp.
> I have read the description of what INTERMEDIATE does at least a dozen times and I still don't really get it.
It took many reads, but I think I get it.
As a toy example, consider the dependency chain:
foo <- foo.o <- foo.c
^^^^^
`- candidate to be considered "intermediate"
Depending on how we wrote the rules, foo.o may automatically be considered "intermediate". If we didn't write the rules in a way that foo.o is automatically considered intermediate, we may explicitly mark it as intermediate by writing:
.INTERMEDIATE: foo.o
So, what does "foo.o" being "intermediate" mean? 2 things:
1. Make will automatically delete "foo.o" after "foo" has been built.
2. "foo.o" doesn't need to exist for "foo" to be considered up-to-date. If "foo" exists and is newer than "foo.c", and "foo.o" doesn't exist, "foo" is considered up-to-date (if "foo" was built by make, then when it was built, "foo.o" must have existed at the time, and must have been up-to-date with foo.c at the time). This is mostly a hack so that property #1 does not break incremental builds.
These seem like useful properties if disk space is at a premium, but is something that I have never wanted Make to do. Rather than characterizing it as "a hack on the fact the underlying model isn't rich enough", I'd characterize it as "a space-saving hack from a time when drives were smaller".
----
If, like me, you don't like this, and want to disable it even on files that Make automatically decides are intermediate, you can write:
.SECONDARY:
Which tells it 2 things:
a. never apply property #1; never automatically delete intermediate files
b. always apply property #2; always let us hop over missing elements in the dependency tree
I write .SECONDARY: for #a, and don't so much care for #b. But, because #1 never triggers, #b/#2 shoudn't ever come up in practice.
(I'm making a separate comment from the INTERMEDIATE explanation)
I'm a big fan of Make, but appreciated your detailed criticism, and found myself nodding in agreement.
#3: I like to stick MAKEFLAGS += --no-builtin-rules in my Makefiles, for this reason. This, of course, has the downside that I can't take advantage of any of the builtin rules.
With the debugging expansion thing you're mentioning, now I'm craving a make built on some minimalist functional programming language like Racket where "expand the call tree" is a basic operation.
> I've seen plenty of unmanageable Makefiles, but I haven't seen another system that would make them inherently cleaner.
Not to push the particular product, but the approach:
FAKE (https://fake.build/), is an F# "Make" system that takes a fundamentally different tack to handling the complexity. Instead of having a new, restricted, purpose built language they've implemented a build DSL in F# scripts.
That yields build scripts that are strongly typed, just-in-time compiled, have full access to the .Net ecosystem and all your own code libraries, and are implemented in a first class functional language. That is to say: you can bring the full force of programming and abstraction to handle arbitrarily complex situations using the same competencies that one uses for coding.
As the build file grows in complexity and scope it can be refactored, and use functionality integrated into your infrastructure code, in the same way programs are refactored and improved to make them manageable. The result is something highly accessible, supportive, and aggressively positioned for modern build chains... If you can do it in .Net, you can do it in FAKE.
Make was created by Stuart Feldman at Bell Labs in 1976. The fact that it is still in use in any form and still being discussed here is a testament to what an amazingly good job he did at the time. Whether it is the right tool for any given modern use case is up to the people who decide to use it or pass it by. I still work with almost daily, and it's in wide use by backend system engineers if my experience is any guide. Yes, it's quite clunky but also quite powerful and reliable at the few things it does. Its also pretty much guaranteed to already be installed and working on every nix system, and that's not nothing.
Honestly its like any other tool we use: once you know the rules governing its behavior it really isn't that hard to debug issues. Make is very consistent in most cases. There are plenty of traps, and they're made easier to fall into given the archaic syntax, but you don't typically have to fall into them over and over again :).
Actually, you might want to glance at the first few hundred lines of platform specific stuff you are supposed to set manually before the main makefile begins. Literally meant to be set by hand, but the Git developers made educated guesses about the specifics of each platform.
That's how it has always been, though. In the '90s you didn't need to run Perl or Tcl through it because you weren't compiling anything. The Venn diagram of "Platforms that have make" and "Popular compiled languages" comes up with only asm/C/C++.
Many languages want to do things their way, such as Erlang, Common Lisp, Java, etc. Ruby and Python are interpreted and also don't need a build process. JS, until recently, was interpreted. Lots of NIH going around.
> And even those are moving away, see Cmake & co.
Cmake is closer to autoconf/automake/libtool. If you have serious cross-platform needs, then Cmake is a fine tool. But it's hardly less archaic than make (and only slightly less so than autoconf) and I'm dubious that too many people are really moving away rather than just picking up the newer, shiny tool for newer projects.
If I were doing a small static website or something that required a build with standard *ix tools, vanilla make would be my tool of choice, hands down. Tools like autoconf and, as the author pointed out, webpack provide a more specialized need.
> That's how it has always been, though. In the '90s you didn't need to run Perl or Tcl through it because you weren't compiling anything. The Venn diagram of "Platforms that have make" and "Popular compiled languages" comes up with only asm/C/C++.
Pascal/Delphi were wildly popular in the late 80's, early 90's, though. I don't remember it being built with make, though.
Compared to the autotools I'd say make is fairly decent! But yeah, it's a mess. I wonder if it's really possible to improve significantly over the status quo if you're not willing to compromise on flexibility and design the build system alongside the language itself, like Rust does with cargo for instance.
Make on the other hand is completely language agnostic, you can use it to compile C, Java, LaTeX, or make your taxes. Make is a bit like shell scripts, it's great when you have a small project and you just want to build a few C files for instance[1] but when it starts growing there always comes a point where it becomes hell.
[1] And even then if you want to do it right you need compiler support to figure header dependencies out, like GCC's various -M flags.
True, one should not edit makefiles with notepad. Proper editors have support for editing them, though.
> Syntax which relies on bizarre punctuation...
Well documented, though.[1]
> a different dependency-based-programming build tool could replace it... but that's just wishful thinking
Use prolog[2], you should be able to write this in about 42 lines. But you'll end up with the same complaints ("the syntax, the magic variables!") because, in my experience, those are just superficial: The real problem, imho, is that declarative and rule based programming are simply not part of the curriculum, especially not for auto-didactic web developers. OTOH, it only takes an hour or two to grok it, when somebody "who knows" is around and explains. It really is dead simple.
I'm really glad software development has largely moved away from the terse names and symbols that used to be so common. I mean patsubst? What a terrible function name! At the very least I would have added the 'h' in path.
I think that just supports my argument, really. I've never really used makefiles. The blog post talked about using patsubst to change a path. The name of the function essentially gives no clue at all what it does.
Isn't that the problem with using blog posts as educational sources?
(I'm happy I was learning all my toolkit long before blogs became a thing. It's hard to open and read Physics book, when someone on YT talks funny about its second chapter.)
I use it for programming microcontrollers, FPGAs, and doing additional signal testing on those devices. I also use it to control some processes that keep some hobby IoT devices in sync. It's just a really handy way to do parallel jobs like that.
Only really makes sense when you need to generate targets from a series of inputs. The targets don't have to be files, but it helps if they are to preserve the timestamp.
You can look at data reduction tasks as dependency networks. Generate/retrieve data, reduce data, plot data. If the intermediate artifacts are files, you can use make to run the data pipeline.
If part of the data changes, or new data arrives, you just run make again, and the whole pipeline re-does whatever is new. Very slick.
It has free concurrency (make -j N). I've used this effectively several times for medium-scale (~GBs) science data processing.
I use it for almost everything. Take ETL type tasks. It's almost impossible to remember where I downloaded certain data files six months after initially creating a project. I put that into the Makefile. Everything I do on the command line, I put into a Makefile. That way, 6 months later when I need to download new data, do some transformation on that data and load it into the database, 'make' re-traces my steps from 6 months ago. Broadly, my ETL makefiles look like this:
I've built a batch job scheduling system out of 'make' and a handful of supporting tools.
It works surprisingly well, but it is definitely not the right tool for the job. After a number of years of adding/removing/changing jobs, maintaining it is turning into a nightmare.
At one point I had a makefile to build large latex documents. The use case was still essentially compilation, but it got me thinking outside the box with make which was neat.
Anything that gets automated, and can also be shortcutted depending on how much of it has been cached already. That really is the beauty of Make to me - it's very easy to express what parts of a job need and need not be done, because the last run produced a state that was good enough to use again.
I once built a World of Warcraft addon downloader & manager (including patching) with Make. My friends thought I was insane. Personally, I wondered how else you'd do it :-)
I use Makefiles extensively for Docker, all sorts of individual projects (it's much easier to "make serve" than remember the specific invocation for getting a dev server up when you use multiple programming languages) and, of late, for Azure infrastructure deployments:
I started my career with C++ development and never used a direct makefile in any of my projects. When I write C/C++, I use CMake. My current job has me programming in Go where people seem to love makefiles, but I consistently find bugs in the implementation (usually has too many phony targets, etc.), Why don't people use makefile generators outside of the C/C++ community?
I don't use CMake for embedded systems because the syntax and options are even more obtuse for what I need to do with it. When I get down to it, every make-based build system I've ever used boils down to using the macro language to generate all the targets, and then building your output board image by dependencies. But it's the details of how to do this that vary widely depending on your application for the board.
CMake does a great job of compiling executable code and linking it using your compiler of choice, but where it falls flat is in giving me a convenient mechanism for fitting that executable into the system image.
I'd need to know more about the details to answer completely. I think CMake does have a complicated syntax, yet I think it is worth the nuisance in most cases. Many tools can use CMake's compilation database for configuration, such as clang-format and clang-tidy.
Can Yocto or CMake build QNX systems or bare-metal binaries with TI's compiler? The tool I'm looking for is Make because Make gives me a ton of flexibility to build whatever freaky combination of binaries might go into whatever product I'm working on, and then combine those binaries into a system image.
The point I'm trying to make is that every build system except Make solves a really specific class of build problem(building applications, or building Linux systems using GCC) and then pretentiously claims to support everything that matters. What you're actually getting is a small sliver of what you might need in my world.
This article is another example of how Make can do something really unexpected by providing really simple features and letting you decide what matters to you. I've yet to see another build system that is as generally useful.
If you need to write in C or C++, use CMake. Otherwise, use whatever modern build tool your language provides. It's 2018, and Make should have died in the 70s,
The concepts behind make are quite good, but the interface it provides is not decidedly not. It reminds me of Git in that respect. I'd think replacing the opaque symbols used everywhere with more descriptive words would be helpful. I don't suppose anyone knows if modern Make versions support alternatives to $@, $%, $?, etc. that can be read rather than memorized?
A long time ago, in a developer's paradise called the 1970's there was an automated build tool called "make".
It had this and only this syntax:
If a line begins at character 0, that is a list of files whose timestamps should be checked, if any are 'old', then execute the series of command lines beneath, identifying them as starting with a tab character.
That was it, the entire syntax of "make", and it was complete. Then some smart people fucked it up and we have that piece of shit we call "make" now.
the first file in the list is the one whose timestamp is compared against all the other files in the list after it. If the timestamp of any file is newer than that first file, the following lines starting with a tab are executed.
I once (back in the mid-1980's) spent several weeks hunting down a makefile bug. In the end it turned out to be a SPACE character that preceded a TAB character.
Back in that 1970's developer's paradise, it was not possible to lose a space next to a tab, because we did not have WYSIWYG editors, we had "vi" which has an easy toggle to show white space characters. "Back in the day" everything had a command line interface, and non-developers were afraid of computers. Ah, the good old daze...
I think the reason make is both so controversial and also long-lived is that despite how everyone thinks of it, it isn't really a build tool. It actually doesn't know anything at all about how to build C, C++, or any other kind of code. (I know this is obvious to those of us that know make, but I often get the impression that a lot of people think of make as gradle or maven for C, which it really isn't.) It's really a workflow automation tool, and the UX for that is actually pretty close to what you would want. You can pretty trivially just copy tiresome sequences of shell commands that you started out typing manually into a Makefile and automate your workflow really easily without thinking too much. Of course that's what shell scripts are for too, but make has an understanding of file based dependencies that lets you much more naturally express the automated steps in a way that's a lot more efficient to run. A lot of more modern build tools mix up the workflow element with the build element (and in some cases with packaging and distribution as well), and so they are "better than make", but only for a specific language and a specific workflow.
Yeah. There is a metric crap-ton of the design of Make that is solely for the purpose of compiling and linking and document processing. That's actually part of what makes it annoying to use it for projects other than C or C++, when you don't need to compile or transform or depend on different formats.
The core of make is really just a control flow model that understands file dependencies as a first class thing, and permits arbitrary user supplied actions to be specified to update those files. All those default rules around how to handle C files are really more like a standard library and can be easily overridden as desired.
IMHO what makes it annoying for projects other than C or C++ is that there isn't an equivalent portion of makes "standard library" that applies to e.g. java, but this is largely because java went down a different path to develop its build ecosystem.
In an alternate reality java tooling might have been designed to work well with make, and then make would have a substantial builtin knowledge base around how to work with java artifacts as well as having a really nice UX for automating custom workflows, but instead java went down the road of creating monolithic build tooling and for a long time java build tooling really sucked at being extensible for custom workflows.
The thing about Java is that it has its own dependency system embedded in the compiler. This design decision made it difficult to integrate with a tool like make.
I don't think having dependencies built into the language and/or compiler means it needs to be difficult to integrate with something like make. In fact gcc has dependency analysis built into it. It just knows how to output that information in a simple format that make can then consume.
I feel like this choice has more to do with early java culture and/or constraints as compared to unix/linux. With "the unix way" it is really common to solve a problem by writing two separate programs that are loosely coupled by a simple text based file format. When done well, this approach has a lot of the benefits of well done microservices-style applications built today. By contrast, (and probably for a variety of reasons) this approach was always very rare in early java days. It seemed for a while like the norm was to rewrite everything in java and run it all in one giant JVM in order to avoid JVM startup overhead. ;-) The upshot being you often ended up with a lot more monolithic/tightly coupled designs in Java. (I think this is less true about Java today.)
> There is a metric crap-ton of the design of Make that is solely for the purpose of compiling and linking and document processing.
Not really. The bit being pointed out here certainly isn't. It's not any special design going on, it's just a built-in library of rules and variables for C/C++/Pascal/Fortran/Modula-2/Assembler/TeX. These rules are no different than if you had typed them in to the Makefile yourself. And if you don't like them, you can say --no-builtin-rules --no-builtin-variables.
The only actual bit of C-specific design I can think of is .LIBPATTERNS library searching.
Don't know why that would be. I'm using GNU Make 4.1, but this has worked for years and years as far as I knew. Not a particularly useful feature, so I it doesn't really matter, but you messed up my fun fact.
dima@fatty:/tmp$ mkdir dir
dima@fatty:/tmp$ cd dir
dima@fatty:/tmp/dir$ touch foo.c
dima@fatty:/tmp/dir$ make -n foo
cc foo.c -o foo
dima@fatty:/tmp/dir$ make --version
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
> and the UX for that is actually pretty close to what you would want.
That is so not true. Make has deeply woven into it the assumption that the product of workflows are files, and that the way you can tell the state of a file is by its last modification date. That's often true for builds (which is why make works reasonably well for builds), but often not true for other kinds of workflows.
But regardless of that, a tool that makes a semantic distinction between tabs and spaces is NEVER the UX you want unless you're a masochist.
> Make has deeply woven into it the assumption that the product of workflows are files, and that the way you can tell the state of a file is by its last modification date.
I've always wondered whether Make would be seen as less of a grudging necessity, and more of an elegant panacea, if operating systems had gone the route of Plan 9, where everything is—symbolically—a file, even if it's not a file in the sense of "a byte-stream persisted on disk."
Or, to put that another way: have you ever considered writing a FUSE filesystem to expose workflow inputs as readable files, and expect outputs as file creation/write calls—and then just throw Make at that?
It's just representing writing and reading a database as file operations, they map pretty cleanly. Keep in mind that Plan 9 has per process views of the namespace so you don't have to worry about other processes messing up your /mnt/sql.
I think you're missing the point. I have a workflow where I have to perform some action if the result of a join on a DB meets some criterion. How does "make" help in that case?
In that case, it doesn't help much. For Plan 9 mk, there's a way to use a custom condition to decide if the action should be executed:
rule:P check_query.rc: prereq rules
doaction
Where check_query may be a small shell script:
#!/bin/rc
# redirect stdin/stdout to /mnt/sql/ctl
<> /mnt/sql/ctl {
# send the query to the DB
echo query
# print the response, check it for
# your condition.
cat `{sed 1q} | awk '$2 != "condition"{exit(1)}'
}
But I'm not familiar with an alternative using Make. You'd have to do something like:
OK. So here's a scenario: I have a DB table that keeps track of email notifications sent out. There is a column for the address that the email was sent to, and another for the time at which the email was sent. A second table keeps track of replies (e.g. clicks on an embedded link in the email). Feel free to assume additional columns (e.g. unique ids) as needed. When some particular event occurs, I want the following to happen:
1. An email gets sent to a user
2. If there is no reply within a certain time frame, the email gets sent again
3. The above is repeated 3 times. If there is still no reply, a separate notification is sent to an admin account.
That is a common scenario, and trivial to implement as code. Show me how make would help here.
I wouldn't; I don't think that make is a great fit for that kind of long running job. It's a great tool for managing DAGs of dependent, non-interactive, idempotent actions.
You have no DAG, and no actions that can be considered fresh/stale, so there's nothing for make to help with. SQL doesn't have much to do with that.
With respect to Make, does the database (mounted as a filesystem) retain accurate information that Make needs to operate as designed (primarily the timestamps). To what level of granularity is this data present within the database, and what is the performance of the database accessed in this way? Will it tell you that the table was updated at 08:40:33.7777, or will it tell you only that the whole database was altered at a specific time?
You're talking about a theoretical implementation of a filesystem with a back-end in a relational database. The question is only whether the information is available.
Say directories map to databases and files map to tables and views. You can create new tables and views by either writing data or an appropriate query. Views and result files would be read-only while data files would be writable. Writing to a data file would be done with a query which modifies the table and the result could be retrieved by then reading the file -- the modification time would be the time of the last update which is known.
Views and queries could be cached results from the last time they were ran which could be updated/rerun by touching them or they could be dynamic and update whenever a table they reference is updated.
> but often not true for other kinds of workflows.
Examples? I mean, there are some broken tools (EDA toolchains are famous for this) that generate multiple files with a single program run, which make can handle only with subtlety and care.
But actual tasks that make manages are things that are "expensive" and require checkpointing of state in some sense (if the build was cheap, no one would bother with build tooling). And the filesystem, with its monotonic date stamping of modifications, is the way we checkpoint state in almost all cases.
That's an argument that only makes sense when you state it in the abstract as you did. When it comes down to naming a real world tool or problem that has requirements that can't be solved with files, it's a much harder sell (and one not treated by most "make replacements", FWIW).
Anything where the relevant state lives in a database, or is part of a config file, or is an event that doesn't leave a file behind (like sending a notification).
To be serious, those are sort of contrived. "Sending a notification" isn't something you want to be managing as state at all. What you probably mean is that you want to send that notification once, on an "official" build. And that requires storing the fact that the notification was sent and a timestamp somewhere (like, heh, a file).
And as for building into a database... that just seems weird to me. I'd be very curious to hear about systems that have successfully done this. As just a general design point, storing clearly derived data (it's build output from "source" files!) in a database is generally considered bad form. It also introduces the idea of an outside dependency on a build, which is also bad form (the "source" code isn't enough anymore, you need a deployed system out there somewhere also).
I need to send an email every time a log file updates, just the tail, simple make file:
send: foo.log
tail foo.log | email
watch make send
Crap, it keeps sending it. Ok, so you work out some scheme involving temporary files which act as guards against duplicate processing. Or you write a script which conditionally sends the email by storing the hash of the previous transmission and comparing it against the hash of the new one.
That last option actually makes sense and can work well and solves a lot of problems, but you've left Make's features to pull this off. For a full workflow system you'll end up needing something more than files and timestamps to control actions, though Make can work very well to prototype it or if you only care about those timestamps.
================
Another issue with Make is that it's not smart enough to know that intermediate files may change without those changes being important. Consider that I change the comments in foo.c or reformat for some reason. This generates a new foo.o because the foo.c timestamp is updated. Now it wants to rebuild everything that uses foo.o because foo.o is newer than those targets. Problem, foo.o didn't actually change and a check of its hash would reveal that. Make doesn't know about this. So you end up making a trivial change to a source file and could spend the afternoon rebuilding the whole system because your build system doesn't understand that nothing in the binaries are actually changing.
How would you fix that with your preferred make replacement? None of that has anything to do with make, you're trying to solve a stateful problem ("did I send this or not?") without using any state. That just doesn't work. It's not a make thing at all.
Lisper was replying to the OP who suggested using Make for general workflows. Make falls apart when your workflow doesn't naturally involve file modification tasks.
With regard to my last comment (the problem with small changes in a file resulting in full-system recompilation), see Tup. It maintains a database of what's happened. So when foo.c is altered it will regenerate foo.o. But if foo.o is not changed, you can set it up to not do anything else. The database is updated to reflect that the current foo.c maps to the current foo.o, and no tasks depending on foo.o will be executed. Tup also handles the case of multiple outputs from a task. There are probably others that do this, it's the one I found that worked well for my (filesystem-based) workflows.
With regard to general workflows (that involve non-filesystem activities), you have to have a workflow system that registers when events happened and other traits to determine whether or not to reexecute all or part of the workflow.
I love Tup, and have used it in production builds. It is the optimal solution for the problem that it solves, viz, a deterministic file-based build describable with static rules. To start using it, you probably have to "clean up" your existing build.
I don't use it anymore, for several reasons. One is that it would be too off-the-wall for my current work environment. The deeper reason is that it demands a very static view of the world. What I really want is not fast incremental builds, but a live programming environment. We're building custom tooling for that (using tsserver), and it's been very interesting. It's challenging, but one tradeoff is that you don't really care how long a build takes, incremental or otherwise.
I mean you're just describing make but with hashes instead of file modification times. It's probably the most common criticism of make that its database is the filesystem. If file modification times aren't meaningful to your workflow then of course make won't meet your needs. But saying the solution is 'make with a different back-end' seems a little silly, not because it's not useful, but because they're not really that different.
GNU make handles multiple outputs alright but I will admit they if you want something portable it's pretty hairy.
Correct, that works for this example. But if you have a lot of tasks that involve non-filesystem activities you'll end up littering your filesystem with these empty files for every one of them. This can lead to its own problems (fragility, you forgot that `task_x` doesn't generate a file, or it used to generate one but no longer does, etc.).
What about, for example, a source file that needs to be downloaded and diffed from the web? What about when you need to pull stuff from a database? You can hack your way around but it's not the most fun.
curl -z only works if the server has the proper headers set - good luck with that. The point is, it's great to be able to have custom conditions for determining "needs to be updated", for example.
You can always download to a temporary location and only copy to the destination file if there is a difference. You don't need direct support from curl or whatever other tool generates the data.
Sometimes things in workflows are sending/retrieving data over a network. It may be turning on a light. It could be changing a database. Make has no way of recognizing those events unless you've tied them to your file system. Do you really want an extra file for every entry or table in a database? It becomes fragile and error prone. A real workflow system should use a database, and not the filesystem-as-database.
> Sometimes things in workflows are sending/retrieving data over a network. It may be turning on a light. It could be changing a database. Make has no way of recognizing those events
Why should Make violate basic software design rules and fundamental Unix principles? Do you want your build system to tweak lights? Setup a file interface and add it to your makefile. Do you want your build system to receive data through a network? Well, just go get it. Hell, the whole point of REST is to access data as a glorified file.
> Nothing stops you from creating an interface that maps that row to a file.
That's true, nothing stops you, though it is worth noting that no one actually does this, and there's a reason for that. So suppose you did this; how are you going to use that in a makefile?
This seem downvoted, but I would second the opinion. If you're capable of representing a dependency graph, you should be able to handle the tabs. If `make` does your job and the only problem is the tabs, it's not masochism, just pragmatism.
HN has a pretty strong anti-make bias. People here would much rather use build tools that are restricted to specific languages or not available on most systems. Using some obscure hipster build tool means it's a dependency. Though these people who are used to using language-specific package manager seem to take adding dependencies extremely lightly.
A language that uses lots of parens to delimit expressions is incredibly bad UX, especially when you try to balance a complex expression, but hopefully there are tools like Paredit to deal with that, so that I can write my Emacs Lisp with pleasure about every day. Similarly, any decent editor will help you out with using correct indentation with Makefiles.
Last modification date is not always a correct heuristic to use, but it's quite cheap compared to hashing things all the time.
Make is a tool for transforming files. I wonder how it's not quite natural and correct for it to assume it's working with files?
I have used make for years and am very familiar with it. My two major complaints are:
1. The assumptions that it makes. Everything in and out are files (phony files notwithstanding). It is hard and painful if you want outputs to rely on and rebuild from configuration in the makefile itself. It's not impossible to implement, but it's difficult to precisely implement it: often times I've seen systems that just rebuild everything after configuration changes.
2. The mix of declarative and imperative styles, while useful for quickly throwing a build together, gets difficult to deal with as things scale up. The make language itself is pretty restricted, too (without using $(eval)).
I know that recent versions support guile extensions and even C(/C++?) extensions, but at that point it's not giving you all that much. I have often wished the make functionality was exposed in some "libmake" for me to extend.
For this reason (and others), I've recently refactored a huge build system to use shake[1] instead. Now builds are precise and correct, and properly depend on configuration and build environment.
* Parallel execution of build rules comes for free in a lot of implementations. This is really noticeable when you do heavy asset pre-processing.
* Cleanly written build rules are re-usable across projects as long as those projects have the same structure (directory layout).
* Cleanly written build rules provide incremental compilation/assembly for free: You express intermediate steps as targets and those are "cached". I put the "cached" in quotes here, because you essentially define a target file which is regenerated when it's dependencies are updated. Additional benefit: Inspection of intermediate results is easy - they are sitting there as files right in your build's output tree.
524 comments
[ 4.5 ms ] story [ 355 ms ] threadWhat is the best approach to make sure your Makefile will work as expected on every platform, meaning, windows included?
For example: Can we write file paths using forward-slashes, or is it still an issue? I imagine running it via git bash (bash provided by the git installer on windows).
Some systems might not support the `-p` flag in `mkdir -p`, or the `-rf` flags in `rm -rf`. There are scripts distributed on NPM called `mkdirp` and `rimraf` to work around such issues.
One target to build (prod), another to run with fsevents doing auto-rebuild when a file is saved (instant gratification during dev), then a few targets for cleaup & housekeeping. All said, the file is 1/4 the size of any other JS-based build systems.
Regardless of whether the editor can or not, I just don't see how insisting on tab and not space is remotely defensible.
> remotely defensible
Feel free to write your own make superset that supports spaces. It's an irrelevant factor to the usability of make as a whole.
Multiple times.
Actually writing a preprocessor for Make sounds like a pretty good idea.
Vim does handle this. Here is Vim (and I've removed my .vimrc file, so this is not a setting I have set personally), on a Makefile where the third line erroneously uses spaces:
https://i.imgur.com/xMHXeL4.png
> I just don't see how insisting on tab and not space is remotely defensible.
This is like insisting that people haven't been having religious wars over this sort topic for years. You can see the Wiki[1] page on it (which starts "This is one of the eternal Holy Wars") for arguments as to why someone might prefer tabs. I personally like them because they allow the reader of the code to choose the visual layout of the indentation.
Now, while I love tabs, I work in Python all day. But good editors, like Vim, allow you to customize the indentation settings sufficiently to handle both cases like Python (spaces for indent) and Makefile (tabs) gracefully. So "Indent" and "Dedent" just do The Right Thing™ and it mostly never matters.
[1]: http://wiki.c2.com/?TabsVersusSpaces
However, tabs are beautiful, and every occasion to use them should be cherised as precious.
My pet theory is that the more experience you have, the more likely you are to have worked with Python, which is a lot easier to work with if you just always s/\t/ / (replace tabs with spaces).
https://git.musl-libc.org/cgit/musl/tree/Makefile
FWIW, Webpack 4 (which just went final a few days ago) now has "zero-config" defaults out of the box. You may want to give that a shot.
If your makefile fixes up a file using sed and your system has gnu sed, your makefile may fail on a system with BSD sed (e.g., a mac). If you rely on bash-isms, your makefile may not work on a debian system where it will be run with dash instead of bash. And so on.
Also, I use javascript where it makes sense: I rely on Browserify to resolve the web of `require`d files.
If you look in your package.json, you'll surely see a dozen or so "scripts" lines that run through the same shell that make does and have all the problems you just mentioned.
I'd also like to point out that Linux and almost certainly your production environment (because it's most likely *ix) will be case sensitive. Your macOS or Windows file system? Not so much. Point is, you're already up to your neck in portability issues. My macOS coworkers often forget this detail.
huh?? No it's not. That's the whole discussion we are having now. I even specifically mentioned my macOS coworkers who develop on their Mac and are oblivious to case-sensitivity issues. Because "it worked for me."
> If an external package has scripts in it, those scripts have very likely been run by somebody on a mac and somebody on linux and worked in both cases.
Oh, I'd love for that to be true. But also not what I was referring to. Most large projects have a "scripts" section and those are no different than Makefile commands. If you're paranoid about your own Makefiles then you're going to need to be paranoid about your own package.json.
This makefile snippet:
The source and the output are both called index.js? Why, God, WHY???The capitalized rending-of-garments is silly. This is transpiling; file names should remain the same from input to output.
It is not realistic/optimal to try to write your javascript to be supported by all browsers or runtime clients. It's better to write using the latest spec, and just have it dumbed down for you by transpilers at build time
Some more explanation here: https://www.excella.com/insights/typescript-vs-es6-vs-es2015
Of course that sentence contains it's own level of craziness, but the problem does not lie in the build step.
Previous discussion on this: https://news.ycombinator.com/item?id=15154994
How and why? And what do you do of compilers which can do either based simply on the backend you select?
What about assemblers or disassemblers? Given a suitably broad definition of compiler that includes transpilers, are they not also included? Wouldn't the same arguments apply?
I asked why you might want this here and there's been no answer yet. Having a term for something you don't need or want to know isn't actually useful.
> What about assemblers or disassemblers? Given a suitably broad definition of compiler that includes transpilers, are they not also included? Wouldn't the same arguments apply?
Because these are actually useful qualifier, in the same way that "a C compiler" is a useful qualifier.
In this article's case, there a reasonable exception to made when JavaScript is being "compiled" into JavaScript itself. For that, I don't know what the term is (or if there even is one).
Transpiler?
Transpiler has been around as a term in CS for a while now. Initially it was used for compilers that compiled from one dialect of assembly to another.
I don't think it's too much of a stretch to shorten that to "transpiler". I think people hate the term because they associate it with JavaScript hipsters who have no sense of CS history and presume they've invented something new. The same as how you cringe when someone refers to the "#" character as "hashtag".
But "transpiler" is a useful term. We're in a world now where source-to-source compilers are much more prevalent than they were ten years ago, and there are real workflow differences between working with a compiler that targets a low-level language versus a high-level one. Things like how you debug the output are very different.
Like what? When I've had to debug GCC, I just dumped out the GIMPLE et al.; what do you do differently with a 'transpiler'?
With a transpiler, it's fairly common to actually debug using the generated output. That pushes many transpilers to place a premium on readable output that structurally resembles the original source code, sometimes at the expense of performance or code size.
Other times, users will rely on things like source maps in browsers. That gives more flexibility to the transpiler author but can make user's debugging experience weirder because things like single-stepping can get fuzzy when a "step" in their source program may correspond to something very different in the transpiled output.
In compilers to lower-level languages (machine code, bytecode, etc.) there is a strong implication that most users can't debug the output, so the compilation pipeline and tools are obligated to have pretty sophisticated debugging support.
In other words, different tools, priorities, constraints, etc. Having a separate word to help tease these apart seems useful to me.
I remember crappy C toolchains that only gave you a .map file for debugging. It was just table of where the linker put the symbols from the intermediate objects. Hell, there were a lot of really crappy C compilers then that looked way more like babel than gcc. They basically just parsed into an AST and walked it to spit out asm, hardly doing any optimizations.
Well, given
trans : (+ acc.) across.
and
ipse ipsa ipsum : himself, herself, itself.
I propose we call that ipsumpilation ;)
More seriously though if the input is js and the output is js, and it’s not different versions of the language — e.g. es6 -> es5 (because imo that qualifies as different languages) — I’d call it an optimizer or a packer depending on what it does.
Compiler: collects sources from various places, assembles the result into machine code.
Transpiler: collects sources from various places, converts the source to another language.
Personally I cringe at the newspeak you seem to imply we should all be applying to our lives. These words mean things - quite different things, it turns out. There's a difference between human-readable source code language, and machine-executable binary code. These tools function in different ways entirely; your optimization to the language is not only un-warranted, but leads to a desultory effect: programmers get stupider when they don't know what their tools are actually doing.
http://foldoc.org/transpiler
^ no match.
transpiler is the newspeak.
However, this is a fun game, so lets play:
http://www.yourdictionary.com/transpiler
https://en.wiktionary.org/wiki/transpiler
https://www.thefreedictionary.com/transpiler
https://www.quora.com/What-is-a-transpiler
I doubt you'll find 'transpiler' at the origins of computing history; see also C++/cfront post
https://en.wikipedia.org/wiki/Cfront
http://www.softwarepreservation.org/projects/c_plus_plus/cfr...
ohey, noone calls it a 'transpiler'.
I'm not so angry about the word transpiler; perhaps it might improve things.. but to imply that using compiler as a common term for all things here is a regression/neologism is patently wrong - transpiler is the newcomer.
gcc compiling some C and Babel compiling some JavaScript are both performing the same role. So much so the similarities in how they carry it out are striking.
https://babeljs.io
Babel is a JavaScript compiler.
Well, I know they are not magic ;). But why would I want them when I can actually use normal names like "deps"/"entries" and "target"?
It gets substantially worse as we go down the rabbit whole. Where webpack can easily walk the entire dependency tree by itself, we have to invoke
Where we can use same webpack to seamlessly output the resulting file into an output directory, we need to do the (very unintuitive) pattern substitution: or even And when we want to watch for changes? Well, we need an external program anyway. The art of Makefiles is often lost for good reasons: because Makefile don't really cut it anymore.https://github.com/webpack/webpack-cli/issues/152
I am sure Make still has it's uses, but I am sure glad package managers of have made makefiles irrelevant to me.
I'd argue it's entirely fair with other examples (e.g. compare and contrast to, say, Rust's cargo build.) You can write a Makefile which fetches and builds your dependencies. Makefiles might be like a recipe - but these days we're building recipes for building entire OS images, including grabbing the dependencies in the first place. A single software project by comparison is trivial.
The problem is Makefiles are a jack of all trades, and master of none. You have to write or copy a distressing number of rules yourself, per project, per subproject, and know the underlying build chain in relative depth to do even some rather basic things. I'd argue makefile authors not automating their dependency fetching is a symptom of this.
As a programmer, I avoid them, because I'll end up stuck maintaining them and end up lowering the build system's bus factor. With perhaps the exception of a couple of simple rules that forward to a "proper" build system, because I've written enough of them in the past that "make" still feels like the right default action that should generally work. But if it all possible, never for the meat of the build system.
I definitely think that we can do better than Makefiles, but again they have the benefit to be extremely versatile so they can be bent to many uses. (personally I do not use them, I use CMake and generate ninja files)
Good makefiles are simple, and should Just Work™. The dependencies should (theoretically) be listed in the project documentation, though it often isn't.
Package managers are definitely better for non-developers, though. Build systems generally aren't geared toward regular users, so users will find them confusing.
What would you gain if you're using it for other, less supported languages, like Javascript?
I've only ever used it for C++, but I don't see why it (or something like it) couldn't be used for other languages too.
This tutorial was making the rounds (on HN and elsewhere) last week, but in case anyone missed it, here's a link: https://github.com/pyk/cmake-tutorial I found it to be a great primer/refresher.
https://en.wikipedia.org/wiki/Punched_card#IBM_80-column_pun...
CMake is a nightmare of implied complexity. I've run into multiple situations (usually but not always involving cross-compilation) where it was simply incomprehensible. No amount of time invested would let me figure out why CMake blew up. There is no way I can build something a little out of the ordinary on top of CMake and be confident that it's solid. It's just too complicated.
This sort of balancing, upfront ease of use vs hidden complexity comes up often and I have been burned enough in the past that experience dictates to pretty much always go for model simplicity and pay the upfront costs. I don't see this often however. A lot of the time people will go for what feels or looks right, superficially, without bothering to look beneath the surface or think about model complexity costs. It's an attitude that has led to many disasters in this space.
I'd much rather a Makefile where I can override the compiler and flags for my configuration. I can easily figure out what I need to do based on compiler and linker errors for missing headers and libraries.
Another member of the team subsequently taught make about the reverse dependencies so that you can split jobs across Travis nodes by the top-ish level image they depend on.
My favorite addition was the ability to generate a dependency graph using graphviz straight from the Dockerfiles.
N.B. Project is now moribund, the team was disbanded. May not build at all. Don't know if any of the forks are active.
I regret not considering Make for Docker administration months/years ago. I've taken to using bash scripts or, worse, tagged bash comments to recall commonly used complex commands.
- I do major.minor.yy-mm-dd versioning, so this handles that automatically
- shell is for dropping you into a prompt with your present directory mapped to the working directory. I find this super helpful for debugging and testing containers
- run is for testing your CMD and entrypoint scripts
Thanks for sharing! I'll be using this as inspiration in the future.
I'm of an opinion that Makefiles with Docker are mostly a fancy way to write `case "$1" in build) ... run) ... esac` except that one needs to have make(1) installed to run them (in my experience, Bourne shell is much more commonly available than any make variant)
I started doing makefiles as a reaction to having a bunch of bash scripts cluttering up my directory. More or less just a collection of useful commands. But as I continue to do this, I find it encourages making consistent build routines/versioning. Also, being able to set up dependencies ( like "run relies on build") can also be helpful. Finally, variable substitution is useful for versioning.
I think what makes makefiles great are you can do as much or as little as you want with them. I put an example of a common one in another reply.
We're stuck with Make because of network effects. I wish that it could just become "lost" forever and a different dependency-based-programming build tool could replace it... but that's just wishful thinking. The pace of our progress is doomed to be held back by the legacy of those poor design decisions for a long time to come.
Make, in contrast, works within a shell and invokes programs which may be either wildly or subtly incompatible across platforms. Add in the lacking support for conditionals in POSIX make and portability is a nightmare.
(I am aware that this advice does not hold true in all cases; just that for many of them overly complicated build systems is a code smell.)
If you are invoking your functions (or functions imported from random library), I have to check them for what they do.
Docker is new and didn’t have any network effect until recently.
EDIT: Oh, and parallel execution, but smart parallel execution. Independent tasks can be allowed to run simultaneously. Very useful when you have a lot of IO bound tasks. Like in compilation, or if you set it up to retrieve or transmit data over a network.
It's not too hard to do that in your custom script, but more care is required because until you custom script reaches make level internal complexity you will have to manually track dependencies or make sub-optimal assumptions.
If for some reason the project is ruby allergic, I'll try to use Invoke [0].
Sometimes I feel like people's usage of Make in web projects is akin to someone taking an axe and hand saw out to break down a tree for firewood when there are multiple perfectly functioning chainsaws in the garage.
[0] http://www.pyinvoke.org/
I use it for anything where I need job scheduling around creating output files; even my DVD ripping is managed with redo calling into ffmpeg.
When I do, there are two cases:
1) All dependencies are on only one output file (e.g. link stages that generate .map files, compile phases that generate separate .o and debug files). I just treat these as normal
2) I may need to depend on each of the files (I don't use yacc/bison, but it sounds like this would qualify). I select one file to be the main output and have the .do file for that ensure that the secondary outputs go to a mangled name; I then have the .do files for the secondary outputs rename the mangled file.
quick example for generating foo.c/foo.h
foo.c.do:
foo.h.do:I once solved it by making a tar file or all the outputs and extracting it to both .c and .h but that’s incredibly kludgy, still looking for a better solution
https://cr.yp.to/redo.html
Incremental builds by looking for changed dependencies, a configuration file with its own significant identifiers (i.e. a build DSL shoehorned into JSON or YAML), generalized target rules, shelling out commands, sub-project builds, dry runs, dependencies for your own script, parallelization, and a unique tool with (making an generalization here) insufficient documentation.
If you're really unlucky, you'll even end up with the equivalent of a configure.sh to transpile one DSL and run environment into the DSL for your custom tool.
I'd argue that make is a half-baked and buggy implementation of make - so that's not really a drawback so much as the status quo.
E.g. I have scripts that exist mainly to carefully select the "correct" version of make for a given project to deal with path normalization and bintools selection issues on windows - and none of these ~3 versions of make work on all our Makefiles. One of those versions appears to have some kind of IO bug - it'll invoke commands with random characters missing for sufficiently large Makefiles, which I've already gone over with a hex editor to make sure there wasn't some weird control characters or invisible whitespace that were to blame. So, buggy and brittle.
make was never intended to be a cross-platform tool. If you face problems using it on Windows, then that's on you.
I'm currently doing this for a cross platform c++ project. Same makefile used for Linux, Mac, and windows+msys+cl.exe. (yes, fair amount of extra variables and ifdefs to support the last one...)
That said, Windows has never been a strong platform on which to run make (or git, gcc, or any other [u/li]nix originated CLI tools). When I hear of folks using make, I tend to make the assumption that they're running on a [u/li]nix or BSD derivative.
I've already had one upstream patch rejected on account of the additional complexity fixing it introduces, and would rather not indefinitely support my own fork of other people's build setups.
Or if I am going to indefinitely support my own fork, I might as well rewrite the build config to properly integrate with the rest of whatever build system I happen to be using - at least then I'll get unified build/compilation options etc.
And no, I don't respect the community. The community very often makes "The Way Things Are Done" its personal religion and refuses to ever change anything for the better.
Plus, to me C syntax is particularly good. You're writing real words and the computers does the things you tell it to. To the letter.
Because it's obvious people need a minimalistic portable assembler.
History is re-written by winners as usual.
But what were those other OSs and languages? Sounds interesting!
https://en.wikipedia.org/wiki/System_programming_language
https://en.wikipedia.org/wiki/Category:Systems_programming_l...
Some examples, out of my head.
- Burroughs, now being sold as Unisys ClearCase, used ESPOL, later replaced by NEWP, which already used the concept of UNSAFE code blocks;
- IBM used PL/8 for their RISC research, before switching to C, when they decide to go commercial selling RISC hardware for UNIX workstations
- VAX/VMS used Bliss
- Xerox PARC started their research in BCPL, eventually moved to Mesa (later upgraded to Mesa/Cedar), these languages are the inspiration for Wirth's Modula-2 and Oberon languages
- OS/400, nowadays known as IBM i, was developed in PL/S. New code started to be replaced by C++. It was probably the first OS to use a kernel level JIT with a portable bytecode for its executables.
BitSavers and Archive web sites are full of scanned papers and manuals from these and other systems.
Is it though? It's definitely available on a huge number of platforms, but are the implementations compatible?
And even if they are, there is so much undefined behavior that taking advantage of the cross-platform nature is not nearly as easy as it should be.
Network effect. If you wanted to write for unix, you almost had to use C originally.
> Plus, to me C syntax is particularly good.
It's meh. It's not the most straightforward when defining complex types like arrays or function pointers.
> You're writing real words and the computers does the things you tell it to. To the letter.
So yeah, about that...
IMHO, that's a self-fulfilling prophecy. If it were reasonably easy to do that in C, it would be done more.
Also, functional programming and returning functions does not mean they are created at runtime.
And "creating functions" means: closing over variables ("closures"), or partial application. I think it takes at least that to be able to return interesting functions.
(and whether go is a systems programming language is at least debatable. I think the creators have distanced themselves from that. It depends on your definition of "systems". You can't really write an OS in go).
OK.
> That is my point.
Not unless your first two sentences have absolutely nothing to do with each other. Your point appears to be that because it's a low-level language it doesn't have these features, which is false.
> I happen to like that fact, but you don't have to agree with me.
Or I just think you don't have any experince using better languages. That isn't to say that other language could supplant C, but just that it's difficult for me to image actually liking the C type system (or lack thereof) and lack of first class functions. It's incredibly limiting and requires a lot of hoops to be jumped through to do anything interesting.
> And "creating functions" means: closing over variables ("closures"), or partial application.
Well, you said at runtime. Of course you can "create" functions at compile or programming time!
I would think that first class closures do indeed _not_ belong in a low-level language. They hide complexity, and you want to avoid that in low-level programming. Not necessarily for performance reasons, but more from a standpoint of clarity (which in turn can critically affect performance, but in subtler ways).
> Or I just think you don't have any experince using better languages.
Nah, I have experience in many other languages, including Python, C++11, Java, Haskell, Javascript, Postscript. The self-containment, control, robustness and clarity I get from a cleanly designed C architecture is just a lot more appealing to me. The only other language I can stand is Python, but for complex things, it becomes actually more work. For example, because it's so goddamn hard to just copy data as values in most languages (thanks to crazy object graphs).
> It's incredibly limiting and requires a lot of hoops to be jumped through to do anything interesting.
It depends on what you are doing. It's a bad match for domains where you have to fight with short-lived objects and do a lot of uncontrolled allocations and string conversions. My experience in other domains (including some types of enterprise software) is more the opposite, though. Most software projects written in more advanced languages are so damn complicated, but do nothing impressive at all. They are mostly busy fighting the complexity that comes from using the many features of the language. But those features help only a bit (in the small), and when you scale up they come back and bite you!
Here's a nice video from a guy who gets shit done, if you are interested: https://www.youtube.com/watch?v=khmFGThc5TI
> Well, you said at runtime. Of course you can "create" functions at compile or programming time!
Closures and partial application are done at runtime. The values bound are dynamic. So in that sense, the functions are indeed created at runtime. I sense that you were of the impression that a closure would actually have the argument "baked" in at compile time (resulting in a different code, at a low level) instead of the argument being applied at runtime. That's not the case, unless optimizations are possible. If that was really your impression, this makes my point regarding avoiding complexity and that closures do not belong in a low-level language. (Look up closure conversion / lambda lifting)
C became popular because Unix became popular, and when designing a language that intends to become popular one aims for a ratio of 10% novelty and 90% familiarity. Like, ever wonder why Javascript's date format numbers the months starting from zero and the days starting from one? It's because Eich was told to make JS as much like Java as he could, and java.util.Date numbers the months from zero and the days from one, which Java itself got from C's time.h. (Not coincidentally, Java and JS are both in the C-like language family.)
> You're writing real words
In C? Not compared to ALGOL, COBOL, Pascal, and Ada, you're not. :)
> the computers does the things you tell it to. To the letter.
As long as you're not using a modern compiler, whose optimizations will gleefully translate your code into whatever operations it pleases. And even if one were to bypass C and write assembly code manually, that still doesn't give you complete control over modern CPUs, who are free to do all sorts of opaque silliness in the background for the sake of performance.
But even without accounting for compiler optimizations and cpu architecture, the C language just straight up lies to its user. You could code something entirely with void*.
PS: what I meant by "real words" is that you're naming functions and calling them by their name. Which in itself is very powerful.
To cut to the chase:
* syntax = structure
* semantics = meaning
C syntax would include things like curly braces and semicolons, whereas C semantics would include things like the functionality of the reserved keywords.
This SO answer gives a more detailed explanation:
https://stackoverflow.com/a/17931183/1863924
$@ is target, because @ looks sort of like a bullseye.
significant tabs are gross, yes, but this is a one-time edit to your vimrc then you can forget about it forever.
it’s also installed everywhere and has minimal dependencies. it could be a lot worse. (see also: m4, autoconf, sendmail.cf)
In all seriousness, what's wrong with it? Significant tabs aren't great, but I feel like that's a relatively minor wart. The simple things are _very_ simple and straightforward. The more complex things are more complex, but usually still manageable...
I've seen plenty of unmanageable Makefiles, but I haven't seen another system that would make them inherently cleaner. (I love CMake, but it's a beast, and even harder to debug than make. If it weren't for its nice cross-platform capabilities, I'm not sure it would see much use. It's also too specialized for a generic build tool. Then again, I definitely prefer it to raw Makefiles for a large C++ project.)
$< $> $* $^ ... Not particularly explicit. You also have the very useful substitution rules, like $(SRC:.c=.o) which are probably more arcane than they ought to be. You can make similar complaints about POSIX shell syntax but at least the shell has the excuse of being used interactively so it makes sense to save on the typing I suppose.
That's my major qualm with it however, the rest of the syntax is mostly straightforward in my opinion, at least for basic Makefiles.
manual: ('make' on a bsd is PMake)
https://www.freebsd.org/cgi/man.cgi?query=make&apropos=0&sek...
but most linux flavors have a package somewhere..
I use
instead, which I find easier to remember.1. Claiming a rule makes a target, but then fails to make that target, ought to be a runtime fatal error in the makefile. I can hardly even guess at how much time this one change alone would have saved people.
2. String concatenation as the fundamental composition method is a cute hack for the 1970s... no sarcasm, it really is... but there's better known ways to make "templates" nowadays. It's hard to debug template-based code, it's hard to build a non-trivial system without templates.
3. Debugging makefiles is made much more difficult than necessary by make's default expansion of every target to about 30 different extensions for specific C-based tools (many of which nobody uses anymore), so make -d output is really hard to use. Technically once you learn to read the output it tends to have all the details you need to figure out what's going wrong, but it is simply buried in piles of files that have never and will never be found in my project.
4. The distinction between runtime variables and template-time variables is really difficult and annoying.
5. I have read the description of what INTERMEDIATE does at least a dozen times and I still don't really get it. I'm pretty sure it's basically a hack on the fact the underlying model isn't rich enough to do what people want.
6. Sort of related to 2, but the only datatype being strings makes a lot of things harder than it needs to be.
7. Make really needs a debugger so I can step through the build, see the final expansions of templates and commands, etc. It's a great example of a place where printf debugging can be very difficult to make work, but it's your only choice.
That said, I'd sort of like "a fixed-up make" myself, but there's an effect I wish I had a name for where new techs that are merely improvements on an old one almost never succeed, as if they are overshadowed by the original. Make++ is probably impossible to get anybody to buy in to, so if you don't want make you pretty much have to make something substantially different just to get people to look at you at all.
Also, obviously, many of the preceding comments still apply to a lot of other build tools, too.
As soon as more JavaScript developers start writing very very simple Makefiles, tooling will improve and maybe someone will come up with a better make.
The other option is to let people keep using webpack and gulp until they come up with another JS-based build-system, webpack 5 or 6, and grulpt or whatever comes after grunt/gulp.
It took many reads, but I think I get it.
As a toy example, consider the dependency chain:
Depending on how we wrote the rules, foo.o may automatically be considered "intermediate". If we didn't write the rules in a way that foo.o is automatically considered intermediate, we may explicitly mark it as intermediate by writing: So, what does "foo.o" being "intermediate" mean? 2 things:1. Make will automatically delete "foo.o" after "foo" has been built.
2. "foo.o" doesn't need to exist for "foo" to be considered up-to-date. If "foo" exists and is newer than "foo.c", and "foo.o" doesn't exist, "foo" is considered up-to-date (if "foo" was built by make, then when it was built, "foo.o" must have existed at the time, and must have been up-to-date with foo.c at the time). This is mostly a hack so that property #1 does not break incremental builds.
These seem like useful properties if disk space is at a premium, but is something that I have never wanted Make to do. Rather than characterizing it as "a hack on the fact the underlying model isn't rich enough", I'd characterize it as "a space-saving hack from a time when drives were smaller".
----
If, like me, you don't like this, and want to disable it even on files that Make automatically decides are intermediate, you can write:
Which tells it 2 things:a. never apply property #1; never automatically delete intermediate files
b. always apply property #2; always let us hop over missing elements in the dependency tree
I write .SECONDARY: for #a, and don't so much care for #b. But, because #1 never triggers, #b/#2 shoudn't ever come up in practice.
I'm a big fan of Make, but appreciated your detailed criticism, and found myself nodding in agreement.
#3: I like to stick MAKEFLAGS += --no-builtin-rules in my Makefiles, for this reason. This, of course, has the downside that I can't take advantage of any of the builtin rules.
#7: There is a 3rd-party GNU Make debugger called Remake https://bashdb.sourceforge.net/remake/ https://sourceforge.net/projects/bashdb/files/remake/ It comes recommended by Paul Smith, the maintainer of GNU Make.
Not to push the particular product, but the approach:
FAKE (https://fake.build/), is an F# "Make" system that takes a fundamentally different tack to handling the complexity. Instead of having a new, restricted, purpose built language they've implemented a build DSL in F# scripts.
That yields build scripts that are strongly typed, just-in-time compiled, have full access to the .Net ecosystem and all your own code libraries, and are implemented in a first class functional language. That is to say: you can bring the full force of programming and abstraction to handle arbitrarily complex situations using the same competencies that one uses for coding.
As the build file grows in complexity and scope it can be refactored, and use functionality integrated into your infrastructure code, in the same way programs are refactored and improved to make them manageable. The result is something highly accessible, supportive, and aggressively positioned for modern build chains... If you can do it in .Net, you can do it in FAKE.
https://9fans.github.io/plan9port/
http://www.vitanuova.com/inferno/papers/mk.html
Make was created by Stuart Feldman at Bell Labs in 1976. The fact that it is still in use in any form and still being discussed here is a testament to what an amazingly good job he did at the time. Whether it is the right tool for any given modern use case is up to the people who decide to use it or pass it by. I still work with almost daily, and it's in wide use by backend system engineers if my experience is any guide. Yes, it's quite clunky but also quite powerful and reliable at the few things it does. Its also pretty much guaranteed to already be installed and working on every nix system, and that's not nothing.
You know what you have to do to build git? Type make. It's amazing.
Not necessarily.
> It’s also pretty much guaranteed to already be installed and working on every nix system, and that's not nothing.
First mover advantage.
The fact that no modern language, basically nothing outside of C/C++ uses it, says a lot. And even those are moving away, see Cmake & co.
That's how it has always been, though. In the '90s you didn't need to run Perl or Tcl through it because you weren't compiling anything. The Venn diagram of "Platforms that have make" and "Popular compiled languages" comes up with only asm/C/C++.
Many languages want to do things their way, such as Erlang, Common Lisp, Java, etc. Ruby and Python are interpreted and also don't need a build process. JS, until recently, was interpreted. Lots of NIH going around.
> And even those are moving away, see Cmake & co.
Cmake is closer to autoconf/automake/libtool. If you have serious cross-platform needs, then Cmake is a fine tool. But it's hardly less archaic than make (and only slightly less so than autoconf) and I'm dubious that too many people are really moving away rather than just picking up the newer, shiny tool for newer projects.
If I were doing a small static website or something that required a build with standard *ix tools, vanilla make would be my tool of choice, hands down. Tools like autoconf and, as the author pointed out, webpack provide a more specialized need.
Pascal/Delphi were wildly popular in the late 80's, early 90's, though. I don't remember it being built with make, though.
Make on the other hand is completely language agnostic, you can use it to compile C, Java, LaTeX, or make your taxes. Make is a bit like shell scripts, it's great when you have a small project and you just want to build a few C files for instance[1] but when it starts growing there always comes a point where it becomes hell.
[1] And even then if you want to do it right you need compiler support to figure header dependencies out, like GCC's various -M flags.
True, one should not edit makefiles with notepad. Proper editors have support for editing them, though.
> Syntax which relies on bizarre punctuation...
Well documented, though.[1]
> a different dependency-based-programming build tool could replace it... but that's just wishful thinking
Use prolog[2], you should be able to write this in about 42 lines. But you'll end up with the same complaints ("the syntax, the magic variables!") because, in my experience, those are just superficial: The real problem, imho, is that declarative and rule based programming are simply not part of the curriculum, especially not for auto-didactic web developers. OTOH, it only takes an hour or two to grok it, when somebody "who knows" is around and explains. It really is dead simple.
[1] http://pubs.opengroup.org/onlinepubs/009695399/utilities/mak...
[2] https://en.wikipedia.org/wiki/Prolog
The "pat" in "patsubst" means pattern, not path, so adding an h would be incorrect. (https://www.gnu.org/software/make/manual/html_node/Text-Func...)
Isn't that the problem with using blog posts as educational sources?
(I'm happy I was learning all my toolkit long before blogs became a thing. It's hard to open and read Physics book, when someone on YT talks funny about its second chapter.)
.RECIPEPREFIX option has been available for 7 years. Stop whining about non-issues.
Make one that's generic and provides significant enough improvements that people care.
If part of the data changes, or new data arrives, you just run make again, and the whole pipeline re-does whatever is new. Very slick.
It has free concurrency (make -j N). I've used this effectively several times for medium-scale (~GBs) science data processing.
all: data.loaded data2.loaded
data.csv: wget 'some url'
%.sql: %.csv sed/awk/custom-python-script $< > $@
%.loaded: %.sql psql < $< && touch $@
It works surprisingly well, but it is definitely not the right tool for the job. After a number of years of adding/removing/changing jobs, maintaining it is turning into a nightmare.
I once built a World of Warcraft addon downloader & manager (including patching) with Make. My friends thought I was insane. Personally, I wondered how else you'd do it :-)
(actually, looking back at it, I apparently built it on plan9's mk (https://9fans.github.io/plan9port/man/man1/mk.html), but it's close enough)
- https://github.com/rcarmo/azure-docker-swarm-cluster/blob/ma...
- https://github.com/rcarmo/azure-acme-foundation/blob/master/...
one tip -- remember to use .PHONY on your targets -- https://www.gnu.org/software/make/manual/html_node/Phony-Tar...
CMake does a great job of compiling executable code and linking it using your compiler of choice, but where it falls flat is in giving me a convenient mechanism for fitting that executable into the system image.
What exactly are you looking for with regards to deployment, and what tool do use, instead of CMake, to give you that ability?
The point I'm trying to make is that every build system except Make solves a really specific class of build problem(building applications, or building Linux systems using GCC) and then pretentiously claims to support everything that matters. What you're actually getting is a small sliver of what you might need in my world.
This article is another example of how Make can do something really unexpected by providing really simple features and letting you decide what matters to you. I've yet to see another build system that is as generally useful.
https://github.com/Kitware/CMake/blob/master/Modules/Platfor... https://github.com/Kitware/CMake/blob/master/Modules/Compile...
Use on the Makefile:
And invokeAlso you probably meant ?= instead of := as the command you gave would still use /some/path instead of /other/path if you use :=.
It had this and only this syntax:
If a line begins at character 0, that is a list of files whose timestamps should be checked, if any are 'old', then execute the series of command lines beneath, identifying them as starting with a tab character.
That was it, the entire syntax of "make", and it was complete. Then some smart people fucked it up and we have that piece of shit we call "make" now.
The list of files would be the declared dependencies.
not sure which make you are criticizing here..
BSD Make (aka PMake) is imho light years beyond GnuMake in terms of coherence of the extensions to the base language and usability..
unfortunately most people in linux land think make==gnumake, and so it is hardly used outside of the base build system of BSD systems..
This does seem to happen in our industry over and over, another example is the way we turn every small, well designed language into Java.
Then some other people fucked it up...
I still remember how Java was hailed as a perfect replacement for C++ as the language of choice when writing applications - it truly was.
Today though I was faced with a class name that was 46 characters long. There's a reason why "Fizz Buzz Enterprise Edition" is written in Java.
It's some weeks of my life I will never get back.
I hate make and it's 'entire syntax'.
https://github.com/nzoschke/gofaas/blob/master/Makefile
It started very verbose, one target for every Go program, until I figured out target patterns.
I’m also enjoying the -j flag to do things in parallel.
Now it’s 3 lines of Make to build 10 go programs in parallel in seconds.
Parallel is also enough job control to run the development server and to watchexec rebuilding all go programs on code change.
I guess it depends on how you define "know", but there are implicit rules.
IMHO what makes it annoying for projects other than C or C++ is that there isn't an equivalent portion of makes "standard library" that applies to e.g. java, but this is largely because java went down a different path to develop its build ecosystem.
In an alternate reality java tooling might have been designed to work well with make, and then make would have a substantial builtin knowledge base around how to work with java artifacts as well as having a really nice UX for automating custom workflows, but instead java went down the road of creating monolithic build tooling and for a long time java build tooling really sucked at being extensible for custom workflows.
I feel like this choice has more to do with early java culture and/or constraints as compared to unix/linux. With "the unix way" it is really common to solve a problem by writing two separate programs that are loosely coupled by a simple text based file format. When done well, this approach has a lot of the benefits of well done microservices-style applications built today. By contrast, (and probably for a variety of reasons) this approach was always very rare in early java days. It seemed for a while like the norm was to rewrite everything in java and run it all in one giant JVM in order to avoid JVM startup overhead. ;-) The upshot being you often ended up with a lot more monolithic/tightly coupled designs in Java. (I think this is less true about Java today.)
Not really. The bit being pointed out here certainly isn't. It's not any special design going on, it's just a built-in library of rules and variables for C/C++/Pascal/Fortran/Modula-2/Assembler/TeX. These rules are no different than if you had typed them in to the Makefile yourself. And if you don't like them, you can say --no-builtin-rules --no-builtin-variables.
The only actual bit of C-specific design I can think of is .LIBPATTERNS library searching.
https://asciinema.org/a/zVu7sYyh7lQZTNAgAsbKUmocr
That's true.
> and the UX for that is actually pretty close to what you would want.
That is so not true. Make has deeply woven into it the assumption that the product of workflows are files, and that the way you can tell the state of a file is by its last modification date. That's often true for builds (which is why make works reasonably well for builds), but often not true for other kinds of workflows.
But regardless of that, a tool that makes a semantic distinction between tabs and spaces is NEVER the UX you want unless you're a masochist.
I've always wondered whether Make would be seen as less of a grudging necessity, and more of an elegant panacea, if operating systems had gone the route of Plan 9, where everything is—symbolically—a file, even if it's not a file in the sense of "a byte-stream persisted on disk."
Or, to put that another way: have you ever considered writing a FUSE filesystem to expose workflow inputs as readable files, and expect outputs as file creation/write calls—and then just throw Make at that?
How are you going to make the result of a join in a relational database into a file, symbolically or otherwise?
With that said, there are NoSQL databases these days whose query language is easily expressed as file paths. CouchDB, for example.
A file that represents the temporary table that has been created. Naming it is harder, unless the SQL query writer was feeling nice and verbose.
That's exactly right, but notice that you are not actually using make at all any more at this point, except as a vehicle to run
check_query.sh && doaction
which is doing all work.
1. An email gets sent to a user
2. If there is no reply within a certain time frame, the email gets sent again
3. The above is repeated 3 times. If there is still no reply, a separate notification is sent to an admin account.
That is a common scenario, and trivial to implement as code. Show me how make would help here.
You have no DAG, and no actions that can be considered fresh/stale, so there's nothing for make to help with. SQL doesn't have much to do with that.
Say directories map to databases and files map to tables and views. You can create new tables and views by either writing data or an appropriate query. Views and result files would be read-only while data files would be writable. Writing to a data file would be done with a query which modifies the table and the result could be retrieved by then reading the file -- the modification time would be the time of the last update which is known.
Views and queries could be cached results from the last time they were ran which could be updated/rerun by touching them or they could be dynamic and update whenever a table they reference is updated.
Examples? I mean, there are some broken tools (EDA toolchains are famous for this) that generate multiple files with a single program run, which make can handle only with subtlety and care.
But actual tasks that make manages are things that are "expensive" and require checkpointing of state in some sense (if the build was cheap, no one would bother with build tooling). And the filesystem, with its monotonic date stamping of modifications, is the way we checkpoint state in almost all cases.
That's an argument that only makes sense when you state it in the abstract as you did. When it comes down to naming a real world tool or problem that has requirements that can't be solved with files, it's a much harder sell (and one not treated by most "make replacements", FWIW).
Anything where the relevant state lives in a database, or is part of a config file, or is an event that doesn't leave a file behind (like sending a notification).
To be serious, those are sort of contrived. "Sending a notification" isn't something you want to be managing as state at all. What you probably mean is that you want to send that notification once, on an "official" build. And that requires storing the fact that the notification was sent and a timestamp somewhere (like, heh, a file).
And as for building into a database... that just seems weird to me. I'd be very curious to hear about systems that have successfully done this. As just a general design point, storing clearly derived data (it's build output from "source" files!) in a database is generally considered bad form. It also introduces the idea of an outside dependency on a build, which is also bad form (the "source" code isn't enough anymore, you need a deployed system out there somewhere also).
That last option actually makes sense and can work well and solves a lot of problems, but you've left Make's features to pull this off. For a full workflow system you'll end up needing something more than files and timestamps to control actions, though Make can work very well to prototype it or if you only care about those timestamps.
================
Another issue with Make is that it's not smart enough to know that intermediate files may change without those changes being important. Consider that I change the comments in foo.c or reformat for some reason. This generates a new foo.o because the foo.c timestamp is updated. Now it wants to rebuild everything that uses foo.o because foo.o is newer than those targets. Problem, foo.o didn't actually change and a check of its hash would reveal that. Make doesn't know about this. So you end up making a trivial change to a source file and could spend the afternoon rebuilding the whole system because your build system doesn't understand that nothing in the binaries are actually changing.
With regard to my last comment (the problem with small changes in a file resulting in full-system recompilation), see Tup. It maintains a database of what's happened. So when foo.c is altered it will regenerate foo.o. But if foo.o is not changed, you can set it up to not do anything else. The database is updated to reflect that the current foo.c maps to the current foo.o, and no tasks depending on foo.o will be executed. Tup also handles the case of multiple outputs from a task. There are probably others that do this, it's the one I found that worked well for my (filesystem-based) workflows.
With regard to general workflows (that involve non-filesystem activities), you have to have a workflow system that registers when events happened and other traits to determine whether or not to reexecute all or part of the workflow.
I don't use it anymore, for several reasons. One is that it would be too off-the-wall for my current work environment. The deeper reason is that it demands a very static view of the world. What I really want is not fast incremental builds, but a live programming environment. We're building custom tooling for that (using tsserver), and it's been very interesting. It's challenging, but one tradeoff is that you don't really care how long a build takes, incremental or otherwise.
GNU make handles multiple outputs alright but I will admit they if you want something portable it's pretty hairy.
These files are information just like files that are not empty.
How about "touch send"?
Now "touch -t" will allow you to control the timestamp.
md5sum, diff would be your friends.
Anyway, my C compiler doesn't provide that info, anyway.
DB are harder (yet possible) but not a common request that I’ve seen.
You're referring to a standard Unix tool, an operating system where EVERYTHING is a file.
Why should Make violate basic software design rules and fundamental Unix principles? Do you want your build system to tweak lights? Setup a file interface and add it to your makefile. Do you want your build system to receive data through a network? Well, just go get it. Hell, the whole point of REST is to access data as a glorified file.
Says who? Nothing stops you from creating an interface that maps that row to a file.
That's the whole point of Unix.
Heck, look at the /proc filesystem tree. Even cpu sensor data is available as a file.
Then a process! You spawn a process by opening a file! Erm... again, no.
You want me to continue?
That's true, nothing stops you, though it is worth noting that no one actually does this, and there's a reason for that. So suppose you did this; how are you going to use that in a makefile?
Something something username parens...
GNU make has had an option (.RECIPEPREFIX) to change this for a long time now. Stop whining.
Last modification date is not always a correct heuristic to use, but it's quite cheap compared to hashing things all the time.
Make is a tool for transforming files. I wonder how it's not quite natural and correct for it to assume it's working with files?
1. The assumptions that it makes. Everything in and out are files (phony files notwithstanding). It is hard and painful if you want outputs to rely on and rebuild from configuration in the makefile itself. It's not impossible to implement, but it's difficult to precisely implement it: often times I've seen systems that just rebuild everything after configuration changes.
2. The mix of declarative and imperative styles, while useful for quickly throwing a build together, gets difficult to deal with as things scale up. The make language itself is pretty restricted, too (without using $(eval)).
I know that recent versions support guile extensions and even C(/C++?) extensions, but at that point it's not giving you all that much. I have often wished the make functionality was exposed in some "libmake" for me to extend.
For this reason (and others), I've recently refactored a huge build system to use shake[1] instead. Now builds are precise and correct, and properly depend on configuration and build environment.
[1]: https://shakebuild.com/
* Parallel execution of build rules comes for free in a lot of implementations. This is really noticeable when you do heavy asset pre-processing.
* Cleanly written build rules are re-usable across projects as long as those projects have the same structure (directory layout).
* Cleanly written build rules provide incremental compilation/assembly for free: You express intermediate steps as targets and those are "cached". I put the "cached" in quotes here, because you essentially define a target file which is regenerated when it's dependencies are updated. Additional benefit: Inspection of intermediate results is easy - they are sitting there as files right in your build's output tree.