There are a few things which are either enforced directly by Bazel (i.e., Bazel takes an opinion on how build systems should work and gives an error if you try to make it work otherwise). There are also a few assumptions which are now harmless, because Bazel will either check that your assumptions are correct, or make the build work as if those assumptions hold.
For example, "Build graphs are acyclic." Bazel enforces this. "Every build step updates at least one file." If your build step doesn't produce output, Bazel will complain.
"It's possible to tell the compiler which file to write its output to." "It's possible to tell the compiler which directory to write its output to." Bazel does this with sandboxes.
I'm no blaze/bazel expert (hopefully someone will add their own thoughts), but:
1. build graphs are trees/acyclic : bazel enforces this, no cyclic dependencies allowed
2. Compilers will always modify the timestamps on every file they are expected to output : if a write happens why not update mtime? nanosecond precision, what year is it?
3. It's possible to tell the compiler which file to write its output to : makes no sense to me, we expect our compilers to be writing to the correct files in the correct directories.
4. It's possible to predict in advance which files the compiler will update : yes it is, this is how caching works again see (1)
5. It's possible to determine the dependencies of a target without building it : yes yes it's completely possible given (1)
6. Detecting changes via file hashes is never the right thing : Bazel i think uses a combination of hashes and stamps, but why would using hashes not work 99.99999999999999998% of the time?
7. Non-experts can reliably write portable shell script : lol, experts can't reliably write portable shell scripts either
8. Your build tool is a great opportunity to invent a whole new language: or just coopt pieces of one (python: starlark, skylark)
9. Said language does not need to be a full-featured programming language : yes, it doesn't (again .bzl files)
10. Said language should be based on textual expansion : not sure what the alternative to textual expansion (parsing?) is
11. Adding an Nth layer of textual expansion will fix the problems of the preceding N-1 layers : again how is this different from parsing
It's not because the lines are long but because they're prefixed by four spaces (and for anyone reading, please don't do that unless it's actual code).
It's wrapping lines prefixed with two spaces now. The comment I just posted a couple minutes ago with indented shell input and output definitely used two spaces and it is wrapping on my phone and on my desktop if I narrow the window enough.
> 2. Compilers will always modify the timestamps on every file they are expected to output : if a write happens why not update mtime? nanosecond precision, what year is it?
It has nanosecond precision, but on most systems it does not have nanosecond resolution.
Note that the difference is either 0 or 0.004000062. This is on a Debian 9 system running on an Amazon Lightsail instance with an ext4 filesystem.
Here's a quick and dirty script to play with this:
C=0
R=0
while :
do
R=$((R+1))
touch a1; sleep $1; touch a2
T1=$(stat a1 | grep Modify | sed -e 's/.*://' | sed -e 's/ .*//')
T2=$(stat a2 | grep Modify | sed -e 's/.*://' | sed -e 's/ .*//')
if [ $T1 != $T2 ]; then
dc <<HERE
20k $T2 $T1 - p
HERE
C=$((C+1))
if [ $C -eq 100 ]; then
echo "$R runs"
exit 0
fi
fi
done
> not sure what the alternative to textual expansion (parsing?) is
Just realized they other day that PowerShell does this (parsing instead of expansion). Which means much less worrying about escaping output of arguments.
These are some of my personal opinions from using, and violently advocating for, bazel:
> Build graphs are trees. Build graphs are acyclic.
Bazel uses some tricks/magic to hide outputs of build rules from other build rules. Bazel also prevents cyclic dependencies (A depends on B which depends on A).
> Compilers will always modify the timestamps on every file they are expected to output.
Source files a target depends on will be hashed. The operations applied to the files will also be hashed.
> It's possible to determine the dependencies of a target without building it.
Bazel forces people to explicitly define all deps that the source code uses. It uses sandboxing to hide unspecified deps from the source code. All of the definition of these rules is defined statically-ish (if you ignore WORKSPACE which technically applies to Blaze).
> Targets do not depend on the rules used to build them.
Bazel hashes the operations the rules will perform.
> Targets depend on every rule in the whole build system.
Bazel understands the full DAG and knows what each target depends on and what depends on each target.
> Detecting changes via file hashes is always the right thing. Detecting changes via file hashes is never the right thing.
It's a tradeoff that someone has to decide. Neither is right or wrong. Using a file content+metadata hash allows you gauntness that the build is always correct.
> It's possible to narrow down the set of possibly-updated files to a small hand-enumerated set.
As long as you use file hashing and are explicit about your inputs and outputs of every build rule, you can.
> Nobody will ever want to rebuild a subset of the available dirty targets.
You can build a specific target and it's deps at any time. Unrelated dependency trees are ignored.
> stat is slow on modern filesystems.
Anything is slow given a large enough scale. Any syscall is slow once you have millions of source files.
> Non-experts can reliably write portable shell script.
In bazel people interact with higher level rules. You don't need to write shell scripts unless you're implementing something that requires shell scripts. If you'd like you can also use an output of a target as a tool to do further compiling. For example: build protoc then use protoc to compile protos into libraries.
Every build step updates at most one file. Every build step updates at least one file.
It's possible to tell the compiler which file to write its output to.
It's possible to tell the compiler which directory to write its output to.
It's possible to predict in advance which files the compiler will update.
> Your build tool is a great opportunity to invent a whole new language. Said language does not need to be a full-featured programming language. In particular, said language does not need a module system more sophisticated than #include. Said language should be based on textual expansion. Adding an Nth layer of textual expansion will fix the problems of the preceding N-1 layers.
It is a great opportunity to implement a domain specific language as "building software" is a domain that we would like to define programs for. Bazel uses skylark, it's a "fully featured" programming language in that there's loops, conditionals, functions (called "macros"), and a module-ish system ("load()"). It does not operate on textual expansion.
> It's totally OK to spend over four hours calculating how much of a 25-minute build you should do.
Bazel's target analysis takes a while but for my scale of code base it doesn't take hours, it takes seconds. If you have a huge codebase then these "wasted" seconds would be earned back by it's effective caching and remote build execution.
> All the code you will ever need to compile is written in precisely one language.
Yea y'all can pry blaze from my cold, dead hands. It may have its own problems, but working on a team that compiles things in like 10-15 different languages (c++, c#, go, java, javascript, typescript, lisp, haskell, blaze itself (whatever they call it now... skylark?), python, swift, kotlin, soon rust, a couple flavors of sql, etc), it saves us a lot of hassle.
Bazel is cute but doesn't solve the real problem that if one changes only a single line in a C file in general it is wasteful to recompile that file from scratch.
While yes, this is a falsehood... in practice, you generally want to figure out a way to break cycles and remove them from the build system. In general, make the user responsible for breaking cycles in the build system.
The only major use case for cycles that I can think of is self-hosting compilers.
I would add some additional falsehoods:
- It is always the right thing to rebuild files when the inputs have changed.
- The host and target systems are the same (you are never cross-compiling).
- The host and target systems are different (you are always cross-compiling).
- You can reliably figure out the version number of whatever you are building.
- The build steps always produce the correct outputs if they do not fail.
- You always want to build third-party dependencies from source.
- You never want to build third-party dependencies from source.
- The input files do not change while the build is running.
- All of the inputs are files (and not, say, the current timestamp).
- The entire build should use the same build settings (optimization / debug), you don't want to do a partial-debug, partial-optimized build.
- You don't want to rebuild when e.g. files in /usr/include change.
- You always want to rebuild when e.g. files in /usr/include change.
- Installing additional software will not break the build.
- Running the same build twice will result in the same output.
- Build products do not contain absolute paths.
- You can move an executable from one path to another and it will still work.
- You can delete object files and everything will still work.
Normal C code, not that I can think of. But it is not difficult to give a real-world example:
A LaTeX document with a table of contents and an index has a cyclic dependency: The final document, including its pagination, depends on the table of contents, and the table of contents depends on the final document.
I think this is an impossible problem to fix -- inserting page numbers into a document can change formatting enough to cause things to move to a different page.
There are LaTeX documents which never reach a fixed point (I've made one by accident exactly once), and need slight adjustment.
Then "iteratively computing a fixed point by simply rerunning the formatter from scratch" is not a solution to the problem.
Actually I think you can do it iteratively by never decreasing the space occupied by an element between iterations. Of course this might yield ugly results but at least it halts with a solution.
When Knuth was writing TeX, computer systems had really small memories - I'm sure it strongly affected TeX's architecture, because TeX reads the source document and generates output pages one by one, like some kind of online streaming processor. It never holds the entire document in memory, because old computers couldn't do it.
Seems reasonable to me. One of the first things I’d reach for if I wanted to build a flexible tool that’d perform well, last, and be broadly useful is the ability to chop up the workload, ideally into a fixed size, but into some practically-by-convention-probably-not-too-big size as a (riskier) fallback if fixed size wasn’t practical.
IME any time you need performance & stability with unknown input, it’s gonna end up looking like streaming.
This is pretty easy to encapsulate into an individual build step. Rerun LaTeX until the TOC doesn’t change. Fail if this takes too many iterations or repeats.
I use Plain TeX instead of LaTeX, although whichever system you use, these would be the same problems, and there are some possibilities:
- Use separate page numbering for the table of contents.
- Since the number of table of contents entries will not change, just use two passes; you don't need a separate page numbering for the table of contents.
- Never reference page numbers within the main text; use section numbers instead. This avoids problems with their size changed.
Anyways, when I need to do this, I do it purely in TeX rather than using a separate build system. (There is a way to use insertions containing alternating marks and penalties to extract the information needed without shipping out the page.)
> The only major use case for cycles that I can think of is self-hosting compilers.
Yup. In the build system for a self-hosting compiler I wrote I solved it using an optional target. If the compiler is present, generate the boot code using it, else download the boot code from a site on the internet, do md5sum checking and bootstrap from that. In a good build system, modeling this is a piece of cake.
Build systems are tools just the same as an editor. In the same way that editors are opinionated about keymappings, build systems are opinionated about things like timestamps vs hashing.
If you're using a tool, you really should know how it works and where it will break. You should also choose the tool based on the problem you're trying to solve. If you've got a build system that rebuilds when /usr/include changes, and you're trying to do cross-compiles, choose another build system.
Timestamps vs hashes shouldn't be even a thing. Why does this even matter?
Sure ok maybe you touched a file and the timestamp wasn't updated (it should, and your tool is making things harder on you). Or if you're going by hash, same hash, same file. Period. (Unless you're doing something funny with file metadata, and then again, you're shooting yourself in the foot)
Same with some comments on the original article saying "tool doesn't return correct status codes for success/failure" well, maybe you need tools that work then
You see, we all realized that just building code wasn't enough! What your 100 line JavaScript microservice really needs to be accompanied by is a build file that determines how to create an entire Linux VM. Also, because scale, your project also needs to know how to scale that VM out across multiple clusters!
The only answer of course is more build systems!
Oh but JavaScript sucks, so make that Typescript, so shove a transpiler in front of your scripting language. Because somehow we've gone down the path of needing build systems for interpreted languages.
I use all the above tech and get why it is there, but... seriously as a field this is the best we can do? :/
(I legit like Babel and TS, but you have to admit it is /weird/ that we're going through the complexity of having a build system but are reaping none of the performance benefits!)
And re: build systems for interpreted languages, what would the alternative be?
Asking Firefox and Chrome to integrate a language they didn't design?
Making it a runtime module like MoonScript, and letting everyone who loads the page pay for compilation, on battery-constrained computers, millions of times over?
It's the classic question of "Reform or Revolution?"
These systems were built as incremental mitigations to old problems. I don't think it'll change until they collapse under their own weight and people switch over to something else completely.
From my perspective, the main reason for most browser JS projects having a build system is to essentially concatenate modules and to transpile newer JS features so they work on older browsers. That’s really just not as crazy as you make it sound. I don’t know about the Linux VM stuff being part of a browser JS project, maybe I’m misunderstanding that part but it does sound crazy.
> I don’t know about the Linux VM stuff being part of a browser JS project, maybe I’m misunderstanding that part but it does sound crazy.
A bunch of stuff I see comes with a dockerfile these days.
While I get building in docker for reproducibility, kind of -- shipping with one is a pretty big red flag. It means that it's likely to be very painful to port the code out of docker and integrate it into some other build. Which means that it'll be painful to integrate it with other code. Leave the build environment set up to the users. Embrace build diversity, it makes your code less fragile.
At my job, a colleague shared a solution for tunneling into servers that involved having a Docker image running for each server, with some tunneling tricks inside the image.
Those are the times I wonder whether I’m getting old. I kept using a shell alias that does ssh -ND, even though that requires maintaining a config file.
But on the other hand, is a dependency on having Docker available really worse for portability than having a dependency on having a reasonably standard Unix environment?
> But on the other hand, is a dependency on having Docker available really worse for portability than having a dependency on having a reasonably standard Unix environment?
I often need to run it somewhere other than docker -- possibly integrating it into a different program.
Docker doesn't make this impossible, but it's a sign that portability was not considered when the code was written, and I'm in for pain.
> It means that it's likely to be very painful to port the code out of docker and integrate it into some other build.
No it wouldn't. Dockerfiles are stupid simple. A bunch of ADD/COPY commands to add files, and a bunch of RUN commands that are basically wrapping shell scripts.
Plus, find me a modern build system (CI/CD system, rather) that doesn't natively support Docker. It's become a lingua franca at this point.
I just wish it was faster on its own without me having to do all the legwork of maintaining lower layer containers etc. to avoid lengthy package install times.
I had a problem yesterday where I had so many docket images laying around from builds that starting a new container pegged my CPU at 100% and slowed everything in a docker container to a crawl.
A colleague predicted the problem and I learned about one more bit of random fragility that exists in today's ecosystem.
> I had a problem yesterday where I had so many docket images laying around from builds
$ Docker system prune
It makes as much sense to complain about Docker because you let images accumulate in your system as it would make to criticize file systems because you never emptied your system's trash bin.
> No it wouldn't. Dockerfiles are stupid simple. A bunch of ADD/COPY commands to add files, and a bunch of RUN commands that are basically wrapping shell scripts.
Yes. And the code sitting beside them starts to make assumptions -- so when I try to port it to Android so that I can run it under the NDK, for example, it turns into a shitshow because I discover that the code depends on the quirks of the docker environment, and I need to untangle a whole bunch of issues to just get it to build on anywhere.
Containers -- dockers, jails, zones, etc -- are useful tools, but I'd rather see them left up to the people doing deployment, rather than the people shipping libraries. As a library author, I don't know where people want to run my code, so I serve them better by keeping the code portable.
Of course it is only a tool and it does not convert an overcomplicated bloated system into a three-liner. But unnécessary complexity in build systems sucks big time, and interferes with decent testing, so I think this is a real gem.
> (Disclaimer: my current implementation is not as fast as make for some things, because it's written in python. Eventually I'll rewrite it an C and it'll be very, very fast.)
Lol. Whatever, python community...
Anyway, I don't think redo really provides an answer to the insanity that is the js build system(s). I'm also a bit of a DJB fan, like python as a language, etc. However, this implementation in python... eh... I'm not sure about it. It's annoying to require someone to have some particular version of python to utilize a simple build system. He uses SQLite instead of the file system as a database paradigm favored by DJB.
DJB's software is all about technical correctness and simplicity of design. There's nothing wrong with this per se, but it wouldn't surprise me in the least to run this on some version of python that it isn't quite coded for and have it fail in some weird way without me realizing it.
The whole "I'm moving my python code to C tomorrow" thing is becoming a bit of a smell IMHO.
Shell script or a compiled language would be preferable in my opinion.
> However, this implementation in python... eh... I'm not sure about it. It's annoying to require someone to have some particular version of python to utilize a simple build system.
I think that Python as a draft/concept language, for getting the semantics right, and after that rewriting it into a standalone C program is sound, and that is the stated plan for the project.
I think what originally matters is not how it is implemented, but but how it is defined, what are the exact semantics, this is what was the Python development about. Because it is very simple and around 3400 lines of python, creating additional implementations should be not too much work, and my understanding is that a C implementation is the next planned step.
> He uses SQLite instead of the file system as a database paradigm favored by DJB.
Speed is one issue here, it also needs to support efficient multi-processing. Based on my own limited experience with something similar, my personal favorite for this kind of thing is LMDB (https://en.wikipedia.org/wiki/LMDB), which is well-tested, but I guess it would be very hard to make that into a portable version which runs, for example, on Windows, too.
> There's nothing wrong with this per se, but it wouldn't surprise me in the least to run this on some version of python that it isn't quite coded for and have it fail in some weird way without me realizing it.
From following the list (https://groups.google.com/forum/#!forum/redo-list) and blog, my impression is that the main author is a bit of a quality and reliability fanatic. So, for that kind of project, exactly the right person. He provided a test suite which can be used to test alternative implementations, as there are already a few around.
> Shell script or a compiled language would be preferable in my opinion.
There exists a basic (non-parallel) implementation in shell. shell is however going to be a bit too slow for a "production" implementation for large projects where speed matters. My favorite language for this might be guile (I think it matches Guix well), or perhaps Common Lisp. For maximum portability, I believe C is really the best thing.
I'd have nothing to say against an ultra-fast implementation for POSIX systems, written in Rust which calls into LMDB. But while this might be beautiful from a technical perspective, I don't think that there will be a noticeable speed difference in normal use.
Your remark is funny, as I was looking at Bazel for TS last night. It seems that Bazel and PNPM don't work together, so I pretty much just have to punt on it, unfortunately (yes, I have a monorepo).
Alas, it seems like a ton of extra effort to have BUILD files everywhere that repeat the imports of their sibling TS files. And on top of that, how am I going to ensure they stay up to date? What if someone forgets to manually add a dependency to the BUILD file after they've already done an `import {} from ...`? What if I don't want to vendor dependencies?
Unfortunately, it seems like Bazel + TS could be really cool, but the amount of extra effort won't pay off because there are foot-guns all over the place.
Make is actually pretty amazing for its simplicity, longevity, how it gets so much of the job done without doing very much. Yes there are pitfalls and caveats, good uses and bad, but the balance reached between simplicity and ability to be productive and get you most of the way is amazing.
I feel that when I hear somebody say they don't like make, usually they misunderstand this, or they think somebody's bad experiences with makefiles is somehow representative of the capability of the tool.
One person created make in a weekend in 1976. We are still talking about it. Most humans and most software will never achieve this.
I reserve the right to be fully satisfied only after the majority of random Makefiles I find on the net or emitted from build systems will use that convention.
To add another thing to the pot, let me repeat some things already mentioned or implied:
Make is too focused on files. Not every input is a file, and not every artifact is supposed to be a file either.
Make doesn't allow choosing the update criterion per file. Sometimes it's the has, sometimes the timestamp, and sometimes the parsed AST of the source code that should trigger a rebuild when changed.
Make has phony targets for when you don't want a file. Any update criterion can be delegate to a script or a helper tool such as with latexmk for building LaTeX documents with Make.
The phony target is a problem because it can't be an input to another stage of the process.
It's a good thing that the update criterion can be delegated, but this should really be a thing that has a wide range of builtin options, both for ergonomics, and for avoidng mistakes. "Goals" comes to mind: http://git.annexia.org/?p=libguestfs-talks.git;a=blob;f=2020...
- If you have an atomic step that produces multiple targets, it confuses make, and encoding the fact properly in the Makefile is *really* not straightforward.
- tabs are indeed a pain
- make's DSL (the part that pre-processes the Makefile before feeding it to whatever shell you want) is really ugly. In particular, using '$' in the commands is a disaster coz make will chew on those before passing them to the shell.
- Oh yeah, and $ABC is in fact understood as "evaluate variable A" and concatenate with the string "BC". Ugh.
- managing "generated" dependencies (.e.g. includes) is a pain, especially if you try to have make manage the generating step
- managing dependencies on the actual content of the Makefile itself is a pain (as in: if I change the Makefile, what should be rebuilt?)
- Make has no (or rather very little) introspection capabilities and even less self-introspection capabilities. If you want a proof of that fact, try to build a graphviz graph of the dependencies, or better: know what commands will be executed if you change a specific file.
- The many warts an bags on the side that make grew over the years (.PHONY anyone?)
- Recursive Makefiles are a nightmare and make is simply not equipped to deal with them
- etc...
> If you have an atomic step that produces multiple targets, it confuses make, and encoding the fact properly in the Makefile is really not straightforward.
Make does have a lot of warts, but this isn’t one of them any more. Grouped targets were added in the most recent release of GNU Make, and they address exactly this problem.
Re: your other points, many of them boil down to “the syntax is weird”. I agree 100%, and also it’s not so tough once you get used to it. I use GNU Make for tons of things, and I like it a lot.
> . Grouped targets were added in the most recent release
Very good to know, thank you (that's indeed in release 4.3 IIUC)
> e: your other points, many of them boil down to “the syntax is weird”.
I don't think lack of introspection or recursive Makfiles or generated deps. falls in the "syntax is weird" column. Weird syntax I can leave with. Lack or very difficult to use essential features are a different story.
If the build system isn't first a DSL for specifying a DAG, then it fails at being at least as good as Make. Maybe "build graphs are acyclic" is a falsehood, but I would never want to use a build system that isn't fundamentally constructing a DAG. There are plenty of imperative build systems (or build systems that people us imperatively) and they're all horrible.
> "Programmers don't want a system for writing build scripts; they want a system for writing systems that write build scripts."
It is a truth universally acknowledged that a shitty scripting language in possession of a large user base will be in search of a new scripting language that automatically generates the shitty scripting language.
> And since cmake's inception, all cmake alternatives are still pure crap.
…including CMake. :-)
Although, to be fair, I think CMake did at least achieve to prove that creating a turing complete "jack of all trades" meta build system is not a good idea in pretty much every case (although I'd argue that boost had done that well in advance).
It's too sad that with the myriad of legacy patterns around in the C/C++ build space (in production), there is pretty much 0 chance of simplifying the problem for a majority of cases instead.
Yeah cmake is pretty crappy, and it's a shame that nothing better popped up in the last, say, 20 years.
However, to be fair cmake's main problem is it's appalingly poor documentation and user experience. It's very telling that since the release of cmake 3 the main source of info on how to do cmake the right way is not anythig churned out by cmake, whose docs are pretty much unintelligible, but random blog posts that manage to put together basic idiomatic code snippets.
This poor state of affairs is reflected in collaborative sources of info such as stack overflow, where well-meaning users reiterate poor or straight-out wrong ways of doing things mainly because they don't and can't know better.
As I said, to me the whole thing looks like a giant log jam made of bad tools causing other bad tools and there is no simple way of cleaning up the landscape without whipping out a chainsaw and breaking existing workflows and use cases first.
However, if you are in the privileged position of being able to address every use case by compiling and/or cross-compiling from a nix or Guix setup for a project, I'd certainly go with that, too.
The problem there is that this is likely never the case if you can't just containerize your application or prescribe which system it will run on -- which holds true for a lot of software.
IMHO the greatest falsehood I believed about build systems was:
- If build system X is used, people will read its documentation and write build rules according to the documentation.
No, people will just copy what's already there. And if it pulls in 35 external dependencies, compile 500 unnecessary files, and fails 25% of the time because of race conditions between external dependencies, people will add "Note: if build fails, run again."
And then there's `make`. I've seen a philosophically correct Makefile maybe 3 times in my life. Most of the time I see people making every target .PHONY and using patterns to save keystrokes rather than match multiple files.
You'd tend to think people learn over time but no. It's gotten worse. Right when you thought the industry was on the cusp of a build revolution the go community decides to push an old tool that most people can't utilize even 3% of when paired with `go build`. I kinda blame the go ecosystem for pushing Makefile adoption when they already have a caching build tool just so the maintainers didn't have to give the compiler a polished UI. Simplicity at the cost of functionality even if it's simply broken amirite?
I think this is an argument that the user interface of a build system needs to go out of its way to make it hard to misuse it. It's not realistic to believe that all users will find and read documentation from start to finish any more than you can expect them to read EULAs. I'd love to really grok the philosophy of every tool I use so I can apply them in the most intended and synergistic way possible, but things keep getting in the way...
Is this not true?!? While working on my build times I benchmarked that something like a quarter of my time in clang was due to my having large numbers of sparse -I folders as the stat calls were so expensive (this was a 12" MacBook running macOS 10.14 on, I assume, APFS).
I think it started with developers not wanting to write all of the source and artifact files into the makefile.
Then there were some portability issues that should have been handled by the compiler, but by that time there were already metabuilders, so nobody bothered.
- yet it has excellent defaults for most situations
- it is full blown programming language,
- yet it's syntax is extremely specialized for the purpose and familiar at the same time (it's bash)
- it's basic assumptions/structure are extremely simple (timestamps, dependencies)
- yet it's extremely powerful and can take into account most situations.
It's be extremely Unix-style tool with very simple building blocks that combine together in extremely powerful way. Its fast to learn and easy to master and as such, the best kind of design.
Sometimes making a build with it is a small programming project, yet after using all kinds of built tools, I end up just wishing that I could use make instead.
bash is a pretty terrible programming language and it's terribleness increases geometrically with size. This is a large part of why people don't like make I think.
Make is an orchestrator that defaults to sh (not bash, actually). But what I've seen in some really good ones is taking advantage of the "orchestrator" part rather than the "sh/bash" part: One-line recipes that simply call a short script in any other language, including python, that do the desired thing for that recipe. The Makefile is then just used for coordinating partial-running those scripts.
On top of that, you could in fact set SHELL/ONESHELL and then the recipes themselves are python inside the Makefile:
Whoah! I didn't know about that feature. For most build tasks sh is probably best but this could be incredibly useful for some non build related things.
"Build tools are only for building software" is probably another falsehood to add, make can be a great tool anywhere you've got a dependency graph.
That's pretty neat, but I have never ever seen it used (not even in python projects). I'd be skeptical until I'd seen it used in anger.
It also would need to basically support virtualenvs and pip install to be really compelling, I think. Python without those is a pale imitation. That CSV module is known to be pretty bad, for instance (like many built in modules).
Most of the listed assumptions are bad, but some of them, to me, seem essential. In particular, dealing with cyclic build graphs sounds like an absolute nightmare, and I doubt that any build system that allows for such a thing can be easy-to-use.
This "falsehoods programmers believe" is fundamentally different from the more classic ones (e.g. "about names") because its topic is also wholly controlled by programmers. That means in the end it doesn't share much with those documents, but can instead also be characterized as "just" a list of bugs in other programs.
If a program doesn't handle my name, that's a problem with the program. My name is my name; the program doesn't decide that. Depending on the context some government might; the program also generally isn't supposed to decide for the government. Often in practice the program controls both of us; this is the core complaint of "falsehoods" articles, that a program artifact is overstepping its ethical/legal/otherwise extra-computational authority.
But there's no moral demand that e.g. a program exit with 0 status even when it's failed, or that build trees should be cyclic. These are tools by programmers and for programmers concerning programs and when we run into issues it's usually because we chose a poor modeling tool (e.g. a build system should not be ideal to calculate the fixed point of a data processing pipeline, independent of any cycles) or because of a flat-out mistake (there's no reason a compiler can't be given some output directive other than inventor oversight).
Other "falsehoods" are criticism of the software encoding the falsehood, but I read probably 90% of the complaints here as equal or greater failings on the tools invoked. Not only for the sake of my build system do I want DAG pipelines, updated timestamps, or meaningful exit codes - if anything these are the demands I wish to place on other software for the sake of my brain! More like "falsehoods compilers believe about their primacy in program generation."
Or as one of the comments suggests,
>
build systems are a symptom of software languages that are not designed to build software systems
While I agree with a bunch of what you're saying, I expect that FAANG compatibility is not people's top priority when they're not getting paid FAANG salaries.
Some of those aren't believed assumptions, but asserted ones. For instance, I would think a build system that asserts that builds are acyclic is good. People who want a cyclic build are unimportant and can go elsewhere, while I enjoy my simpler build system because of it.
115 comments
[ 5.5 ms ] story [ 382 ms ] threadFor example, "Build graphs are acyclic." Bazel enforces this. "Every build step updates at least one file." If your build step doesn't produce output, Bazel will complain.
"It's possible to tell the compiler which file to write its output to." "It's possible to tell the compiler which directory to write its output to." Bazel does this with sandboxes.
It has nanosecond precision, but on most systems it does not have nanosecond resolution.
Note that the difference is either 0 or 0.004000062. This is on a Debian 9 system running on an Amazon Lightsail instance with an ext4 filesystem.Here's a quick and dirty script to play with this:
Output on the above system: and on a VMWare Fusion VM running Debian 8 on my Mac:Just realized they other day that PowerShell does this (parsing instead of expansion). Which means much less worrying about escaping output of arguments.
> Build graphs are trees. Build graphs are acyclic.
Bazel uses some tricks/magic to hide outputs of build rules from other build rules. Bazel also prevents cyclic dependencies (A depends on B which depends on A).
> Compilers will always modify the timestamps on every file they are expected to output.
Source files a target depends on will be hashed. The operations applied to the files will also be hashed.
> It's possible to determine the dependencies of a target without building it.
Bazel forces people to explicitly define all deps that the source code uses. It uses sandboxing to hide unspecified deps from the source code. All of the definition of these rules is defined statically-ish (if you ignore WORKSPACE which technically applies to Blaze).
> Targets do not depend on the rules used to build them.
Bazel hashes the operations the rules will perform.
> Targets depend on every rule in the whole build system.
Bazel understands the full DAG and knows what each target depends on and what depends on each target.
> Detecting changes via file hashes is always the right thing. Detecting changes via file hashes is never the right thing.
It's a tradeoff that someone has to decide. Neither is right or wrong. Using a file content+metadata hash allows you gauntness that the build is always correct.
> It's possible to narrow down the set of possibly-updated files to a small hand-enumerated set.
As long as you use file hashing and are explicit about your inputs and outputs of every build rule, you can.
> Nobody will ever want to rebuild a subset of the available dirty targets.
You can build a specific target and it's deps at any time. Unrelated dependency trees are ignored.
> stat is slow on modern filesystems.
Anything is slow given a large enough scale. Any syscall is slow once you have millions of source files.
> Non-experts can reliably write portable shell script.
In bazel people interact with higher level rules. You don't need to write shell scripts unless you're implementing something that requires shell scripts. If you'd like you can also use an output of a target as a tool to do further compiling. For example: build protoc then use protoc to compile protos into libraries.
Every build step updates at most one file. Every build step updates at least one file. It's possible to tell the compiler which file to write its output to. It's possible to tell the compiler which directory to write its output to. It's possible to predict in advance which files the compiler will update.
> Your build tool is a great opportunity to invent a whole new language. Said language does not need to be a full-featured programming language. In particular, said language does not need a module system more sophisticated than #include. Said language should be based on textual expansion. Adding an Nth layer of textual expansion will fix the problems of the preceding N-1 layers.
It is a great opportunity to implement a domain specific language as "building software" is a domain that we would like to define programs for. Bazel uses skylark, it's a "fully featured" programming language in that there's loops, conditionals, functions (called "macros"), and a module-ish system ("load()"). It does not operate on textual expansion.
> It's totally OK to spend over four hours calculating how much of a 25-minute build you should do.
Bazel's target analysis takes a while but for my scale of code base it doesn't take hours, it takes seconds. If you have a huge codebase then these "wasted" seconds would be earned back by it's effective caching and remote build execution.
> All the code you will ever need to compile is written in precisely one language.
Bazel is multilingual.
> Everything lives in a single repository.
What is a "repository"? ...
I mean what do you expect? People cram Turing completeness everywhere and anywhere.
https://news.ycombinator.com/newsguidelines.html.
As in any mutually assured destruction scenario, we must all restrain ourselves.
https://news.ycombinator.com/item?id=7609289
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
While yes, this is a falsehood... in practice, you generally want to figure out a way to break cycles and remove them from the build system. In general, make the user responsible for breaking cycles in the build system.
The only major use case for cycles that I can think of is self-hosting compilers.
I would add some additional falsehoods:
- It is always the right thing to rebuild files when the inputs have changed.
- The host and target systems are the same (you are never cross-compiling).
- The host and target systems are different (you are always cross-compiling).
- You can reliably figure out the version number of whatever you are building.
- The build steps always produce the correct outputs if they do not fail.
- You always want to build third-party dependencies from source.
- You never want to build third-party dependencies from source.
- The input files do not change while the build is running.
- All of the inputs are files (and not, say, the current timestamp).
- The entire build should use the same build settings (optimization / debug), you don't want to do a partial-debug, partial-optimized build.
- You don't want to rebuild when e.g. files in /usr/include change.
- You always want to rebuild when e.g. files in /usr/include change.
- Installing additional software will not break the build.
- Running the same build twice will result in the same output.
- Build products do not contain absolute paths.
- You can move an executable from one path to another and it will still work.
- You can delete object files and everything will still work.
Normal C code, not that I can think of. But it is not difficult to give a real-world example:
A LaTeX document with a table of contents and an index has a cyclic dependency: The final document, including its pagination, depends on the table of contents, and the table of contents depends on the final document.
One of the uglier problems of LaTeX, which people of the likes of Leslie Lamport and Donald Knuth should have been able to solve, one might think.
There are LaTeX documents which never reach a fixed point (I've made one by accident exactly once), and need slight adjustment.
Actually I think you can do it iteratively by never decreasing the space occupied by an element between iterations. Of course this might yield ugly results but at least it halts with a solution.
IME any time you need performance & stability with unknown input, it’s gonna end up looking like streaming.
I have written LaTeX rules for Bazel.
- Use separate page numbering for the table of contents.
- Since the number of table of contents entries will not change, just use two passes; you don't need a separate page numbering for the table of contents.
- Never reference page numbers within the main text; use section numbers instead. This avoids problems with their size changed.
Anyways, when I need to do this, I do it purely in TeX rather than using a separate build system. (There is a way to use insertions containing alternating marks and penalties to extract the information needed without shipping out the page.)
Yup. In the build system for a self-hosting compiler I wrote I solved it using an optional target. If the compiler is present, generate the boot code using it, else download the boot code from a site on the internet, do md5sum checking and bootstrap from that. In a good build system, modeling this is a piece of cake.
If you're using a tool, you really should know how it works and where it will break. You should also choose the tool based on the problem you're trying to solve. If you've got a build system that rebuilds when /usr/include changes, and you're trying to do cross-compiles, choose another build system.
Sure ok maybe you touched a file and the timestamp wasn't updated (it should, and your tool is making things harder on you). Or if you're going by hash, same hash, same file. Period. (Unless you're doing something funny with file metadata, and then again, you're shooting yourself in the foot)
Same with some comments on the original article saying "tool doesn't return correct status codes for success/failure" well, maybe you need tools that work then
Most of the advice in the article is sound though
You see, we all realized that just building code wasn't enough! What your 100 line JavaScript microservice really needs to be accompanied by is a build file that determines how to create an entire Linux VM. Also, because scale, your project also needs to know how to scale that VM out across multiple clusters!
The only answer of course is more build systems!
Oh but JavaScript sucks, so make that Typescript, so shove a transpiler in front of your scripting language. Because somehow we've gone down the path of needing build systems for interpreted languages.
I use all the above tech and get why it is there, but... seriously as a field this is the best we can do? :/
(I legit like Babel and TS, but you have to admit it is /weird/ that we're going through the complexity of having a build system but are reaping none of the performance benefits!)
And re: build systems for interpreted languages, what would the alternative be?
Asking Firefox and Chrome to integrate a language they didn't design?
Making it a runtime module like MoonScript, and letting everyone who loads the page pay for compilation, on battery-constrained computers, millions of times over?
Some build systems do build proper VM images, like AMIs.
These systems were built as incremental mitigations to old problems. I don't think it'll change until they collapse under their own weight and people switch over to something else completely.
A bunch of stuff I see comes with a dockerfile these days.
While I get building in docker for reproducibility, kind of -- shipping with one is a pretty big red flag. It means that it's likely to be very painful to port the code out of docker and integrate it into some other build. Which means that it'll be painful to integrate it with other code. Leave the build environment set up to the users. Embrace build diversity, it makes your code less fragile.
Those are the times I wonder whether I’m getting old. I kept using a shell alias that does ssh -ND, even though that requires maintaining a config file.
But on the other hand, is a dependency on having Docker available really worse for portability than having a dependency on having a reasonably standard Unix environment?
I often need to run it somewhere other than docker -- possibly integrating it into a different program.
Docker doesn't make this impossible, but it's a sign that portability was not considered when the code was written, and I'm in for pain.
No it wouldn't. Dockerfiles are stupid simple. A bunch of ADD/COPY commands to add files, and a bunch of RUN commands that are basically wrapping shell scripts.
Plus, find me a modern build system (CI/CD system, rather) that doesn't natively support Docker. It's become a lingua franca at this point.
I just wish it was faster on its own without me having to do all the legwork of maintaining lower layer containers etc. to avoid lengthy package install times.
A colleague predicted the problem and I learned about one more bit of random fragility that exists in today's ecosystem.
$ Docker system prune
It makes as much sense to complain about Docker because you let images accumulate in your system as it would make to criticize file systems because you never emptied your system's trash bin.
Why the hell should the very existance of unused docker images cause the single image I am using to peg multiple cores?
And no warning was given. Heck if my hard drive gets close enough to full that it's gonna be a problem my OS tells me.
Yes. And the code sitting beside them starts to make assumptions -- so when I try to port it to Android so that I can run it under the NDK, for example, it turns into a shitshow because I discover that the code depends on the quirks of the docker environment, and I need to untangle a whole bunch of issues to just get it to build on anywhere.
Containers -- dockers, jails, zones, etc -- are useful tools, but I'd rather see them left up to the people doing deployment, rather than the people shipping libraries. As a library author, I don't know where people want to run my code, so I serve them better by keeping the code portable.
Client side Babel makes sense. And packagers make sense. And it all unfortunately makes sense, but that doesn't make it good.
(Ok being able to try out new syntax with Babel is cool)
I think it can be done better, and above all, much simpler.
I think "redo" looks like a very simple, good solution - here is the documentation: https://redo.readthedocs.io/en/latest/
Of course it is only a tool and it does not convert an overcomplicated bloated system into a three-liner. But unnécessary complexity in build systems sucks big time, and interferes with decent testing, so I think this is a real gem.
Lol. Whatever, python community...
Anyway, I don't think redo really provides an answer to the insanity that is the js build system(s). I'm also a bit of a DJB fan, like python as a language, etc. However, this implementation in python... eh... I'm not sure about it. It's annoying to require someone to have some particular version of python to utilize a simple build system. He uses SQLite instead of the file system as a database paradigm favored by DJB.
DJB's software is all about technical correctness and simplicity of design. There's nothing wrong with this per se, but it wouldn't surprise me in the least to run this on some version of python that it isn't quite coded for and have it fail in some weird way without me realizing it.
The whole "I'm moving my python code to C tomorrow" thing is becoming a bit of a smell IMHO.
Shell script or a compiled language would be preferable in my opinion.
I think that Python as a draft/concept language, for getting the semantics right, and after that rewriting it into a standalone C program is sound, and that is the stated plan for the project.
I think what originally matters is not how it is implemented, but but how it is defined, what are the exact semantics, this is what was the Python development about. Because it is very simple and around 3400 lines of python, creating additional implementations should be not too much work, and my understanding is that a C implementation is the next planned step.
> He uses SQLite instead of the file system as a database paradigm favored by DJB.
Speed is one issue here, it also needs to support efficient multi-processing. Based on my own limited experience with something similar, my personal favorite for this kind of thing is LMDB (https://en.wikipedia.org/wiki/LMDB), which is well-tested, but I guess it would be very hard to make that into a portable version which runs, for example, on Windows, too.
> There's nothing wrong with this per se, but it wouldn't surprise me in the least to run this on some version of python that it isn't quite coded for and have it fail in some weird way without me realizing it.
From following the list (https://groups.google.com/forum/#!forum/redo-list) and blog, my impression is that the main author is a bit of a quality and reliability fanatic. So, for that kind of project, exactly the right person. He provided a test suite which can be used to test alternative implementations, as there are already a few around.
> Shell script or a compiled language would be preferable in my opinion.
There exists a basic (non-parallel) implementation in shell. shell is however going to be a bit too slow for a "production" implementation for large projects where speed matters. My favorite language for this might be guile (I think it matches Guix well), or perhaps Common Lisp. For maximum portability, I believe C is really the best thing.
I'd have nothing to say against an ultra-fast implementation for POSIX systems, written in Rust which calls into LMDB. But while this might be beautiful from a technical perspective, I don't think that there will be a noticeable speed difference in normal use.
Alas, it seems like a ton of extra effort to have BUILD files everywhere that repeat the imports of their sibling TS files. And on top of that, how am I going to ensure they stay up to date? What if someone forgets to manually add a dependency to the BUILD file after they've already done an `import {} from ...`? What if I don't want to vendor dependencies?
Unfortunately, it seems like Bazel + TS could be really cool, but the amount of extra effort won't pay off because there are foot-guns all over the place.
2018 https://news.ycombinator.com/item?id=16196899
Why? What's wrong with Make?
I feel that when I hear somebody say they don't like make, usually they misunderstand this, or they think somebody's bad experiences with makefiles is somehow representative of the capability of the tool.
One person created make in a weekend in 1976. We are still talking about it. Most humans and most software will never achieve this.
Yes, I realize GNU Make has a config option to fix this. That doesn't seem to be widely adopted.
On a second thought, the shellisms it borrows are also annoying. POSIX shell syntax needs to die on its own right.
I reserve the right to be fully satisfied only after the majority of random Makefiles I find on the net or emitted from build systems will use that convention.
To add another thing to the pot, let me repeat some things already mentioned or implied:
Make is too focused on files. Not every input is a file, and not every artifact is supposed to be a file either.
Make doesn't allow choosing the update criterion per file. Sometimes it's the has, sometimes the timestamp, and sometimes the parsed AST of the source code that should trigger a rebuild when changed.
It's a good thing that the update criterion can be delegated, but this should really be a thing that has a wide range of builtin options, both for ergonomics, and for avoidng mistakes. "Goals" comes to mind: http://git.annexia.org/?p=libguestfs-talks.git;a=blob;f=2020...
The list is long, but here's a random sample:
Make does have a lot of warts, but this isn’t one of them any more. Grouped targets were added in the most recent release of GNU Make, and they address exactly this problem.
Re: your other points, many of them boil down to “the syntax is weird”. I agree 100%, and also it’s not so tough once you get used to it. I use GNU Make for tons of things, and I like it a lot.
Very good to know, thank you (that's indeed in release 4.3 IIUC)
> e: your other points, many of them boil down to “the syntax is weird”.
I don't think lack of introspection or recursive Makfiles or generated deps. falls in the "syntax is weird" column. Weird syntax I can leave with. Lack or very difficult to use essential features are a different story.
It is a truth universally acknowledged that a shitty scripting language in possession of a large user base will be in search of a new scripting language that automatically generates the shitty scripting language.
But then you have two problems.
Any deficiency in a complex, widely used and difficult to change development tool will be fixed by adding an extra layer over it.
If the deficiency is something like complexity or lack of reliability that can not be fixed by adding extra layers, read this comment again.
And since cmake's inception, all cmake alternatives are still pure crap.
The definition of "well" is relative.
…including CMake. :-)
Although, to be fair, I think CMake did at least achieve to prove that creating a turing complete "jack of all trades" meta build system is not a good idea in pretty much every case (although I'd argue that boost had done that well in advance).
It's too sad that with the myriad of legacy patterns around in the C/C++ build space (in production), there is pretty much 0 chance of simplifying the problem for a majority of cases instead.
However, to be fair cmake's main problem is it's appalingly poor documentation and user experience. It's very telling that since the release of cmake 3 the main source of info on how to do cmake the right way is not anythig churned out by cmake, whose docs are pretty much unintelligible, but random blog posts that manage to put together basic idiomatic code snippets.
This poor state of affairs is reflected in collaborative sources of info such as stack overflow, where well-meaning users reiterate poor or straight-out wrong ways of doing things mainly because they don't and can't know better.
"I thought I was dumb until I realized it's the book".
Top Amazon review of the top book on CMake. And I am still wondering whether it is the book, or the tool.
However, if you are in the privileged position of being able to address every use case by compiling and/or cross-compiling from a nix or Guix setup for a project, I'd certainly go with that, too.
The problem there is that this is likely never the case if you can't just containerize your application or prescribe which system it will run on -- which holds true for a lot of software.
- If build system X is used, people will read its documentation and write build rules according to the documentation.
No, people will just copy what's already there. And if it pulls in 35 external dependencies, compile 500 unnecessary files, and fails 25% of the time because of race conditions between external dependencies, people will add "Note: if build fails, run again."
Sigh.
You'd tend to think people learn over time but no. It's gotten worse. Right when you thought the industry was on the cusp of a build revolution the go community decides to push an old tool that most people can't utilize even 3% of when paired with `go build`. I kinda blame the go ecosystem for pushing Makefile adoption when they already have a caching build tool just so the maintainers didn't have to give the compiler a polished UI. Simplicity at the cost of functionality even if it's simply broken amirite?
Sigh.
To be fair, I'm still yet to see a document clarifying how to use patterns with make.
Is this not true?!? While working on my build times I benchmarked that something like a quarter of my time in clang was due to my having large numbers of sparse -I folders as the stat calls were so expensive (this was a 12" MacBook running macOS 10.14 on, I assume, APFS).
How did this go wrong??
Then there were some portability issues that should have been handled by the compiler, but by that time there were already metabuilders, so nobody bothered.
falsehood #1
The worst are crackpot systems like ant. Whoever thought a half-baked DSL with a XML concrete syntax was a good idea?
> Whoever thought a half-baked DSL with a XML concrete syntax was a good idea?
You kinda answered your own question there no?
Make does so many things correctly:
- it gives user total freedom to modify the built
- yet it has excellent defaults for most situations
- it is full blown programming language,
- yet it's syntax is extremely specialized for the purpose and familiar at the same time (it's bash)
- it's basic assumptions/structure are extremely simple (timestamps, dependencies)
- yet it's extremely powerful and can take into account most situations.
It's be extremely Unix-style tool with very simple building blocks that combine together in extremely powerful way. Its fast to learn and easy to master and as such, the best kind of design.
Sometimes making a build with it is a small programming project, yet after using all kinds of built tools, I end up just wishing that I could use make instead.
> and familiar at the same time (it's bash)
Make is an orchestrator that defaults to sh (not bash, actually). But what I've seen in some really good ones is taking advantage of the "orchestrator" part rather than the "sh/bash" part: One-line recipes that simply call a short script in any other language, including python, that do the desired thing for that recipe. The Makefile is then just used for coordinating partial-running those scripts.
On top of that, you could in fact set SHELL/ONESHELL and then the recipes themselves are python inside the Makefile:
And:"Build tools are only for building software" is probably another falsehood to add, make can be a great tool anywhere you've got a dependency graph.
It also would need to basically support virtualenvs and pip install to be really compelling, I think. Python without those is a pale imitation. That CSV module is known to be pretty bad, for instance (like many built in modules).
For build it mostly works fine. Doing anything else with it would be a total pain.
If a program doesn't handle my name, that's a problem with the program. My name is my name; the program doesn't decide that. Depending on the context some government might; the program also generally isn't supposed to decide for the government. Often in practice the program controls both of us; this is the core complaint of "falsehoods" articles, that a program artifact is overstepping its ethical/legal/otherwise extra-computational authority.
But there's no moral demand that e.g. a program exit with 0 status even when it's failed, or that build trees should be cyclic. These are tools by programmers and for programmers concerning programs and when we run into issues it's usually because we chose a poor modeling tool (e.g. a build system should not be ideal to calculate the fixed point of a data processing pipeline, independent of any cycles) or because of a flat-out mistake (there's no reason a compiler can't be given some output directive other than inventor oversight).
Other "falsehoods" are criticism of the software encoding the falsehood, but I read probably 90% of the complaints here as equal or greater failings on the tools invoked. Not only for the sake of my build system do I want DAG pipelines, updated timestamps, or meaningful exit codes - if anything these are the demands I wish to place on other software for the sake of my brain! More like "falsehoods compilers believe about their primacy in program generation."
Or as one of the comments suggests,
> build systems are a symptom of software languages that are not designed to build software systems
- It's OK to embed/vendor dependencies
- It's OK to hardcode VCS URLs for your dependencies
- you can pull random stuff from the Internet at compile time and large companies will accept it
- It's OK to release libraries without version numbering or using git commitish
- Everybody uses containers
- Nobody cares about reproducible builds
- Nobody cares about security, license compliance and being able to rebuild things reliably
- All FAANGS will use your project even if it cannot packaged into OS packages
It now takes me weeks to setup a fresh machine to do local dev on. I guess I should make a docker image of my dev box.
...would be nice, now that I think about it.However, if I'm dealing with something special, or if the build system is unstable, I always fall back to Make. Such a beautiful tool!
Tup is also worth to mention: http://gittup.org/tup/
https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
Here is a teaser, the 1st table:
which shows that Excel can be viewed as a dynamic build system!Simplifying assumptions are good.