Show HN: Brioche – A new Nix-like package manager (brioche.dev)
This is a project I've wanted to write for a long time now. I really love the ideas from Nix and I still have a ton of respect for the project, but Nix-the-language never felt intuitive to me and I wanted something with more approachable tooling (although this was circa 2016, so I'm sure Nix has improved a lot since then too-- that was before Flakes were around!)
Anyway, I started on the current iteration of Brioche about 6 months ago, and I finally cut an initial release. I'd still consider this a "technical preview" version (performance especially is pretty painful, so that'll be a focus of mine in the coming weeks). But it's finally at a point where it does work end-to-end and folks can take it for a test drive!
89 comments
[ 3.1 ms ] story [ 157 ms ] threadI'm doing a project in Julia, and I'm using Nix Flakes to do it, but it's been immensely annoying to actually get that working [1]. As a result, I've had avoid using the `nix build` command entirely (though the flakes are still useful for the `nix develop` command).
All that said, do you plan on having Brioche work with reproducible builds, and if so do you have a plan to make what I mentioned a bit less irritating?
[1] I know Julia2Nix exists, and I have never managed to actually get that working on any platform.
I've generally stayed away from using the term "reproducible build" when talking about Brioche, because I don't feel like it fits the reproducible-builds.org definition (though I don't think Nix does either). But, if a build is cached locally or in the registry, then you're guaranteed to get the same result, since it'll just re-use the cache
The sandboxing also gives pretty strong guarantees around hermetic builds[1]. So I think you could do reproducible builds _within_ Brioche (and I'd like to add tools to make this even easier), but I'd say Brioche itself doesn't give you reproducible builds out of the box
[1]: https://bazel.build/basics/hermeticity
It looks like the project files aren't radically dissimilar to Flakes, so I think you're really on the right track for making something that could be really useful for a lot of people. Great work!
This is a bad assumption
If the underlying package manager is good (Rust, Go) then there’s just no need to reinvent the wheel.
Only if building with sandbox enabled. Can disable it if network access is required. Seems someone opened an issue asking[1] for granular permissions (explicit network restriction) but has been marked as stale. In same issue someone else has made comment providing an hybrid approach.
Since you mentioned Julia, it's possible to build Julia environments (with arbitrary packages) using the `.withPackages` function. E.g. `julia.withPackages ["Plots"]`.
[1]: https://github.com/NixOS/nix/issues/4584
Obviously this introduces a potential impurity. I use it for installing NeoForge into a minecraft server derivation; the downloaded files could change behind my back, but so long as the version doesn't change it should still be compatible. It hasn't yet caused trouble.
NixOS has a lot of escape hatches once you look around, and you shouldn't be afraid to use them. The downside of impure derivations aren't quite as bad as not-using-derivations, anyway.
Another option is to declare it as a fixed-output derivation, in which case network access is enabled by default. This doesn't work for most installers, though; at a minimum you'll need to delete logfiles that might contain timestamps.
I agree sometimes is a pain tho.
But most of the time, If you need internet access, if you can specify the "hash" of the result file, anything that you do to achieve that file can have internet access.
1: I believe all software without exception needs a full rewrite at least every 10 years.
I agree with the torrent idea though.
What are your plans for cross-compilation or heavy package customization? One of nixpkgs coolest party tricks imo is that you can just change the stdenv and get a musl static binary or cross-compiled binary.
So in general, I don't think I'm going to have anything quite as powerful as Nix's overrides. But I'm hoping most of the use-cases for it will be covered by some simpler options:
- Since build definitions are functions, package authors can just take arguments for the things they want downstream users to be able to customize (e.g. `configure` flags, optional features and plugins, etc.)
- I haven't built it yet, but I think adding support for dependency overrides would be fairly easy, a la Cargo. Basically, you'd just fork or clone the package you want to tweak, make your tweaks, then set an "overrides" option to use it instead. I know that's not a super satisfying answer, but that should help cover a lot of use cases
- For toolchains specifically, I have an idea in mind for this as well (also not implemented at all). At a high level, the idea is that packages could use "dynamic bindings", which you can then override for downstream recipes (this would require some new runtime features in Brioche itself). The toolchain itself would effectively be a dynamic binding, letting you pick a different recipe (so you could swap glibc for musl, or gcc for clang, etc). Cross-compilation would also be built on this same feature
But the Nix language itself is really quite annoying. I mean, I've more or less gotten used to its annoyances, and I do think that some of the DSLs it has are excellent (I really like the Nginx and systemd configuration stuff, for example), and a lot of the configs are just `services.myservice.enable = true` which is fine, but a lot of the time I'm kind of confused about what syntax is allowed and how loops work and the like. It's not horrible or anything, just a bit annoying because I'll occasionally have to do a nixos-rebuild like three or four times because I messed up some subtle syntax, and it's especially annoying if I have to go dig at the root Nix package to find out what I did wrong [1].
I think decentralizing stuff in the form of flakes might be able to help with this, if for no other reason the area in which you'd be forced to look for configuration stuff could be reduced, but I do think NixOS would benefit from some rearchitecture.
[1] Which happened yesterday with an ethernet card configuration: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modu...
that + learning the like 5 or so idioms that pervade nixpkgs and you can use Nix quite successfully imo.
When a build fails, I use the option to keep the output directory and inspect things. That's another main useful method.
In my opinion the issue with Nix is that the data model is not crisply defined -- it's there, but hidden under a lot of goop that is the Nix language itself and the various assumptions and baggage that goes with it.
What I want is a primarily declarative syntax supporting a rich set of data structures, ideally a non-Turing set of primitives, with a much more intuitive way of gluing things together. So basically bash (or even sh) with a well-defined way of transmitting environment variables and setting up the environment.
The idea of importing a language that has broader support (typescript) as an alternative to the Nix language seems appealing at first, but typescript is such a high-dependency system that it's hard to get excited about it.
TS is certainly excessively powerful though.
It may seem that any definition is final, but none is. Since the last definition of a struct field overwrites a previous one, the order in which e.g. files are interpreted matters. It's like writing a config by using Python dicts, with the added complication that creation of a new one and updating of an existing one are indistinguishable.
This may be okay in practice, given some discipline.
You can have wider types being overridden by more closed types. e.g. string => "this literal". But you cant have two disagreeing values.
I use this to create kubernetes manifests at work, at a reasonably large scale. The inability to join multiple files without fear of collision is specifically one of the best features.
Edit: I'm struggling to see where it fits in the whole k8s ecosystem.
Then I have a tool file that loops through that and renders it into fall files. Then argocd picks that up in chunks with application files.
Its awesome cause it lets me set project wide defaults and restrictions. It also lets me do stuff like. "If there is at least one manifest called x create an argocd application for it automatically" or " loop through these repos and create an app for each one with the following properties"
This in turn requires additional tooling to catch errors early, and also means that a starlark-repl for Bazel will never really be all that useful, since the build graph doesn't exist in starlark alone.
In my experience, this makes Bazel a significantly harder build system to truly grok, tho perhaps easier to use it without understanding it.
Contrast with nix, where the entire build graph exists as a nix expression. In my experience, you can gain a surprisingly deep understanding of nix armed only with knowledge of nix-the-language (and without knowing any implementation details of nix-the-binary-that-builds-derivations).
https://flox.dev
Tried a lot of them, and after a while I found the nix the package manager on non NixOS requires too many workarounds. Things don't just work. For example, installing alacritty requires an OpenGL wrapper. Neovim can't find libraries to build some plugins. Basically, anything GUI had issues.
In the end, `cargo install`, `go install` and download a release archive from github are simpler to script for most of the tools I use.
The people I want to help are those who are unknowingly reviewing malicious commits, and I think that declarative configuration languages have a part to play there.
Since part of the sales pitch of using a general lang is emphasis on code reuse, it rings kind of hollow for me there, and here too.
My mantra has been to avoid "getting bored" and inventing DSLs for the longest time. I initially sought to use Nickel-lang, but it was missing some features that would make it an ideal candidate for this. I started writing my own (you may find this in the history) before realizing "WTF are you doing writing another shitty DSL?" I have subsequently decided that shell scripts (or anything you can shebang) are good enough, i.e. pkgbuild inspiration.
I also plan to avoid making a derivation the source of reproducibility. Instead, a lockfile will offer that. This should alleviate the issue whereby updating the like of glibc cascades into an entire rebuild.
Any *OS and home-manager would need to bring in a configuration language. I think Cue really makes the most sense, but that's still a long way off.
Nice to see you used JS instead of yet another DSL :)
1: https://www.nushell.sh/
2: https://www.nushell.sh/book/thinking_in_nu.html#variables-ar...
One neat thing about Brioche is that Bash isn't really baked into it the same way as the Nix ecosystem. In a lot of the docs and examples, I used the `std.runBash` Brioche function for running scripts, but that's a fairly simple 11-line function[^1]. I wanted to make it just as easy to add `runZsh`, `runFish`, `runNushell`, etc. functions that work just as well as Bash (and they could either be put in the std library or in their own packages). So hopefully, there will be a Brioche `runNushell` function in the near future :)
[^1]: https://github.com/brioche-dev/brioche-packages/blob/9fd5109...
Definitely agreed about the lockfile ideas! I went with a fairly similar design, although I haven't really escaped the "rebuild the world" situation yet and I've rebuilt a _lot_ of copies of gcc from source by now! (that's also partially because I set up the packages repo as a workspace[1], which I felt was easier to iterate on in the early days... I might eventually split each package into their own projects or separate repos so they can keep lockfiles independent of each other)
and yeah, I definitely felt the temptation to write my own DSL but I stayed strong! I just knew that it'd be a huge uphill battle, especially because I wanted to provide good editor support (IMO implementing a language is (relatively) easy, but implementing a language with good error messages and LSP support is crazy hard)
[^1]: https://brioche.dev/docs/core-concepts/workspaces/
If you run out of steam with shell, keep Hay in mind -- which is part of Oils.
It upgrades shell with declarative data, like YAML but not YAML syntax :)
https://www.oilshell.org/release/0.22.0/doc/hay.html
Discussion - https://lobste.rs/s/phqsxk/hay_ain_t_yaml_custom_languages_f...
It's not set in stone yet -- I'm planning an overhaul based on some feedback, so I welcome any more (e.g. on Github or Zulip)
As somebody who knows nix but doesn't know typescript, I found myself looking for a rosetta stone page where I could look at two chunks of code that do the same thing, but in separate languages. This would let me use the familiar language to scrutinize the other.
I wouldn't normally ask for such a thing, but if you're putting "Nix-like" in the title then maybe it might be worth adding a comparison page to the docs.
And yep, I think having a "Brioche for Nix users" guide makes a lot of sense, although it's not the first time that question so I'll probably stand up a first-pass version of it sooner rather than later (my Nix skills are also pretty rusty-- I'll need to brush up a bit first before I write it!)
On the other hand the monolithic nature of the nixpkgs package set is one of the authors gripes with nix, so performance at that scale may be a non-goal.
In other words, the eager part is basically constructing the build graph. Maybe I'm wrong but I don't that this would necessarily be slower than the lazy version. In practice the most complex build graph I've made is basically the full chain of Linux From Scratch builds (that's the basis for my toolchain currently), and I think that takes about 400-500ms to evaluate. It's about 160 build steps, so it's not _simple_ but I know build graphs can also get a lot more complex, so I'll just have to keep an eye on performance as I start to get into more and more complex builds
Maybe I'm missing something but intuitively I'd expect this approach to be fairly efficient-- as long as build scripts only call these functions when they're used as part of the build graph
I mentioned in another comment that this is why Bazel uses simple strings to form dependencies on other targets. That way Bazel can manage the laziness and only evaluate what is needed without needing to use or invent a language with lazy evaluation.
But that is also the big downside (in my opinion) - the full build graph necessarily can't exist purely in starlark (at least for Google-scale projects) which increases complexity of the tool overall.
Edit: I'd like to add, though, that I think it's perfectly fine to not scale to Google scale or nixpkgs scale! Many many projects could still benefit from a great build tool.
Also, no matter how much I might not like it as a language nerd, I think Starlark is simply far more "familiar" for your-average-bear than the Nix language is, which matters quite a bit? It might be more complex in some dimension, but the problem space is fundamentally complex I think. So other factors like how approachable the language is matters. (And at least in Buck2, you can use MyPy style typing annotations, thank God.)
I think you get more or less the same property with Nix. You can have all kinds of errors, even certain syntax errors in the same file, but if they are unneeded for the current evaluation, they won't cause any problems.
As for language familiarity/approachability - this will always be a matter of opinion, but I personally don't think it makes sense to optimize for the casual contributor. Plenty of people know python, but I never see casuals making anything besides trivial changes to bazel build files. I don't think they gain anything by familiarity with python, they could very well copy paste nix or any other language. And if they get in to trouble, they will call in the experts.
Dropping the snark, though -- most times I have seen folks attempt to fold other build systems into some polyglot common one, it seems to create a lot of problems -- for at the end of the day, build tools (whether npm, cargo, gradle, docker, etc.) each have their issues. When these issues arise, they require understanding the original build tool. Bazel, for example, is a "polyglot" system which many try to use to manage npm. In my experience, when folks use bazel but have never used npm directly, they often get lost, because they now have to deal with two abstractions (bazel, npm) simultaneously neither have a way for bazel to do the thing they need, nor an understanding of how to get bazel to do that thing (nor npm itself to do the thing), also because they have never worked with npm directly.
Same goes for cargo, incidentally. Switching over to that build tool -- like any major build tool, it has an insane amount of well-honed documentation and thousands of answered questions online (not to mention LLMs trained somewhat well on it). Users when they need to do something with brioche or have an issue with it will not necessarily have the benefit of the massive collaborative support available for cargo.
Incidentally, I don't think that nix is free of this problem, though it does attempt to manage these issues -- sometimes reasonably successfully -- by virtue of keeping interfaces to other build tools in simple, direct ways.
Even then, however, the best way I have seen nix used is as a system for declaring all binary dependencies or other dependencies that are either not manageable or especially competently managed by other build tools -- which nix allows for in a virtual environment by shipping a shell.nix or flake.
In other words -- bringing in the right version of java, gradle, node, etc. But then just directing users to the specific, native build tools that often have a massive number of contributors.
The closest thing to a tool that does this (guides developers to other build tools) is flox, which uses TOML, but uses nix for its backend via C API. I haven't had a chance to use it much yet, but it looks very promising https://flox.dev.
===
Okay, all this rambling aside -- building a tool like this for yourself or a small team, especially when you have the time to do it, it works well, and you're having fun... well, there's an insane amount to be said for that; and it will doubtlessly lead you to learning a ton about other build tools and likely offer you deep insights that are only possible when attempting to build something like brioche. So, on that count, major kudos to you.
"It should be possible to write an evaluator that plugs in the Guile language (for compatibility with GNU Guix), to use arbitrary builders, and to replace the store implementation."
https://tvix.dev/
Haskell is great in a pure world, but it has an undue amount of friction when in the IO world. It does work in the IO world just fine, but languages like Java handle the actual concerns of the IO world better than Haskell. And for that reason, I say that no, Haskell is not the one true way to write correct code.
It definitely is the One True Way to write happy path code though.
This is me with nix. I'm ok with all the stuff that's emerged in my field (infra/sysadmin) since the 90s. But nix makes me instantly glaze over. Several times I tried, but it's like my brain unplugs itself. I'm sure it does some wonderful stuff at an extremely high price that I'm not willing to pay.
Nix has a huge mindshare, which I don't think you can win back if Nix keeps fixing the core issues mentored in these comments. Best of luck with this, though.
If you never had to solve this problem then it probably isn't for you. If you did, then being able to finally replace the old contraptions with something properly engineered is a breath of fresh air.
Windows called, it wants its Administrators back.
These solutions, abstracted the convoluted solutions away. Who didn't want to form these solutions themselves (which is fair, because it needs intricate knowledge about close-to-OS stuff) built "contraptions" instead.
The core solutions to core problems didn't change at all. We sometimes use the more modern approaches, because we feel lazy sometimes.
Here you come to realize that Nix already comes with all that out of the box, with some minimal tweaking.
Jails/chroots/containers are an orthogonal thing, though often whatever comes out of your software supply chain you'd want to run containerized too.
It's like the difference between building a single building and building an entire city, in terms of scale and total effort. That's why it's taken Nix nearly 20 years to get where it is.
> We sometimes use the more modern approaches, because we feel lazy sometimes.
No, we often use them because they are actually materially better. And Nix is vastly better than all the prior attempts at this problem. If you didn't have those problems or don't think you did, that's fine. But it's got nothing to do with "laziness" whatever that means. It's got everything to do with the results. And the results are ultimately quite good.
It reminds me of current AI craze. We had a bit of this in 2016, it didn't go well: https://en.wikipedia.org/wiki/Tay_(chatbot)
That's a good way of thinking about this. [Showing my age here], I remember being unfazed about the web in the mid 90s, thinking to myself "I already have Encarta which does hypertext with multimedia so well, why do I need this new thing populated by random people".
But of course the web did have a lot to offer. I suppose I keep coming back to "Quantity has a quality all its own" - sometimes even a relatively small upgrade makes an existing solution suddenly accessible to more people and a viable solution to more problems.
Absolutely separate -- in your mind -- NixOS from Nix.
The latter I've seen be a very useful tool that does what people here talk about: cleans up dependency and build and installation management in a reasonably nice way.
The former is a religion/world-view that requires a complete mindset change, accepting a whole dogma, and differs entirely from how you're used to using Linux. I gave up and switched back to Debian after I couldn't get a decent answer on how to set it up easily with my github keys to easily, y'know, check projects out. The mental gymnastics required wer silly.
Nix the language seems obscure at first but it's not so bad after a while. It's sort of got a bit of syntax tossed in from the ML-ish programming languages so there's some familiarity there, for me, maybe.
Where it falls down is like anything else like this: it basically requires that all the things you dep on also adhere to this philosophy and have nix setups already or you'll be in a world of pain rolling your own. I gave up on converting our $work stuff when it became clear that the versions of flatbuffers and a few other deps we depend on just weren't out there in the nix ecosystem and I wasn't going to volunteer to make them.
Anyways, I've seen it be very useful. A former employer used it to set up their whole Julia environment such that custom patches and configuration would all be nicely applied. And this was for both developer workstations and CI and production environments. It's a nice step below docker for doing that kind of thing, and is far less intrusive than docker in many ways.
Maybe I'm a bad NixOS user, but I tend to run most of my development work through steam-run, unless the necessary default.nix is really simple. It works fine, and having the rest of the system be immutable is still an advantage.
Huh? NixOS doesn't require any special configuration to use SSH keys or Git or even SSH keys stored on special hardware like Yubikeys. What went wrong here, or did I misunderstand your use case?
That's the best we can do.
What do you want to tell us with category theory and how does your comment fit into the discussed problem?
I also feel my dad doesn't understand or value abstract things because his Romanian communist math education - as an engineer - fell short of category theory. If he knew categories, we could connect about many more things, but he does not, and this is a massive disconnect that makes my dad unable to apply abstract math to the real world
I was projecting that onto op and suggesting the best we can do is to not be short sighted like our fathers. I self-taught category Theory after dropping out from the math program two times. It changed my life, and let me enjoy math again after burnout and homelessness. And it opened my eyes to many applications of math I never knew about.
My suggestion is that if op learns category theory and the functorial semantics of programming languages, systems like nix will become usable, instead of imposing.
And I am not wrong
It is a common and real problem that builds with "state" cause problems very often for people not even doing complex things.
I think Nix is complex, but because it solves a real, common issue I grin and bear it.
1: https://garn.io/
Building something on Nix obviously has a ton of benefits: well-proven tooling, lots of mindshare already, TONS of packages. Everyone knows that Nix-the-language is fairly unintuitive for beginners, and tools like garn, Devenv, Devbox, etc. all seem to be trying to let folks use Nix without exposing them to Nix-the-language. I think this is a good goal to have and helps make Nix more accessible
But I really thought it was worth exploring the Nix-free design space too. For example, I think derivations in Nix are... kinda complicated and weird! Having a fixed set of phases specifically designed for autotools builds always seemed a little weird to me. By ditching Nix, I was able to see what the world looks like when you ditch the idea of phases completely, for example
Another interesting one was ditching the absolute path for the store path. Getting everything working by resolving relative paths was surprisingly tricky, but the end result is that you don't need to be root to install Brioche, and you can also take a Brioche-built binary (along with the resource dir it outputs) and ship it to another Linux machine and it'll just work. That's only something you can do if you see what happens if you explore outside the Nix space
So I'm hoping Brioche can _at least_ be an interesting case study in how you can take a lot of Nix's ideas and get a lot of the same value without having to take Nix's approach directly (or who knows-- maybe some of those ideas end up back in Nix someday!)
I think when people say they like the idea of nix, they're referring to how it builds packages from programmatic package formulae into an isolated store (so e.g. you can easily have different programs running against different versions of dependencies), and the use cases this allows for. -- With an expressive language for describing packaging, you get stuff like declarative developer environments, or even declared Linux configuration with quick/easy rollback.
I think complaints about the language are partly from Nix being unusual enough that it takes time to get an intuition for the basics of how to write anything in it, but largely from the Nixpkgs (the large package repository the Nix package manager uses) being a large codebase, which grew organically with many contributors, with many similar-but-different solutions to problems, and sometimes the codebase is a bit too clever.
But, can we revisit the name? “Brioche” sounds cool but (at least for me) it’s so awkward to type on the terminal and pronounce