Unfortunately, you will need to uninstall Nix[1] and then re-install it using that script. But given how we've built the installer that should be pretty seamless.
Yeah, this is unfortunate. We've built the Determinate Nix Installer to be super reliable and safe. We've built the receipt mechanism to have very clear and safe "revert" phases. This has made it hard to architect an upgrade of the installer's work itself ... but I'll put a ticket it! :)
For me it's that my dotfiles and packages are consistent across all my envs. 2 Linux machines, 2 Mac machines.
I'm not sure if Homebrew has this - but it takes me a few seconds to spin up a new Mac entirely with everything i want (minus build times lol). With home brew i was fiddling with "oh yea, this package/lib/plugin is missing" for weeks.
edit: Also, all repos i have are also consistent. Ie i build my repos with specific versions of npm, rust, etc - all per repo, all consistent across all machines, all managed by Nix. Think node-env or py-env (or whatever they're called), but Nix and cross-language instead of specific to node or python ecosystems.
Homebrew has this. Check out Brewfiles and homebrew-bundle. If you want a (excuse self-promotion) dedicated but simple tool to bootstrap new systems: check out https://github.com/MikeMcQuaid/strap
While using `nix-env` or `nix profile` (the new way) are very homebrew-like a huge reason to use Nix is that it is more than a package manager.
Things like nix-darwin and home-manager allow you to decoratively define your system config and dotfiles. Because it is a language you can override things in same place you're defining what packages or etc you want.
And if you start using `direnv` with it's flakes plugin (enabled by default when installed using home-manager!) you can define a flake.nix which will keep a lock file independent of your system's/configuration's that means anyone with Nix installed should be able to activate a native dev environment with the exact versions of node, python, rust, pnpm, etc. ready to go.
It's quite powerful once you become familiar with how to use it.
Nix aims to solve a much more general problem than homebrew. If all you want is to get some packages on your system, there might not be that much in Nix compared to brew for you. The zen of Nix is that building packages is expressed as pure functions, where inputs, outputs, and side-effects are carefully managed. Thus, if you have specific build, packaging, or environment configuration to do, nix can be really helpful. Since everything is in code and every generated nix configuration is effectively isolated from the rest, it makes rolling back to previous configurations trivial in the event something goes wrong. That, and you can VC a config, and in general once a package is properly expressed in Nix it more or less solves the "works on my machine" problem
It's not just a lockfile, though the 'Nix flakes' feature does have a `flake.lock` file. It locks the revision of your 'inputs', i.e., repositories containing packages. The most common and important input is `nixpkgs`, which is kind of like the default package sources on a Linux system. It has glibc, bash, coreutils, etc. but also an extremely large number of other programs that have been packaged. And in a particular revision of nixpkgs, any package has exactly one version. So, yes, the versions of all packages can be locked.
As for the granularity: that really depends on how you want to go about it. If you manage a NixOS system (a Linux distribution built on Nix), you can control the versions on your system. But any project could have its own environment entirely. In fact, in any project you can have multiple versions of nixpkgs if you really wanted to.
Similar to npm lockfile, basically you can install an specific version of a program globally if you want, but it's not recommended, and you can have multiple configurations for different projects or folders with an specific version of tools/apps-
It can be all of those, which is why its "better" than brew or some other package manager. At its core, its just a reproducible/hermetic way to declare and build artifacts.
A .nix file is a lot like a regular programming language, except the "compile target" is actually something like a big digraph of dependencies. Each package is hashed based on the content of the build steps and dependency inputs, so to nix "python 3.9" and "python 3.10" live in two entirely different places (because the hashes are different)
The next step is to actually take that digraph and instantiate a bunch of packages on your system. Each package has its dependencies symlinked to it using env magic, and bubbles up into your shell session or whatever.
All the different offerings in the nix ecosystem are based off this idea. Home-manager focuses on user session environments, NixOS extends the concept to a linux system (e.g. systemd units are managed in the same way - its just another hashed object in the nix store that gets symlinked to systemd), NixOps lets you do it to other machines, and nix-shell lets you create per-project development environments.
Nix flakes are the next evolution in the ecosystem, where some of the "inputs" get taken out of your env where they were black-box magic and put into a flake.nix (with a pinned version locked in flake.lock) so the input package set is controlled for across builds.
> Is nix like a npm lockfile then, carefully controlling specific versions of packages and dependencies on a per... what, per-machine?... basis?
Installation and management of software with Nix is generally handled via tools I'll call 'profile managers', e.g., nix-env, `nix profile`, home-manager, nixos-rebuild, darwin-rebuild (Nix-Darwin), Disnix, NixOps, etc.
These tools all support version pinning (typically via 'flakes', but alternatives can and do integrate this functionality with various Nix profile managers as well).
These profile managers collectively operate at the cluster, machine, and user levels, but on a given machine you can also manage arbitrary named profiles, and you can additionally use Nix without persisting any environment into which things are 'installed'. In the latter case, one can still use source control to pin those environments, allowing for general-purpose, polyglot, per-project package management.
> Or is it kinda like a Docker container that can contain isolated programs?
Yes, but the kind and level of isolation that Nix provides is thinner than Docker. It's most comparable to something like Python's virtualenv or Ruby's Bundler, where packages are installed natively on the local filesystem, but managed via environment variables and symlinks.
The only real difference there is that Nix packages are configured at build time so that they're very hard to find by the dynamic linker and so they're very bad at dynamically looking for each other, providing some additional 'isolation' in a weak sense.
> Or something like pyenv or nvm that can let you switch between version sets?
I wouldn't say so, but you could manage Nix profiles that way if you wanted to. The CLI would be clunky for this at best.
I updated a comment I wrote earlier about what nix is. Always remember to keep your mind and body pure (I kid, but purity is a common theme in nix).
Nix is a programming language plus utilities. It is designed for packaging software in a reproducible way (https://github.com/nixos/nix/).
Each package is called a "derivation", which is a function that takes inputs and makes output. The inputs are everything that is needed to make the output. It is "pure functional" package management - for the same input arguments, the same output will be produced. Nix is really fast because each derivation is hashed and cached and the language is lazy-evaluated.
Builds are "hermetic", meaning only the inputs specified in the derivation are available at build time. Contrast this to some packaging systems, where the build is done against some staging area where packages get installed as they are built and the output can depend on the non-deterministic order that packages are built.
Nixpkgs (https://github.com/nixos/nixpkgs/) is a large collection of recipes for existing software. It contains both rules to build software as well as "modules" to configure it or extend it. NixOS the linux distribution is also part of nixpkgs. There are lots of design patterns here and it can go pretty deep. There are also tons of hacks and patches and workarounds to make software conform to the way nix works. Nixpkgs also has a lot of useful library modules built in.
Nix is the latin word for snow. Nix "flakes" are a way to combine multiple flakes as inputs as well as pin their version. Kind of like pipenv/requirements.txt or "cargo lock" or "yarn lock" but for anything.
The output of derivations go in the "nix store" which is a path like /nix/store/<hash>/, so all sorts of software can co-exist (think multiple incompatible versions of the same library) and can be referenced in a fixed way.
Any kind of software can be packaged as a ".nix" file. It's really common to make a "shell" for software. For example, today I wanted to run GIMP (the image editor). I didn't have to install it in a traditional sense. Instead, I ran 'nix shell nixpkgs#gimp -c gimp'. That makes a shell environment for the "nixpkgs#gimp" derivation, including adding /nix/store/12naig12mnhrzpn88bvvw2vakyd18sjq-gimp-2.10.34/bin to PATH, and runs a shell script command (gimp). Nix makes sure to fetch all the outputs and runtime dependencies of the gimp package for me (which are cached locally and on the internet).
If you are only using nix as an imperative package manager just like homebrew, there is little advantage.
One area where nix shines is that you can write nix code to build a reproducible dev environment for your project.
You can think of it kind of like if you could use a Brewfile to build a virtualenv that doesn't touch your system at all.
I like this a lot because it makes dusting off old projects way easier. I don't have to worry about the state of my system, or reverting to an old version of ruby, etc. to get the old project running again.
Nix is also much more powerful than a Brewfile. You can override the versions of dependencies or even package your own custom dependencies and use those in exactly the same way. The learning curve is steep, but once you learn you have an incredible amount of flexibility and power all built on a robust foundation.
> But wouldn't you want to update your project to a new(er) version of Ruby in that hypothetical?
Perhaps because it's in maintenance mode and what you really need is just to be able to run it to compare its behavior with that of its replacement, or because you want to perform the upgrade by replacing code that relies on deprecated APIs in an incremental way, in a copy of the codebase that still runs and passes tests at each step along the way.
Or maybe you're a scientist and you want to run the original code because you want your tests to have the exact same performance and other characteristics as the code used in the original paper, so you want to avoid compiler/interpreter optimizations that have come out in the meantime.
If those sound like niches, it really sounds like you've never worked on a project spanning more than a few years. Maintenance isn't 'hot', but it's something that keeps a lot of people employed. I deal with all those use cases on a day-to-day basis and Nix is a godsend.
Having developed on that platform for a long time, I will say that It Depends.
Clearly you shouldn't dust off a Ruby 1.8.7 app and deploy it to prod, but what would be the harm in using a script you wrote in 2012, say, to batch process a CSV to JSON?
In fact, the project you're dusting off might just be an old revision of your current app that you already ported (over the years) to current standards, but you thought you wouldn't need that script so you deleted it 7 years ago rather than keep it up to date. if you could check out the old tag and run the script intact, that would be ideal.
Nix has a fundamentally different approach to package management. Homebrew will install everything into `/usr/local/` or `/opt/homebrew` depending on your architecture. From there, all installed software can discover all other installed software in a standard location. If, for example, you wanted to link 'mimalloc', you may find it in `/opt/homebrew/lib/mimalloc.so`. This is roughly how package management on Linux, BSD and presumably other Unices has always worked.
On the other hand, Nix installs software into `/nix/store/longhash-pkgname`. By default any given package is not aware of anything else you have installed, unless said package explicitly referenced the other during build time, including the exact version of the package.
This makes it possible for every package to get exactly the version of a dependency that it's known to work with and allows you to have different packages working with different versions of libraries. This makes Nix extremely reproducible: once a package works as intended, it will almost certainly continue to work no matter what else you change on your system.
On the other hand, I have experienced issues with my system after `brew upgrade`s on multiple occasions. Particularly related to Python upgrades.
I still use `brew cask` to install most GUI applications, since the repository for those on macOS is larger.
It's worth noting you can let Nix manage Homebrew for you, too. I do this on my MBP on which I have nix-darwin running, not for regular Homebrew formulae but for casks for GUI apps.
Excellent point, thank you! I use that as well and intended to mention it and then I just forgot. Personally, I've been hesitant to turn on autoUpdate, have you found it to work well?
Yep, and we now have nix-homebrew, which installs homebrew for you as a flake input. Taps are also flake inputs so your apps versions get declaratively managed/locked too!
I mostly run NixOS but do work on macOS occasionally and being to bring all my familiar home environment, configuration, tools and a multitude of project specific development shells is a huge time saver.
Brew is fine for what it is. Nix generally shines by being a way more general solution to the problem, which means you can take the nix managed version of things and move them across machines/OSes very easily.
The main advantage of nix against brew purely as a mac package manager is the ability to easily use versions of things that aren't HEAD (or to use multiple different versions of things for different reasons). Homebrew has a really clunky way to do that, but its much more straightforward in nix.
I'd say brew is the least effective option but people on macOS just deal with it because it's basically the only package manager they've ever been exposed to. Lots of Stockholm Syndrome going around.
Heavy Linux user for about a decade before being given a MacBook at work and realizing how much time I’d been wasting for no reason (well, not no reason—I’m a lot more comfortable & capable on Linux servers than most other developers I’ve known). Have continued using Linux on servers, appliances, and checking in on how Desktop Linux is going every couple years, in the 13ish years since. I have a lot of experience with two major RPM-based ecosystems, with Portage, and with two major dpkg-based distos. I’ve also done a fair bit with Void, and a little bit with Arch and Nix. After switching to Mac, I used Macports for a little over a year before switching to Brew, IIRC.
The factors I care about in a package manager, in no particular order:
1) How good’s the UI?
2) How often does it break or otherwise make some kind of bad thing happen when I use it?
3) How bad is my day going to be if it breaks or does a bad thing?
> 2) How often does it break or otherwise make some kind of bad thing happen when I use it?
> 3) How bad is my day going to be if it breaks or does a bad thing?
Gosh - I've used FreeBSD's package manager, Gentoo, MacPorts, apt-get, ad-hoc bash scripts, brew, nix, and even occasionally manual compilation for package management over the years, and brew is by far the one that's caused me the most pain.
Just in the last year of $DAYJOB I've probably had brew break my dev environment at least three times because I installed a new tool and it auto-updated the others when I did that.
Maybe I'm holding it wrong, but the others haven't done that to me.
Obviously that's not your experience, and I don't intend to dismiss or demean that.
It's just shocking to me how different our experiences of brew are.
FWIW nix is definitely my favorite packager manager so far. I've been using NixOS on my laptop for about a year, and nix on macOS for package management before that, and it has been shockingly robust. I do not recall a single time a tool I got installed and working via nix has broken on me.
Running a Linux desktop has been kind of disappointing in a lot of ways - while they've improved a lot in the last decade, there are tons of ways macOS and even Windows have superior overall UX.
But for robust package management, nix has been everything I ever dreamed of and then some.
I think software systems need at least three kinds of software installation management (and “system” is key, because you need the operating system’s cooperation, or at least indifference, to make them all function right).
1) One to manage the OS software,
2) One to manage the packages the user or users run directly, and
3) One that can be sandboxed or otherwise isolated from the rest of the system, to some degree at least, for software project dependencies.
Brew does not attempt to do #1 (that’s totally separate—which I now regard as the right way to do things, keeping the kernel et c. installer program and the Firefox et c. installer program separate is better) and is bad at #3, but so are most Linux package managers.
Most of the time when I see people having trouble with Brew, they’re using it for #3 and it’s bad for that, so it’s not working well for them. It’s a type-2 package manager (but, again, most Linux package managers are also bad at that type-3 thing).
I use asdf, direnv, and docker (for daemons—99% of my use of docker is as a sandboxing package manager) for my software project dependency management (type 3). Needing three things sucks, but each does a different thing, and it beats screwing around with per-software-ecosystem tools for things like virtual environments. I would also use them if I were back on Linux (most major distos, anyway—NixOS might be an exception)
Nix, notably, is probably the only semi-popular package management system that could plausibly do a really good job at all of #1-3. I’d probably use it on Mac and Linux both if the package selection were as good as Brew (on Mac), and I didn’t hate their DSL.
I think most popular(ish) Linux package managers other than Nix are worse type-2 package managers than Brew, and bad type-3 ones (as is Brew). I don’t just like Homebrew because I’ve not used plenty of other package management systems, though.
Its basically the only package manager targeted by developers, so users follow suit. I think like 80% of brew users wouldnt be able to name a single alternative they could be using on mac instead.
1. Nix installs all necessary libraries as well as part of the install path. That is - if you need two versions of a library, you aren't needing to contend with whatever is globally installed. I ran into this a lot with Python stuff, personally.
2. You can install different versions of the same software without needing dedicated version managers. This is immensely useful if you work on several different projects that might be using different versions of tooling.
--
But that's just Nix the package manager. Brew does not solve problems of OS configuration, dotfiles management, reproducibility, etc., all of things Nix solves off-the-bat.
For example, I mainly use macOS at work but NixOS for anything personal, as well as servers. I can maintain roughly consistent, through code, environments with only minor tweaks per machine. This means I have essentially the same development environment at work and at home for anything in the terminal.
Another big benefit is, if anything breaks, you can always roll back with Nix. Or, since it's code, it's very easy to start with a clean slate. I recently changed Macbooks and, with only a minor tweak to account for the ARM chip instead of Intel, I was up-and-running with my entire environment exactly as it was, with the only time accounted for being package download and installation, essentially.
--
The main problem that Brew solves that Nix does not (well, it does, but it defers to Brew to do so), is installation of graphical applications. There is an open Apple bug on this issue IIRC - I don't recall all the specific details at this moment, however.
> The main problem that Brew solves that Nix does not (well, it does, but it defers to Brew to do so), is installation of graphical applications. There is an open Apple bug on this issue IIRC - I don't recall all the specific details at this moment, however
The issue is that Spotlight doesn't index across volumes other than /, so when you symlink apps into /Applications or whatever from /nix/store, Spotlight won't recognize them.
Installing those applications via Homebrew is one way to manage it, but you can alternatively instruct Nix-Darwin to install those applications by copying them out of the Nix store rather than symlinking, which also works.
All these newfangled abstractions like Nix, Docker, virtualenvs etc. are simply because people find it more interesting (and easier to monetize) to build new frameworks instead of learning to use existing ones.
As an example, Environment Modules solves 98% of these problems and has been available for more than 30 years:
For me, the advantage of Nix is that I can install stuff without worrying about what it will mean for anything else on my computer.
I can type `nix-shell -p <package-name>` and get a new shell with that package. Maybe it's something I just wanted to try out. When I exit the shell, it's basically gone (it's still on my disk, but it hasn't changed anything in my normal shell environment).
I can have different versions of Python or Ruby for different shells. In fact, I can solve some of the "works on my machine" issues with Nix. I can put a shell.nix file into my git repository. Then when someone else wants to run my Python code, they can use `nix-shell` to bring up a shell with everything I expect to be there. I don't have to tell them "ok, first install version X of Python. Oh, you already have version Y installed? ...well, figure that out and then we're moving on to installing..." It creates a nice little semi-isolated shell with all the things needed to run my thing - and I can replicate that environment on other people's machines.
It's also a ton faster than Homebrew when installing things. I was shocked how much faster.
My advice (if you're interested in trying it): start small. You already have Homebrew. Install Nix on your machine and then forget about it - until you want to install a new package. Then type `nix-shell -p <package-name>` and you have a shell with that package. Later, you can create shell.nix files. You can commit these to a git repo so that you can bring them to a new machine.
I don't personally use it that differently from Homebrew, but there are certainly times when it's really nice to be able to install something without it causing chaos in my system and it's nice to have files written down with the things that I want installed so if I want to reformat my machine, I can get things back up nicely. I'm sure there are more use cases for its semi-isolation, but in the meantime I've been quite happy with it.
Homebrew is deceptively simplistic in a "worse is better" way, but their management is arrogant, make poor engineering decisions, and causes headaches at scale beyond individual web developers who don't notice the subtleties of system entropy or configuration management. For example, their recipe system doesn't respect local changes and insists on querying their API "cloud" for package metadata. Useful options for package configuration were removed. Parallel downloading and installation doesn't happen. Every invocation of the brew command is slow, having to traverse ruby and bash. It sends telemetry by default. There are a bunch of side-effects that aren't handled. Formula inclusion is decided not by subject matter experts but by "Wikipedia editors" who decide arbitrarily.
Nix solves version and configuration dependency hell problem by allowing multiple versions and configurations of the same package side-by-side. Unfortunately, it also still wants to make system-wide changes on macOS. It also has the "problem" of less wider accessibility because the functional nature is more complex.
PS: I have worked in client platform management and engineering at scale.
The main advantage for me is that Nix will not run a system-wide upgrade command (that takes quite a while to execute) by default while running a simple install command, causing breakages all over the place
Which breakages does this fix exactly? The fact that macOS upgrades may overwrite /etc/zshrc, etc? Other than that I haven't encountered any breakages whatsoever.
Your hunch is exactly right. macOS upgrades typically do precisely that and it's quite annoying. I work at Determinate Systems as a full-time macOS user and it's one of those things I kind of quietly tolerated and didn't give much thought, but having this fixed removes a real headache.
Right! It's great to see that tackled for potentially all Nix installations! I have a workaround for that particular issue in my nix-darwin and Home-Manager configurations, but I've always disliked that I needed one in the first place.
While we're speaking of things I need workarounds for: having my login shell be Nix-installed currently has a race condition on systems with FileVault. I have, for example, iTerm2 configured to resume on reboot and it will start before the Nix partition is mounted, thus complaining that it cannot exec my shell. My workaround entails placing a thin wrapper on the main partition that simply waits for /nix to be mounted and then `exec`ing the right shell binary depending on the invoking user.
I should probably open an issue about this, but I'm not sure it can be solved at all while `/nix` needs to be on a seperate volume.
I will do tomorrow! In the meantime, on the offchance anyone stumbles across this who has the same problem, my workaround is available here [0]. (Also exported as `darwinModules.chsh` from my Flake)
I absolutely will! It’ s just … 8.40PM in this part of the world and I want to take the time to write a proper bug report and dig up my prior research on the issue.
Can someone ELI5 why someone would want to do this system-wide installation? Are there that many developers who log in to do their development as multiple different users? Also, is this mandatory for Nix to work?
Personally I prefer to even store regular Mac apps I download in ~/Applications, and anything I can install without disturbing system-global stuff, I prefer to do that way. It means I can migrate things completely by copying one directory, rather than using an "Installer" in the Windows tradition.
Nix packages refer to each other using absolute paths. The convention is to put them in /nix/store, but technically this base path (the "nix store") be changed to whatever you want. However, changing it requires all (non-leaf) packages to be rebuilt to refer to the new path. That also means that you won't be able to use the central package cache anymore, and you effectively get The Gentoo Experience.
The default path could be changed too, of course, but putting it under your home directory would mean that you could only share cached packages with other people that share your username. It would also break NixOS, since that uses Nix to manage the entire system.
> It means I can migrate things completely by copying one directory, rather than using an "Installer" in the Windows tradition.
Nix ships with a specific tool for copying packages (including dependency trees) between computers, nix-copy-closure. But this also assumes that the store path is the same.
> However, changing it requires all (non-leaf) packages to be rebuilt to refer to the new path.
Do you know whether Nix has a proposal to fix that big ease-of-relocation issue via path rewriting? Afaik Spack package manager builds with padded paths so those can then be rewritten to practically any install path
I haven't kept up actively with core proposals, so I don't know. But I'd doubt it since there's no way to do this generically.. even if you rewrite all paths referring to /foo to instead refer to /bar, that wouldn't cover hashed strings, and would be difficult to do in a way that would make binary formats happy (even if you pad it, `/foo\0\0\0` and `/foo` aren't universally equivalent).
Yeah seems like a nightmare since so much of the auto-magic that nix does to fix binaries (shebang rewriting and ELF .so patching) does so by injecting absolute paths (to /nix/store). So all of those would need to be reshimmed for your entire system
EDIT: theres also work being done to move to content-addressed nix store, in which case changing these things would invalidate the content hash
In Spack we rewrite strings in binaries, strings in text files (including shebangs), and RPATH/RUNPATH values. It's not hard to find store prefixes in binaries -- they have hashes in them so they're easy to identify. It's hard to lengthen them -- things in text files and in ELF fields can be made longer, but to make strings longer you need to find all references to them, which is impractical. So we build with long paths.
Relocating is really only a matter of relocating prefixes.
I am not sure how it would affect a content-addressed nix store, but I'm also not 100% sure why you want a content-addressed nix store. Addressing by the derivation hash (which is also what Spack does) lets you find valid build substitutes... would you index that separately for a content-addressed store?
Is the goal to allow multiple versions of non-reproducible builds in the same store?
I'm not 100% sure, but I know one benefit of content-addressed derivations is that they are secure by default. i.e. if the hash matches the build is trusted vs the traditional need to have separate "trusted signatures" and signing workflow to whitelist particular builders.
I also prefer to put my Mac apps in ~/Applications (I thought I was the only one!). I was resistant with Nix defaults for this reason when I set up my newest Mac, but I still wanted all of the other benefits of Nix. Ultimately what persuaded me was that it was basically so trivial to remove Nix and anything it installs… that I actually did remove everything, multiple times, whenever I ran into bumps getting my Nix setup going in the right direction. I got to take the Nix happy path to avoid self-inflicted pain, with the reassurance that it’s far less intrusive than anything else I’ve accepted that installs system-wide.
It seems kind of wild that "installing some binaries on my system" requires these sorts of crazy workarounds like Launch Deamons. I'd personally love something like nix that would live nicely in a low-privilege folder somewhere in my user directory, just to install the few tools I need for a particular project. Currently I can achieve this with VSCode devcontainers and Docker low-privilege install. For system-wide tools I just download prebuilt binaries...
I agree, the lengths the Nix installer currently goes to really shouldn't be required. However, this particular issue isn't about binaries being deleted. The Nix installer sets up some environment variables for you (in particular includes its binaries in PATH). For a system wide installation, it modifies /etc/zshrc for lack of a better option. Now, macOS likes to restore /etc/zshrc to its default on upgrades, thus breaking the environment variables. IMO this is no-one's fault except macOS.
The issue is trivial to diagnose and resolve if you understand the basics of shell startup and how Nix relies on bog-standard Unix features like PATH.
But it's a real annoyance, and the kind of thing that could turn off or confuse new and casual users. And workarounds like this are not unusual on macOS, where upgrades verge on developer-hostile and user-expected behaviors are often impossible without hacks.
Right, it's trivial to restore the mangled configuration files, but permanently resolving the issue of OS upgrades blasting away user configuration files is on Apple.
That issue is exactly what we've fixed in this release of the Determinate Nix Installer, though, so if you use it today you should be covered for future upgrades :).
IMO part of the blame also lies with the way the nix store was designed. Nix can only work in the way you would want with the store located at /nix/store. This misfeature makes single-user, less privileged nix quite hard to pull off even on Linux.
This seems like an overstatement to me, at least when it comes to the practicalities rather than the machinery. Single-user, unprivileged Nix wasn't very hard for me as an undergraduate student a decade ago, and there are much better options for achieving that nowadays.
We wrote the Determinate Nix Installer with hopes that the greater project will adopt it[1]. Nix itself is fantastic. Improving the upstream Nix installer has been described as "unreasonably expensive and pointless"[2]. Hopefully it is adopted soon!
This! At any given time, a lot of Nix goodies are being prototyped outside of the Nix and Nixpkgs codebases. So it is with even the installer for Nix itself. But this is really mainly a cultural thing.
I'd switch to nix package management on macOS if it were completely isolated to a single user, didn't demand use of sudo, creating /nix or other global changes, or running anything as root. Unfortunately, like Homebrew (except with some changes), MacPorts, and Fink, they seem to be set on global system changes that don't scale.
Fair enough! Unfortunately that isn't likely to be true any time soon, since macOS doesn't offer the namespacing primitives necessary for that to be possible like it is on Linux. You can bet once it is feasibly a good experience, DetSys will be hot on its tracks to make it happen. Until then, then :).
Baloney. chroot and execvpe work with mere temporary directories. This is roughly how homebrew does "isolated" builds. You don't need virtualization, but you may want it.
We don't need virtualization (that'd be easy.) We'd need mount namespacing and bind mounts to place a directory in your home directory at /nix during the process execution.
EDIT: User and process namespacing would help a lot as well, to improve build time isolation.
I’m no expert, but my understanding is without this you cannot avoid the need to create /nix using sudo.
On a tangent: I’m a big fan of Homebrew, but Nix’s deterministic builds are more isolated than Homebrew’s. Last I heard, Homebrew also tries to avoid keeping around any but the latest version of a formula, which would force you to upgrade. With Nix, your installed libraries are isolated, so I can (for example) open a shell with GCC 5 and Python 2 and another with GCC 12 and Python 3.12 with absolute confidence there’s no “contamination” so to speak. With Nix you come a lot closer (even on macOS) to achieving the dream of having your entire system declaratively managed, so you can get a brand new machine, pull in your config from GitHub, and have an identical setup with minimal hassle. I think even with `brew bundle` it’s not that easy to achieve this.
What do I need to do to switch from an official Nix install to using nix-installer on macOS? Just uninstall the current nix and install using nix-installer?
Nix seems great for terminal-based development environments where starting a nix-shell sets up all the proper symlinks, env vars, etc. But how does this translate to IDEs like VSCode or IntelliJ?
Like if you clone a repo with a shell.nix file, how does the IDE know to use that environment when you, say, use the File > Debug command, instead of executing whatever toolchain that happens to be on your path? Are there plugins available that enable this?
I realize you could just open a terminal within the IDE, start a nix shell, and run the equivalent run/debug/etc commands, but I am curious if anything exists to make the experience seamless to the user.
That said, I believe VSCode (and some IDEs) don't necessarily support direnv in all cases.. e.g. you'd want direnv to load before other plugins (and update the PATH), but might not get that.
In general you have to open the IDE from a terminal that is running in the context of the nix environment, and then everything will "just work" (except for issues sometimes with nix being readonly)
Have read many stories re. how complicated the official Nix configuration language is, so was wondering whether there is a plan for an easier config language (if not official, maybe a language that could be transpiled to the official one) similar to how this Rust-based installer is a big progress vs. the clunky bash?
It's really not that complicated if you know functional languages. If anything it's quite minimal. It just looks a bit weird because it uses semicolons unlike most functional languages. I kind of wish they just kept JSON/Python syntax for attribute sets and lists.
What's complicated (and often not or only lightly documented) is nixpkgs, the package library.
To that effect, https://mynixos.com has been a really awesome resource for docs.
But yeah, the Nix language is definitely not the complicated part. You can get the language down in a day or two with their reference doc. What is difficult is understanding the patterns to NixOS and nixpkg's. Overlays, derivations, packages, flakes.. that's where things get the most confusing.
It's a huge step away from other packaging mechanisms like PKGBUILDs but much more powerful and type-safe.
But some things just aren't thought of, like home-managers `home.sessionPath` for updating the PATH is _append-only_.. so it literally cannot be used to prepend a preferred bin path. So you follow the typical convention as you might already and include it in the shells rc with `export PATH=/custom/path/bin:$PATH`.
Which is the exact kind of safety you'd have today, so it's not worse, it's just another one of the shortcoming of the modules, as opposed to the Nic language itself. The same shortcoming would exist if you used HCL to declaratively define additional PATH paths. It's typically the module design, not the language.
111 comments
[ 3.6 ms ] story [ 175 ms ] thread1. https://github.com/determinateSystems/nix-installer#uninstal...
edit: tried again and got 0.14. Weird.
We're just ramping 0.14 up to 100%. So please try again in about 10 mins
I'm not sure if Homebrew has this - but it takes me a few seconds to spin up a new Mac entirely with everything i want (minus build times lol). With home brew i was fiddling with "oh yea, this package/lib/plugin is missing" for weeks.
edit: Also, all repos i have are also consistent. Ie i build my repos with specific versions of npm, rust, etc - all per repo, all consistent across all machines, all managed by Nix. Think node-env or py-env (or whatever they're called), but Nix and cross-language instead of specific to node or python ecosystems.
Things like nix-darwin and home-manager allow you to decoratively define your system config and dotfiles. Because it is a language you can override things in same place you're defining what packages or etc you want.
And if you start using `direnv` with it's flakes plugin (enabled by default when installed using home-manager!) you can define a flake.nix which will keep a lock file independent of your system's/configuration's that means anyone with Nix installed should be able to activate a native dev environment with the exact versions of node, python, rust, pnpm, etc. ready to go.
It's quite powerful once you become familiar with how to use it.
Is nix like a npm lockfile then, carefully controlling specific versions of packages and dependencies on a per... what, per-machine?... basis?
Or is it kinda like a Docker container that can contain isolated programs?
Or something like pyenv or nvm that can let you switch between version sets?
As for the granularity: that really depends on how you want to go about it. If you manage a NixOS system (a Linux distribution built on Nix), you can control the versions on your system. But any project could have its own environment entirely. In fact, in any project you can have multiple versions of nixpkgs if you really wanted to.
The next step is to actually take that digraph and instantiate a bunch of packages on your system. Each package has its dependencies symlinked to it using env magic, and bubbles up into your shell session or whatever.
All the different offerings in the nix ecosystem are based off this idea. Home-manager focuses on user session environments, NixOS extends the concept to a linux system (e.g. systemd units are managed in the same way - its just another hashed object in the nix store that gets symlinked to systemd), NixOps lets you do it to other machines, and nix-shell lets you create per-project development environments.
Nix flakes are the next evolution in the ecosystem, where some of the "inputs" get taken out of your env where they were black-box magic and put into a flake.nix (with a pinned version locked in flake.lock) so the input package set is controlled for across builds.
> Is nix like a npm lockfile then, carefully controlling specific versions of packages and dependencies on a per... what, per-machine?... basis?
Installation and management of software with Nix is generally handled via tools I'll call 'profile managers', e.g., nix-env, `nix profile`, home-manager, nixos-rebuild, darwin-rebuild (Nix-Darwin), Disnix, NixOps, etc.
These tools all support version pinning (typically via 'flakes', but alternatives can and do integrate this functionality with various Nix profile managers as well).
These profile managers collectively operate at the cluster, machine, and user levels, but on a given machine you can also manage arbitrary named profiles, and you can additionally use Nix without persisting any environment into which things are 'installed'. In the latter case, one can still use source control to pin those environments, allowing for general-purpose, polyglot, per-project package management.
> Or is it kinda like a Docker container that can contain isolated programs?
Yes, but the kind and level of isolation that Nix provides is thinner than Docker. It's most comparable to something like Python's virtualenv or Ruby's Bundler, where packages are installed natively on the local filesystem, but managed via environment variables and symlinks.
The only real difference there is that Nix packages are configured at build time so that they're very hard to find by the dynamic linker and so they're very bad at dynamically looking for each other, providing some additional 'isolation' in a weak sense.
> Or something like pyenv or nvm that can let you switch between version sets?
I wouldn't say so, but you could manage Nix profiles that way if you wanted to. The CLI would be clunky for this at best.
Nix is a programming language plus utilities. It is designed for packaging software in a reproducible way (https://github.com/nixos/nix/).
Each package is called a "derivation", which is a function that takes inputs and makes output. The inputs are everything that is needed to make the output. It is "pure functional" package management - for the same input arguments, the same output will be produced. Nix is really fast because each derivation is hashed and cached and the language is lazy-evaluated.
Builds are "hermetic", meaning only the inputs specified in the derivation are available at build time. Contrast this to some packaging systems, where the build is done against some staging area where packages get installed as they are built and the output can depend on the non-deterministic order that packages are built.
Nixpkgs (https://github.com/nixos/nixpkgs/) is a large collection of recipes for existing software. It contains both rules to build software as well as "modules" to configure it or extend it. NixOS the linux distribution is also part of nixpkgs. There are lots of design patterns here and it can go pretty deep. There are also tons of hacks and patches and workarounds to make software conform to the way nix works. Nixpkgs also has a lot of useful library modules built in.
Nix is the latin word for snow. Nix "flakes" are a way to combine multiple flakes as inputs as well as pin their version. Kind of like pipenv/requirements.txt or "cargo lock" or "yarn lock" but for anything.
The output of derivations go in the "nix store" which is a path like /nix/store/<hash>/, so all sorts of software can co-exist (think multiple incompatible versions of the same library) and can be referenced in a fixed way.
Any kind of software can be packaged as a ".nix" file. It's really common to make a "shell" for software. For example, today I wanted to run GIMP (the image editor). I didn't have to install it in a traditional sense. Instead, I ran 'nix shell nixpkgs#gimp -c gimp'. That makes a shell environment for the "nixpkgs#gimp" derivation, including adding /nix/store/12naig12mnhrzpn88bvvw2vakyd18sjq-gimp-2.10.34/bin to PATH, and runs a shell script command (gimp). Nix makes sure to fetch all the outputs and runtime dependencies of the gimp package for me (which are cached locally and on the internet).
One area where nix shines is that you can write nix code to build a reproducible dev environment for your project.
You can think of it kind of like if you could use a Brewfile to build a virtualenv that doesn't touch your system at all.
I like this a lot because it makes dusting off old projects way easier. I don't have to worry about the state of my system, or reverting to an old version of ruby, etc. to get the old project running again.
Nix is also much more powerful than a Brewfile. You can override the versions of dependencies or even package your own custom dependencies and use those in exactly the same way. The learning curve is steep, but once you learn you have an incredible amount of flexibility and power all built on a robust foundation.
But wouldn't you want to update your project to a new(er) version of Ruby in that hypothetical?
There have probably been security updates along the way.
Perhaps because it's in maintenance mode and what you really need is just to be able to run it to compare its behavior with that of its replacement, or because you want to perform the upgrade by replacing code that relies on deprecated APIs in an incremental way, in a copy of the codebase that still runs and passes tests at each step along the way.
Or maybe you're a scientist and you want to run the original code because you want your tests to have the exact same performance and other characteristics as the code used in the original paper, so you want to avoid compiler/interpreter optimizations that have come out in the meantime.
Clearly you shouldn't dust off a Ruby 1.8.7 app and deploy it to prod, but what would be the harm in using a script you wrote in 2012, say, to batch process a CSV to JSON?
In fact, the project you're dusting off might just be an old revision of your current app that you already ported (over the years) to current standards, but you thought you wouldn't need that script so you deleted it 7 years ago rather than keep it up to date. if you could check out the old tag and run the script intact, that would be ideal.
On the other hand, Nix installs software into `/nix/store/longhash-pkgname`. By default any given package is not aware of anything else you have installed, unless said package explicitly referenced the other during build time, including the exact version of the package.
This makes it possible for every package to get exactly the version of a dependency that it's known to work with and allows you to have different packages working with different versions of libraries. This makes Nix extremely reproducible: once a package works as intended, it will almost certainly continue to work no matter what else you change on your system.
On the other hand, I have experienced issues with my system after `brew upgrade`s on multiple occasions. Particularly related to Python upgrades.
I still use `brew cask` to install most GUI applications, since the repository for those on macOS is larger.
In my configuration, I have this
Here’s how I do it: https://github.com/dustinlyons/nixos-config/blob/main/flake....
The main advantage of nix against brew purely as a mac package manager is the ability to easily use versions of things that aren't HEAD (or to use multiple different versions of things for different reasons). Homebrew has a really clunky way to do that, but its much more straightforward in nix.
The factors I care about in a package manager, in no particular order:
1) How good’s the UI?
2) How often does it break or otherwise make some kind of bad thing happen when I use it?
3) How bad is my day going to be if it breaks or does a bad thing?
4) How good is the package selection?
Brew is easily my favorite, based on that.
> 3) How bad is my day going to be if it breaks or does a bad thing?
Gosh - I've used FreeBSD's package manager, Gentoo, MacPorts, apt-get, ad-hoc bash scripts, brew, nix, and even occasionally manual compilation for package management over the years, and brew is by far the one that's caused me the most pain.
Just in the last year of $DAYJOB I've probably had brew break my dev environment at least three times because I installed a new tool and it auto-updated the others when I did that.
Maybe I'm holding it wrong, but the others haven't done that to me.
Obviously that's not your experience, and I don't intend to dismiss or demean that.
It's just shocking to me how different our experiences of brew are.
FWIW nix is definitely my favorite packager manager so far. I've been using NixOS on my laptop for about a year, and nix on macOS for package management before that, and it has been shockingly robust. I do not recall a single time a tool I got installed and working via nix has broken on me.
Running a Linux desktop has been kind of disappointing in a lot of ways - while they've improved a lot in the last decade, there are tons of ways macOS and even Windows have superior overall UX.
But for robust package management, nix has been everything I ever dreamed of and then some.
1) One to manage the OS software,
2) One to manage the packages the user or users run directly, and
3) One that can be sandboxed or otherwise isolated from the rest of the system, to some degree at least, for software project dependencies.
Brew does not attempt to do #1 (that’s totally separate—which I now regard as the right way to do things, keeping the kernel et c. installer program and the Firefox et c. installer program separate is better) and is bad at #3, but so are most Linux package managers.
Most of the time when I see people having trouble with Brew, they’re using it for #3 and it’s bad for that, so it’s not working well for them. It’s a type-2 package manager (but, again, most Linux package managers are also bad at that type-3 thing).
I use asdf, direnv, and docker (for daemons—99% of my use of docker is as a sandboxing package manager) for my software project dependency management (type 3). Needing three things sucks, but each does a different thing, and it beats screwing around with per-software-ecosystem tools for things like virtual environments. I would also use them if I were back on Linux (most major distos, anyway—NixOS might be an exception)
Nix, notably, is probably the only semi-popular package management system that could plausibly do a really good job at all of #1-3. I’d probably use it on Mac and Linux both if the package selection were as good as Brew (on Mac), and I didn’t hate their DSL.
I think most popular(ish) Linux package managers other than Nix are worse type-2 package managers than Brew, and bad type-3 ones (as is Brew). I don’t just like Homebrew because I’ve not used plenty of other package management systems, though.
1. Nix installs all necessary libraries as well as part of the install path. That is - if you need two versions of a library, you aren't needing to contend with whatever is globally installed. I ran into this a lot with Python stuff, personally.
2. You can install different versions of the same software without needing dedicated version managers. This is immensely useful if you work on several different projects that might be using different versions of tooling.
--
But that's just Nix the package manager. Brew does not solve problems of OS configuration, dotfiles management, reproducibility, etc., all of things Nix solves off-the-bat.
For example, I mainly use macOS at work but NixOS for anything personal, as well as servers. I can maintain roughly consistent, through code, environments with only minor tweaks per machine. This means I have essentially the same development environment at work and at home for anything in the terminal.
Another big benefit is, if anything breaks, you can always roll back with Nix. Or, since it's code, it's very easy to start with a clean slate. I recently changed Macbooks and, with only a minor tweak to account for the ARM chip instead of Intel, I was up-and-running with my entire environment exactly as it was, with the only time accounted for being package download and installation, essentially.
--
The main problem that Brew solves that Nix does not (well, it does, but it defers to Brew to do so), is installation of graphical applications. There is an open Apple bug on this issue IIRC - I don't recall all the specific details at this moment, however.
The issue is that Spotlight doesn't index across volumes other than /, so when you symlink apps into /Applications or whatever from /nix/store, Spotlight won't recognize them.
Installing those applications via Homebrew is one way to manage it, but you can alternatively instruct Nix-Darwin to install those applications by copying them out of the Nix store rather than symlinking, which also works.
As an example, Environment Modules solves 98% of these problems and has been available for more than 30 years:
https://modules.sourceforge.net/docs/Modules-Paper.pdf
I can type `nix-shell -p <package-name>` and get a new shell with that package. Maybe it's something I just wanted to try out. When I exit the shell, it's basically gone (it's still on my disk, but it hasn't changed anything in my normal shell environment).
I can have different versions of Python or Ruby for different shells. In fact, I can solve some of the "works on my machine" issues with Nix. I can put a shell.nix file into my git repository. Then when someone else wants to run my Python code, they can use `nix-shell` to bring up a shell with everything I expect to be there. I don't have to tell them "ok, first install version X of Python. Oh, you already have version Y installed? ...well, figure that out and then we're moving on to installing..." It creates a nice little semi-isolated shell with all the things needed to run my thing - and I can replicate that environment on other people's machines.
It's also a ton faster than Homebrew when installing things. I was shocked how much faster.
My advice (if you're interested in trying it): start small. You already have Homebrew. Install Nix on your machine and then forget about it - until you want to install a new package. Then type `nix-shell -p <package-name>` and you have a shell with that package. Later, you can create shell.nix files. You can commit these to a git repo so that you can bring them to a new machine.
I don't personally use it that differently from Homebrew, but there are certainly times when it's really nice to be able to install something without it causing chaos in my system and it's nice to have files written down with the things that I want installed so if I want to reformat my machine, I can get things back up nicely. I'm sure there are more use cases for its semi-isolation, but in the meantime I've been quite happy with it.
Nix solves version and configuration dependency hell problem by allowing multiple versions and configurations of the same package side-by-side. Unfortunately, it also still wants to make system-wide changes on macOS. It also has the "problem" of less wider accessibility because the functional nature is more complex.
PS: I have worked in client platform management and engineering at scale.
For context, these changes are:
1. Creating and mounting a volume at /nix for the store
2. A daemon used for isolating build environments
3. A zshrc / fish / bash hook to load a few environment variables (PATH, mostly)
4. (new!) an on-boot Launch Daemon to keep those profile hooks working between upgrades.
While we're speaking of things I need workarounds for: having my login shell be Nix-installed currently has a race condition on systems with FileVault. I have, for example, iTerm2 configured to resume on reboot and it will start before the Nix partition is mounted, thus complaining that it cannot exec my shell. My workaround entails placing a thin wrapper on the main partition that simply waits for /nix to be mounted and then `exec`ing the right shell binary depending on the invoking user.
I should probably open an issue about this, but I'm not sure it can be solved at all while `/nix` needs to be on a seperate volume.
1. https://github.com/determinateSystems/nix-installer
[0] https://github.com/Cu3PO42/gleaming-glacier/blob/master/modu...
Personally I prefer to even store regular Mac apps I download in ~/Applications, and anything I can install without disturbing system-global stuff, I prefer to do that way. It means I can migrate things completely by copying one directory, rather than using an "Installer" in the Windows tradition.
The default path could be changed too, of course, but putting it under your home directory would mean that you could only share cached packages with other people that share your username. It would also break NixOS, since that uses Nix to manage the entire system.
> It means I can migrate things completely by copying one directory, rather than using an "Installer" in the Windows tradition.
Nix ships with a specific tool for copying packages (including dependency trees) between computers, nix-copy-closure. But this also assumes that the store path is the same.
Do you know whether Nix has a proposal to fix that big ease-of-relocation issue via path rewriting? Afaik Spack package manager builds with padded paths so those can then be rewritten to practically any install path
EDIT: theres also work being done to move to content-addressed nix store, in which case changing these things would invalidate the content hash
Relocating is really only a matter of relocating prefixes.
I am not sure how it would affect a content-addressed nix store, but I'm also not 100% sure why you want a content-addressed nix store. Addressing by the derivation hash (which is also what Spack does) lets you find valid build substitutes... would you index that separately for a content-addressed store?
Is the goal to allow multiple versions of non-reproducible builds in the same store?
But it's a real annoyance, and the kind of thing that could turn off or confuse new and casual users. And workarounds like this are not unusual on macOS, where upgrades verge on developer-hostile and user-expected behaviors are often impossible without hacks.
I love the Determinate Nix Installer! My interactions with the maintainer have been very positive, too. Can't recommend it enough. :)
1. https://determinate.systems/posts/determinate-nix-installer
2. https://discourse.nixos.org/t/nix-survival-mode-macos-upgrad...
EDIT: User and process namespacing would help a lot as well, to improve build time isolation.
On a tangent: I’m a big fan of Homebrew, but Nix’s deterministic builds are more isolated than Homebrew’s. Last I heard, Homebrew also tries to avoid keeping around any but the latest version of a formula, which would force you to upgrade. With Nix, your installed libraries are isolated, so I can (for example) open a shell with GCC 5 and Python 2 and another with GCC 12 and Python 3.12 with absolute confidence there’s no “contamination” so to speak. With Nix you come a lot closer (even on macOS) to achieving the dream of having your entire system declaratively managed, so you can get a brand new machine, pull in your config from GitHub, and have an identical setup with minimal hassle. I think even with `brew bundle` it’s not that easy to achieve this.
My config is loosely based off https://github.com/dustinlyons/nixos-config if this matters. Using nix-darwin.
Try https://github.com/direnv/direnv-vscode or https://github.com/fehnomenal/intellij-direnv
More background in https://github.com/direnv/direnv/wiki/Nix.
What's complicated (and often not or only lightly documented) is nixpkgs, the package library.
But yeah, the Nix language is definitely not the complicated part. You can get the language down in a day or two with their reference doc. What is difficult is understanding the patterns to NixOS and nixpkg's. Overlays, derivations, packages, flakes.. that's where things get the most confusing.
It's a huge step away from other packaging mechanisms like PKGBUILDs but much more powerful and type-safe.
But some things just aren't thought of, like home-managers `home.sessionPath` for updating the PATH is _append-only_.. so it literally cannot be used to prepend a preferred bin path. So you follow the typical convention as you might already and include it in the shells rc with `export PATH=/custom/path/bin:$PATH`.
Which is the exact kind of safety you'd have today, so it's not worse, it's just another one of the shortcoming of the modules, as opposed to the Nic language itself. The same shortcoming would exist if you used HCL to declaratively define additional PATH paths. It's typically the module design, not the language.