93 comments

[ 3.2 ms ] story [ 160 ms ] thread
I really want to love Nix and have installed it multiple times. It's non-standard file layout gets me every time though. Something as simple as `#!/bin/bash` not working is problematic for me. I also find that trying to install anything not included in the Nix package manager is pretty much impossible (building from source...).

Please correct me if I'm Nix-ing wrong.

You shouldn't be using /bin/bash anyway. The proper she-bang is '#!/usr/bin/env bash', which works on nix. The location of bash in /bin/ is actually not standard at all, despite some linux distributions doing that. In particular, the BSDs also do not have bash in /bin. This is why using '/usr/bin/env bash' is more portable
Conversely, Python's PEP394 explicitly discourages `/usr/bin/env python`, so there's that. (I know why the PEP says so, just pointing out that it differs from Nix style)
I've always mechanically written '#!/bin/bash' as my shebang for bash script, while I normally use '#!/usr/bin/env python' for python scripts. In retrospect, that sounds obviously incoherent.

I would like to know if there is some common practice/standard around using '/usr/bin/env' in shebangs.

Could you please elaborate more?

> /usr/bin/env

That won't work everywhere either as the location of 'env' is not a fixed standard either. Eg. In some *nix's its at /bin/env.

What modern/surviving Unix-likes don't have even a symlink in `/usr/bin/env`?

I'm not 100% sure on all of these, but every one of the dozens of Linux distros I've used has `/usr/bin/env`. macOS has it, and it looks like FreeBSD does, too, and so does NetBSD.

Who are the oddballs? Does it include any of the free Unices?

(I'm not trying to be shitty; I'd really like to know. I spent some time Googling about conventions for the `env` command, and examples of incompatible systems. The only ones I found were two old, proprietary Unices. I got tired of searching and was hoping someone who knows a few examples offhand could help me out.)
Incidentally, macOS, whose CLI is derived from BSD, does in fact have /bin/bash.
There is a `/bin/sh` and `/usr/bin/env` provided - so you can use `/usr/bin/env bash`.

There is also this function which may be necessary for certain software: https://nixos.org/nixpkgs/manual/#sec-fhs-environments

You can compile some software manually in place using, for example, `nix-shell -p libssh2 -p zlib` without having to write a derivation. However, without writing a derivation, it may cease to function after a garbage collection or an upgrade.

Writing a derivation isn't too bad, here's one for one of my repos: https://github.com/jbboehr/handlebars.c/blob/master/derivati...

Sure - you're Nixing wrong. Not to worry, it actually takes ages to figure out how to Nix right - there's not enough documentation on the matter.

The correct way to get almost anything installed is to write a Nix package, and install that. This is actually remarkably easy in comparison to other package managers - you write the package definition in a single file, normally just having to set the source code location and list its dependencies, and you can make it available to install with nix-env by adding it to a packageOverrides declaration in ~/.config/nixpkgs/config.nix.

The "stdenv" build system that nixpkgs uses will automatically install anything that can be configured via "./configure && make && make install" - cmake is supported just by listing that as a build dependency. Stuff written in scripting languages is more fiddly, but the nixpkgs documentation usually has examples. The downside is that a significant amount of the time, libraries written in scripting languages aren't packaged either, and you wind up packaging half a dozen libraries just to get some tool working. The <X>2nix tools can be useful in these cases to generate a package definition.

Thank you for the awesome reply (and everyone else). This is precisely the issue I have though:

> The downside is that a significant amount of the time, libraries written in scripting languages aren't packaged either, and you wind up packaging half a dozen libraries just to get some tool working.

I've gone down this path a few times and days later when I still don't have the original thing I wanted working installed, then I sort of give up. I realize there are many benefits to Nix and I really enjoy them but I've switched to using Arch for practical reasons. I really, really miss the declarative reproducible style though.

What I wind up doing while developing is just creating a nix-shell with my native dependencies, and using pip/cargo/whatever inside that. It's not the "proper" way of doing things - you wouldn't want to install a tool long-term that way - but it's useful for one-off things or while you're playing around.

I've used npm2nix and cargo2nix a couple of times with success - they generate package sets from the relevant sources.

>you wouldn't want to install a tool long-term that way

I'm curious why not.

Aren't pip/cargo/whatever suitable for long-term use?

Consequently, couldn't nix's upgrading routines be modified to call out to pip/cargo/whatever to upgrade things best upgraded by pip/cargo/whatever?

I guess what I want to know is whether the architecture or the philosophy of nix interferes with nix's being modified in the way I just described. (Maybe it is just that nobody's gotten around to doing the implementation work.)

> Consequently, couldn't nix's upgrading routines be modified to call out to pip/cargo/whatever to upgrade things best upgraded by pip/cargo/whatever?

No. pip/cargo/etc are nondeterministic. Nix packages should not be. Where they are, things often break in bizarre ways.

In order to access the Internet, a Nix package must declare the hash of its output - given that pip can produce different output from run to run (it contains a constraint solver which depends on the versions available in the external package repository) this is impossible.

Have you tried using something like `home-manager` (https://github.com/rycee/home-manager)? It gives you declarative package and configuration management with Nix on non-NixOS. It might be a nice compromise until NixOS/Nixpkgs is mature enough to meet your needs more completely. Or maybe you'll never switch, if you really like the convenience of being able to easily grab a trendy package from AUR or whatever. But either way, there's a lot of declarative Nix-y goodness you can get on Arch Linux! :-D

There's a similar tool for macOS called `nix-darwin`, as well: https://github.com/LnL7/nix-darwin

Please don't down-vote criticism. This is a real feedback which I am sure applies to a lot of new-comers.
Can someone explain what is NixOS designed for? (ELI5 please)
Reproducible OS setups.

Like Docker, but it uses install scripts instead of layered images.

and an actual solid language instead of a sequence of whatever
And it does a lot more work to prevent you from shooting yourself in the foot.

Docker scripts that e.g. clone a remote and build from the HEAD are not reproducible. Another example would be downloading fancysoftware-latest.tgz

Nixos will have you either pull from a specific git commit, or will hash the sources, so the default behavior is reproducible.

Reproducibility.

In a nutshell, it's a purely functional Linux distribution. That means all changes you make to your system are non-destructive. For example, you can always roll back to a previous OS state, which is represented by a hash computed from all packages installed, and all options you've set. In turn, package hashes are computed from the package sources and all inputs (buildtime and runtime package dependencies).

I feel like one day this will be how all OSes work and we'll look back and wonder how we ever found the old way acceptable.
Then another 10 years later, we'll think this is all too complicated and reinvent the wheel/simplicity.
(comment deleted)
We migrated from Debian+Ansible to NixOS to easier be able to ensure consistency across servers. Rolling back and forth is really easy for most things.

It's also the most convenient system I ever worked with for creating custom packages, which is lucky, because NixOS does have fewer pages compared to other distributions.

> NixOS does have fewer packages compared to other distributions

It's not bad, though: https://repology.org/repository/nix_stable

Haskell and R packages will be added soon to give a more complete picture.

You're right, it's not bad, but chances are one will need to do a bit of packaging work if using NixOS for many different or complicated things.

But as I said, it's all in all really nice to work with Nix.

>It's also the most convenient system I ever worked with for creating custom packages

Is Arch Build System one of the systems you've ever worked with? (I found ABS very convenient.)

You really do not want to use Arch for production, enterprise systems.
OK, but I still want an answer to my question -- to help me decide what distro to choose for personal use.

(I already get that the design of NixOS prevents the system's ending up in an incoherent state, which will happen on Arch eventually if you wait long enough between upgrades.)

I've developed packages for ABS before, and it's nice. What nix's model gets you is (a) largely avoiding complex bash scripts, and (b) a pile of tools for working with packages that you don't want to install semi-irreversibly. nix-shell, for example, will put you in a shell inside the build environment, and you can run the build steps yourself and see what the outcomes are. nix-build will build a piece of software and create a symlink to the output, but not install it into your user's package namespace.

One of the nice things is that you can install many things without having to sudo - the build is run by a daemon and sandboxed. nix-shell can also be used to create a shell in which a given package set is installed - you can use that to use a piece of software as a one-off or create a development environment that doesn't pollute your general system. Tools like home-manager[0] can help with managing your home directory in a similar way to NixOS's management of your system, too - I have redis and postgres installed using home-manager to run as my own user on demand under systemctl.

[0] https://github.com/rycee/home-manager

Nope, I haven't. Last time I ran Arch as my personal OS, a normal update bricked the system after a week. :P

My experience is mostly with deb and rpm before Nix.

When was that? I think I had stuff like this when first looking into Arch ~8 years ago. It's now been my daily driver for about 6 years, and I haven't had my system bricked. I have had a few updates that broke stuff, which could be solved through a downgrade, but no bricking.

So maybe the Arch maintainers got more disciplined, or maybe I just got better at not breaking things. Probably both.

Coming from Arch I love NixOS for two reasons:

- No separate AUR that some packages are arbitrarily located in

- Not having to care about binary packages: they are just transparently downloaded and used if available

Pacaur makes the AUR a bit more palatable, but you still notice the split.

To supplement and illustrate your claim about NixOS being easy to package for, I'd like to share an example:

I've been using NixOS more or less exclusively for ~2.5 years now. Whenever I want to run software on NixOS which is not already in Nixpkgs, I package it (if I want it bad enough). This week, for example, I packaged KSmoothDock so that I could try it out.

This is the whole thing:

    { mkDerivation, lib, fetchFromGitHub
    , cmake, extra-cmake-modules
    , plasma-framework, kwindowsystem }:

    let
      version = "5.9";
    in

    mkDerivation {
      name = "ksmoothdock-${version}";
      src = (fetchFromGitHub {
        owner = "dangvd";
        repo = "ksmoothdock";
        rev = "v${version}";
        sha256 = "1fbghyd079xk4q5na8msna5zkfg85c4ksyfqy524v2pm96dyl2dr";
      } + "/src");

      nativeBuildInputs = [
        cmake
        extra-cmake-modules
      ];

      buildInputs = [
        plasma-framework
        kwindowsystem
      ];

      postPatch = ''
        substituteInPlace CMakeLists.txt \
          --replace /usr/share/ share/
      '';

    }
It didn't feel like much work and I think it only took a few minutes. In this case, I was able to base the package definition on another 3rd-party dock for Plasma, so it was even easier than usual to get started. I just copied the other package and changed the package name and location of the source code, and everything worked. I then cleaned up by consulting the README for the project and removing as many extraneous dependencies as possible, and smoothed over a quirk, which was pretty painless.

Once you get a feel for the docs (and the Nixpkgs source, just because it's a treasure trove of examples), packaging for/with Nixpkgs is usually pretty easy, and the results pretty readable.

I should also add that the range of packages already included also seems to me to have improved a great deal over the years that I've been using NixOS. And I think once NixOS gains support for Snap packages and Flatpaks (the latter is in the works and has been making good progress recently), it will become a much more viable desktop OS for those unwilling or unable to deal with packaging the odd missing application.

Besides reproducibility, I like it as a nooby Linux user because I can add and remove packages willy nilly with complete confidence that my machine is in a clean, reliable, understandable state. I can even set up conflicting package arrangements and flip between them with a single, fast command.
Yep! I think this point is sometimes underemphasized as NixOS has some rough edges and its users tend to get carried away and geek out about how it works.

But it's a great platform for fearless experimentation, because it's so easy to revert any change you later decide was undesired.

Imagine any single change in your system configuration (packages, configuration, startup scripts, whatever) is loaded into a source version control as you move forward.

If something fails, you can always rollback to a previous revision, and since everything is there, you know your whole system will be working.

If you want to reproduce your system, you can copy the whole content and "checkout" the relevant revision on the new system. Because the hash is the same, every single bit underneath will be the same.

NixOS is a Linux distro built around the Nix package system. Nix is built around the idea of immutability. It makes all packages immutable by giving them their own directory identified by a hash that is derived from ALL of that package's dependencies. This has a number of desirable properties:

* It makes it trivial to have multiple versions of the same package installed at the same time and allows you to switch between them at will.

* It is trivial to roll back your system after a failed upgrade. Difficult system recovers after you upgrade to a new unstable version are a thing of the past.

* Non-privileged users can install software completely securely.

* Projects packaged with nix have the best possible build reproducibility because nix accounts for ALL of your dependencies all the way down to the lowest level system libraries, compilers, etc.

Each time I hear about NixOS, I am immediately thankful that someone modernized the underlying OS for Nixie clocks (http://nixieshop.com/). Then I come back to reality.
Brings a bunch of fixes to building Rust packages, and it can even be used with the Rust nightly from the Mozilla nixpkgs overlay like this:

    let
	  defaultPkgs = import <nixpkgs> {};
	  defaultRust = defaultPkgs.latest.rustChannels.nightly.rust;
	  defaultCargo = defaultPkgs.latest.rustChannels.nightly.cargo;
	  defaultBuildRustPackage = defaultPkgs.callPackage (import <nixpkgs/pkgs/build-support/rust>) {
		rust = {
			rustc = defaultRust;
			cargo = defaultCargo;
		};
	  };
	in
	  { pkgs ? defaultPkgs, rust ? defaultRust, buildRustPackage ? defaultBuildRustPackage }:

	  pkgs.callPackage ./derivation.nix {
		inherit rust buildRustPackage;
	  }
Running it locally. Looking forward to switching our K8s cluster to 18.03.

Contact me if you want to work with NixOS and K8s in Denmark/Copenhagen. Mail on profile page.

I was tempting to install NixOS on my PC but found out their installation method is fully involved of dozens of manual steps [1]. And I feel exhausted just reading their guide, so I don't want to try at all.

1 - https://nixos.org/nixos/manual/index.html#sec-installation

> The NixOS installer doesn’t do any partitioning or formatting yet, so you need to do that yourself. Use the following commands:

Wow, you are not kidding.

Edit: I could swear this comment was originally replying to someone.

Configuring your partitions and storage is scary and dangerous
On the flip side a friend of mine can literally reinstall her OS from backup in about 15 minutes, including all applications and their configuration.
Why? It's very simple. But if you're afraid of CLI tools, you can always partition with GParted.
To each their own I guess. I strongly prefer their method. It let's me set up my system exactly how I want without having to fight with some GUI installer to make sure it doesn't do anything stupid. And hey, you might even learn a thing or two about how an OS works - partitions, boot loaders, init systems, it's all really quite important to understand when things go wrong and you need to recover your system manually.
I'm fine with setting up my boxes with Debian Installer and its preseed way, and it's straight forward and too easy to configure. But Nix is something else.
Learning how an OS works really is not quite important, except for a very small minority concerned with that field. That defeats the entire purpose of an OS, which is to provide an easy to use interface between hardware and software. I can't imagine telling anyone in my family this, and having them take me seriously. If they even understand what I just said.
And the result is that the general population can't fix even simple issues with their computers, and not even reinstall an OS.

Knowledge is great, but ignorance is bliss, I suppose :P

People shouldn't be responsible for knowing how to fix every piece of technology in their life. I doubt Linus could rebuild his car engine. I mean, everyone should know how to fix their ever increasingly complex car right? We have industrialized farming, so we're not spending our entire lives growing our own food, and hunting our own animals. This is how the world works. Should they know the basics of computer usage? Sure. Maybe if reinstalling a computer was one click of a button they should. But NixOS is so far beyond that, I wouldn't touch it either. I've got plenty of other things to work on, rather than fighting with an OS.
I'm not saying everybody needs to know everything about everything, but I'm really not happy about people's lack of IT knowledge in a world that is only getting even more technical.

People really should know just a little bit about stuff they use every day.

Cars have been around a lot longer than home computers have, and yet most people don't change their own oil.
I say this as a NixOS developer-- realistically the audience for a system like NixOS is fairly technical by design. It will likely never receive the kind of polish that would ever satisfy anyone in your family, or possibly even a technical person like yourself! I dislike that, but it's the truth.

Maybe the other developers disagree, but there are already plenty of Linux distributions that try extremely hard to be friendly and gentle, and remove needless technicalities -- and they even make it simple for developers! They could satisfy you or someone else! The developers put a lot of work into them, and that's really fantastic. But we'll probably never beat them at this; our design is a big enough departure from the traditional that it's very difficult to make it transparent. We're better off playing to our own strengths, IMO, rather than trying to polish off every single rough edge for ever user, to do something Linux distros with 10x the users and money, already offer, but better.

It's pretty understandable a lot of people would not like this trade off. I, personally, think NixOS is personally the best Linux distribution there is, of course -- but it's understandable why people would not like it, or these design choices. At all. I don't think I waste as much time as you might expect learning about Linux, but I do spend time doing things you probably wouldn't ever enjoy (like fixing upstream packages...)

-----

All of this said, we really should have a graphical installer. I'd love one. :) I'm just saying even if we fixed that, it's probably going to be the least of your issues in the long run, more or less... The installer is the tip of the iceberg.

Hey thanks for the response. Of course it's for the target audience, which was my overall message. The person I was responding to was arguing that people in general should know how an OS works, which is complete nonsense.
Rereading my response, I don't think that's at all what I was arguing for. I thought I was quite clear in qualifying the specific situation when I thought it would be helpful to know about lower level details. I'm sorry you interpreted it the way you did.
To be honest, at this point, nix is not for non-technical people. There's little benefit, and the design space hasn't been explored. That being said, nix would make a fantastic basis for a more end-user-oriented system, but the work would need to be put in.
The list is a bit thorough and mixes a few possible scenarios. Basically what you need to do is:

1. Copy the ISO onto a USB drive and boot your laptop on it

2. Run `systemctl start display-manager` to start the GUI

3. Format the harddrive however you like using GParted

3b. EDIT: and mount the root drive to /mnt

4. Run `nixos-generate-config --root /mnt` to detect the hardware layout and generate an initial configuration.nix file

5. Open the /mnt/etc/nixos/configuration.nix file with your editor and edit to your convenience

6. Run `nixos-install`

7. Reboot

Thank you, concision makes it easier for me to understand the process.
I've tried that a couple times on my own HP Proliant Server ML110 Gen9. The installation proceeds perfectly, no error whatsoever but no way I can get it to boot from the HD. I'm finding UEFI quite hard to debug when compared to MBR + GRUB... I find it odd that the pendrive boots fine, the installation happens well but the HD will not boot.

I wish there were more tips for debugging UEFIon their guide.

Agreed. I ran Nix for a while.. I love the concept of it, but I ended up bailing for a similar reason. I desired Nix because I wanted simplicity that immutability (conceptually) brings. Yet, the OS and especially dealing with non-immutable packages/etc were just way too much of a headache.

I left Nix hoping that in a few years it would be far friendlier. It really feels like the future.

I use NixOS daily, and the rough areas now are mostly lack of really thorough documentation and dealing with software that is not packaged for Nix and has an unclean way of getting built. It's surprising how many pieces of software have hardcoded paths to e.g. Python binaries.

My temporary fix for the latter is using a Docker with Arch for quick and dirty one-time things.

> the rough areas now are mostly lack of really thorough documentation and dealing with software that is not packaged for Nix

So basically the same problems as every other Linux distribution? Only AppImage doesn't work out of the box either.

The more things change the more they stay the same.

(comment deleted)
Did you know Flatpak support is on the way? I'm hoping that it can help fill the void for applications which haven't made it into Nixpkgs yet. (And maybe for things we can't build from source, as much as I try to stay away from those.)

https://github.com/NixOS/nixpkgs/pull/33371

Yes, my use-case is more complex sadly. It's more about installing some scientific binaries which are generally really dirty to build, in a quick and easy way.

I need them to reproduce other papers so I cannot stay away, sadly.

I think a better FHS or Steam.Env would help me more.

> and especially dealing with non-immutable packages

Are you saying that NixOS allows packages to cheat? That seems like a serious flaw.

If there's one thing worse than a non-purely-functional system, then it must be a system that claims to be purely-functional but turns out to be non-functional.

The packages themselves can't mutate, the nix store is read-only (except during installation). But some packages needs a global, mutable state (think databases), which is fine. What isn't fine is that some packages expect that mutable space to be in certain paths, and things get messy. Gitlab comes to mind as a service that's been extremely painful to get working under nixos because it expects its install path to be mutable.
There are ways to get around the immutability, but they are very highly discouraged and I haven't seen anybody use them other than newcomers. Think of it like `unsafePerformIO :: IO a -> a` in Haskell.

Namely there is:

- Making the Nix store writable. Normally it's mounted read-only, and only the Nix daemon can write finished deterministic builds there. Often wanted by users who haven't learned how to handle Nix packages, as they just want to change some file in the store.

- Disabling the sandbox. Usually all builds are done in a sandbox which doesn't have network access, read access only to the Nix store and write access only to only the destination directory in the Nix store (with some exceptions), which gives strong reproducability guarantees.

Some people enjoy that. Just look at Arch Linux.
Their install guide seems fairly standard for a minimal Linux distribution. They've even attempted to simplify it compared to what you would have to follow for an Arch install.
this is actually way more clear than the arch install "guide", which seem to have become hostile toward the idea of actually helping people in recent years.
You mean the text file in the home directory when installing? I found it very educational and easy to follow. It tells you how to configure e.g. the locale, the timezone, etc. as bare metal as possible, which makes it more reliable, and useful for other distros. Much more flexible and transparent than having an install wizard like they did before. Installing in a directory with pacstrap also made it very clear that it could be used for making archlinux containers. I love it.
I was referring to the removal of the "Beginner's Guide" in the wiki, now replaced by the "Installation Guide" that has way less context. Which is fine if you already have a nontrivial amount of linux/arch experience and just want to glance at a list of To Dos when you install. But is it really too much "bloat" to have a single example partition layout? I get the community doesn't like having to hand hold people, but sometimes people learn best by just getting things up and running and not reading 15 wiki pages before they're able to understand what should be a straightfoward guide. /rant

I honestly wasn't aware there was a text file in the home directory but it sounds like something pretty close to the Installation Guide, which is helpful and good to know, so thanks for that.

In this vein, are there some people here who use Guix as their daily driver? I'd be interested in hearing experiences, comparisons, etc.

I have both nix and guix stores on my (voidlinux) system just trying them out. Package availability seems quite different between the two, and guix feels like it's heavier or slower, but I am biased to guix's shepard instead of systemd.

Not what the OP asked, but:

Initially, I really wanted to use Guix over Nix, but Nix is more mature with more packages, maintainers, and features.

Last I checked, for example, there was no equivalent of a channel or overlay in Guix. Their npm importer was also not released, whereas Nix has a very usable node2nix.

Having done virtually no functional programming, learning Guile Scheme was also a barrier to me, but the Nix expression language felt a lot more natural for whatever reason.

It's unfortunate - I'm very interested in the work in reproducible builds and substitutions Guix has done, maybe it'll take off at some point or those features will be integrated into NixOS.

Guix has an overlay facility called $GUIX_PACKAGE_PATH[0]. Packages found there will take precedence over distribution packages.

As an end user you don't really need to know Scheme. The OS configuration and package definitions are very "DSL-like", but you do have the full power of Scheme available.

[0] https://www.gnu.org/software/guix/manual/guix.html#Package-M...

(comment deleted)
I use GuixSD on all my systems---servers, workstations, laptops. I also use Guix on a 200+ node bioinformatics HPC cluster at the institute where I work.
I am curious how or where NixOS fits in a a cloud or IaaS environment.

I feel like AMIs and containers address the reproducible builds problem and blue/green deploys the rollback issue. Could NIXOS still be complimentary in an AWS/AMI environment?

You can have thousands of packages in a single Nixos installation and if 200 of those packages directly or indirectly depend on some library L, you usually only need one copy of L to fulfill those 200 dependencies.

In contrast, if you put each package in its own container or VM, then you end up with 200 copies of L.

So, the different technologies have different strengths and weaknesses. One weakness of Nix is that it is harder for most people to learn or to understand than containers or VMs are.

Some people use Nix or Nixos as part of their process for building containers or VM images.

(I wonder whether there is any build system that uses containers instead of file names containing cryptographic hashes (like Nix does) or chroots (like Arch Build system does) to "enforce package isolation" during building.)

Thanks for the replies and the links. Makes sense. Cheers.
Yes. NixOS can be set up to output a ready-to-run AMI with your customizations, as well as any other kind of image. For example, there are official binaries at https://nixos.org/nixos/download.html, which I believe are built by https://github.com/NixOS/nixpkgs/blob/master/nixos/maintaine.... You can then either replace the images as needed, or redeploy in place.

NixOps can also automatically create and destroy EC2 VMs, EBS disks, and so on for you, based on your NixOS config. NixOps also makes it very easy to set up a local copy of your network using Virtualbox.

Personally I use a common baseline image that is preseeded with my SSH key, and then deploy whatever setup I need using NixOps. You can read more about that approach at https://nixos.wiki/wiki/Virtualization_in_NixOS.

Immutability seems like the best thing for a package manager. Do any other packaging systems have interesting “killer features”? I’ve heard of PiSi for Pardus, and Conary for rPath/Foresight, but honestly, I can’t quite keep track of the state of the art.