These specific ideas? To the best of my knowledge, no.
Other related ideas like Nix/NixOS/Guix are solving some of the same problems outlined in the article in a much more systematic way that are not reliant on any one file system.
And also, if I understand correctly, NixOS and Guix need much less help from developers than this proposal would have. Specifically, NixOS and Guix don't care how anyone names anything (except, of course, their dogs).
is anyone using these systems in production? I would immediately apply to work in such a place.
It's a catch-22. The biggest gain in using NixOS/Guix are production systems big enough where it matters. And every place that grows big enough where this would be good for the them they usually roll their own ... Because tech-debt doesn't allow them from switching to another OS + they have so much duct-tape that if they'd replace it, would also have to find work for those doing the duct-taping. I think it's not a technical problem at all, nor do I think it's because we lack ideas.
Yes, there are places that are using NixOS in production. I don't know the details, so this is a placeholder for someone else to fill in - or else ask on the NixOS subreddit or IRC channel.
Nix/Guix is a very simple and elegant design, but I find it has a downside: because every package is immutable, a security fix in a low-level shared library (worst case: glibc) requires recompiling every package that depends on it, and because there is no difference between OS and applications - everything is a Nix package - you have to wait until the whole dependency graph is recompiled before you can effectively install the security update.
TFA's design avoids this by separating out both a base OS layer and an application runtime layer, both of which have a bounded size.
But I'm not aware of any OS built upon btrfs volumes. I never used btrfs, but I heard its performance degrades a lot once you get to more than 100 volumes (or was it 100 snapshots? I'm not sure).
Finally, the proposed system wouldn't help that much with security. Malicious programs would still be able to send your .ssh dir over the Internet.
SLES and openSUSE use Btrfs by default, including snapshots and rollbacks.
Snapshot creation is inexpensive. Snapshot deletion can be expensive depending on the changes since snapshot was taken, but this is workload specific. All subvolumes/snapshots share the same extent tree, so that's where things can get expensive.
> We want a unified scheme, how we can install and update OS images, user apps, runtimes and frameworks.
The storage backend for this is OSTree, using HTTP as the transport; it's analogous to Git for binary files and does not require special filesystem features, only hardlinks.
> We want a unified scheme how you can relatively freely mix OS images, apps, runtimes and frameworks on the same system.
Flatpak provides the abstraction layer for this; the base OS can be anything and the applications use only libraries from a Flatpak runtime; by default, Flatpak uses OSTree to store applications.
Snap is similar, although it requires AppArmor, so can't run on many popular distros, and I think there's only one runtime that you're required to use, so no "mix and match".
Docker is similar, although it can't handle desktop applications in a sensible manner (no sandbox).
> We want a fully trusted system, where cryptographic verification of all executed code can be done, all the way to the firmware, as standard feature of the system.
There are several points here:
* firmware/pre-boot: [Edit: TPM can be used to verify this, but it's not set up that way by default; Windows 10 Bitlocker can rely on TPM for this]
* base OS and Flatpak: OSTree can require GPG signatures for installation; to check the files at runtime, you'd need an additional layer such as dm-verity.
> We want to allow app vendors to write their programs against very specific frameworks, under the knowledge that they will end up being executed with the exact same set of libraries chosen.
FlatPak SDKs provide this.
Snap has an SDK as well, although note the singular, Snap is exclusively controlled by one company.
> We want to allow parallel installation of multiple OSes and versions of them, multiple runtimes in multiple versions, as well as multiple frameworks in multiple versions. And of course, multiple apps in multiple versions.
Flatpak can do multiple framework versions by default, and multiple application versions with command line flags.
Multiple different OSes hasn't happened yet, probably you'd want at least one non-experimental OS first...
> We want everything double buffered (or actually n-fold buffered), to ensure we can reliably update/rollback versions, in particular to safely do automatic updates.
OSTree does this.
> We want a system where updating a runtime, OS, framework, or OS container is as simple as adding in a new snapshot and restarting the runtime/OS/framework/OS container.
OSTree/Flatpak again.
> We want a system where we can easily instantiate a number of OS instances from a single vendor tree, with zero difference for doing this on order to be able to boot it on bare metal/VM or as a container.
No idea if this exists anywhere yet.
> We want to enable Linux to have an open scheme that people can use to build app markets and similar schemes, not restricted to a specific vendor.
Flatpak/FlatHub solve this; Snap does not because it's exclusively controlled by one vendor.
> Flatpak provides the abstraction layer for this; the base OS can be anything and the applications use only libraries from a Flatpak runtime; by default, Flatpak uses OSTree to store applications.
Snaps use squashfs, so instead of a hardlink farm, you have self contained fs images.
> Snap is similar, although it requires AppArmor, so can't run on many popular distros,
I'm not sure why you continue to repeat this factually incorrect information. AppArmor is not required. In fact, guessing you being a Fedora user, you can verify that by installing snapd from your distro repositories.
However AppArmor is used for providing confinement of snaps (aside from sandboxing which requires the same interfaces in the kernel as flatpak or docker do).
> and I think there's only one runtime that you're required to use, so no "mix and match".
Currently there are runtimes based on Ubuntu 16.04, 18.04 and upcoming 20.04. There's some work to support freedesktop runtimes, but it got stuck on defining metadata for describing security profiles of apps using given runtime.
> FlatPak SDKs provide this.
> Snap has an SDK as well, although note the singular, Snap is exclusively controlled by one company.
Wouldn't call it an SDK, but there's a tool called snapcraft to build snaps in a convenient manner. You could just as well use 'snap pack' or mksquashfs directly.
>> Snap is similar, although it requires AppArmor, so can't run on many popular distros,
> I'm not sure why you continue to repeat this factually incorrect information. AppArmor is not required. In fact, guessing you being a Fedora user, you can verify that by installing snapd from your distro repositories.
I'm sorry, what I meant to write is, it doesn't run equally well on many distros; you get a first-class or second-class experience depending on which LSM your distro has chosen to use. This is not the case with Flatpak, because it doesn't use any LSM for sandboxing.
Although I've heard there were attempts to enable LSM stacking in the kernel, which would provide a way to solve the problem; do you know the current status of that?
AFAIK the effort is still ongoing. I believe LWN had some nice coverage on that, with the last majir pain point being attaching a security context to sockets in a way that suits all interested LSMs.
Anyways, I would not expect to have LSM stacking enabled in Fedora or RHEL anytime soon after it lands in mainline, mostly due to political reasons, thought I'd very much like to be surprised when the day comes.
> When a developer wants to develop against a specific runtime he installs the right framework, and then temporarily transitions into a name space where /usris mounted from the framework sub-volume, and /home/$USER from his own home directory. In this name space he then runs his build commands. He can build in multiple name spaces at the same time, if he intends to builds software for multiple runtimes or architectures at the same time.
it reminds me of the Clearcase version control system. You would run `cleartool setview <viewname>` to enter the namespace of that "view". Like Las Vegas, everything you did within that view would stay in that view.
This was before containers and VM's became widespread. It felt a like magic, because you could run one command and see a totally different build and tooling environment. Under the hood it meant that running `make` would pull in objects from other teams who used exactly the same view as you (so like ccache but across the network scaled to N-engineers).
Most companies have thrown clearcase out when svn/git came along because poor administration practices could push license / storage costs through the roof (and it was never cheap to begin with).
We have git/mercurial,... today which I'm quite happy for. But considering how old all these ideas are, it's a shame FOSS is still stuck in the stone age in this regard.
FOSS feels stuck in the stone age for many things, because doing something like ClearCase, world class IDEs or game middleware requires typical corporation level budgets, and since most don't want to pay for tooling, only corporate developers get the nice toys.
Another thing that ClearCase was already great at, 20 years ago, was providing a file system cache for C and C++ builds, providing a mechanism to only compile translation units that had outdated object files across all developer team.
And yet ClearCase was notorious for being the bad old world that everyone wanted to get away from. I'm sure someone will be along with war stories to tell us why the reality wasn't so great.
When I was contractor I worked with two companies that used ClearCase sometime around 2000. In theory it looked really good but it just never worked. The whole dev team probably spent more time on ClearCase than they spent on dev work. It was a never ending nightmare.
:) I was one of those who fought hard for many years so that we could replace it with svn/git, and we eventually did. It was still ahead of it's time back then. But I agree it was a nightmare to administrate and if you used clearcase UCM it was one huge horror show for users. UCM meant your whole process is locked into Rational/IBM's way of doing things (and it had to die because Agile came along so clearcase felt very outdated).
23 comments
[ 2.7 ms ] story [ 34.4 ms ] threadOther related ideas like Nix/NixOS/Guix are solving some of the same problems outlined in the article in a much more systematic way that are not reliant on any one file system.
It's a catch-22. The biggest gain in using NixOS/Guix are production systems big enough where it matters. And every place that grows big enough where this would be good for the them they usually roll their own ... Because tech-debt doesn't allow them from switching to another OS + they have so much duct-tape that if they'd replace it, would also have to find work for those doing the duct-taping. I think it's not a technical problem at all, nor do I think it's because we lack ideas.
TFA's design avoids this by separating out both a base OS layer and an application runtime layer, both of which have a bounded size.
But I'm not aware of any OS built upon btrfs volumes. I never used btrfs, but I heard its performance degrades a lot once you get to more than 100 volumes (or was it 100 snapshots? I'm not sure).
Finally, the proposed system wouldn't help that much with security. Malicious programs would still be able to send your .ssh dir over the Internet.
Snapshot creation is inexpensive. Snapshot deletion can be expensive depending on the changes since snapshot was taken, but this is workload specific. All subvolumes/snapshots share the same extent tree, so that's where things can get expensive.
This is from 2017, I'm sure there are other more recent anecdotes, and Btrfs performance has improved since then also, in particular qgroups. https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZ...
However if you look at the design goals, most of them have been implemented, although there is not yet wide usage.
The best way to try this out in a runable desktop OS is probably Silverblue.
https://silverblue.fedoraproject.org/
> We want a unified scheme, how we can install and update OS images, user apps, runtimes and frameworks.
The storage backend for this is OSTree, using HTTP as the transport; it's analogous to Git for binary files and does not require special filesystem features, only hardlinks.
https://ostree.readthedocs.io/en/latest/
> We want a unified scheme how you can relatively freely mix OS images, apps, runtimes and frameworks on the same system.
Flatpak provides the abstraction layer for this; the base OS can be anything and the applications use only libraries from a Flatpak runtime; by default, Flatpak uses OSTree to store applications.
Snap is similar, although it requires AppArmor, so can't run on many popular distros, and I think there's only one runtime that you're required to use, so no "mix and match".
Docker is similar, although it can't handle desktop applications in a sensible manner (no sandbox).
> We want a fully trusted system, where cryptographic verification of all executed code can be done, all the way to the firmware, as standard feature of the system.
There are several points here:
* firmware/pre-boot: [Edit: TPM can be used to verify this, but it's not set up that way by default; Windows 10 Bitlocker can rely on TPM for this]
* bootloader: UEFI Secure Boot handles this
* kernel/initramfs: I'm afraid this isn't typically verified currently, see https://mjg59.dreamwidth.org/54203.html
* base OS and Flatpak: OSTree can require GPG signatures for installation; to check the files at runtime, you'd need an additional layer such as dm-verity.
> We want to allow app vendors to write their programs against very specific frameworks, under the knowledge that they will end up being executed with the exact same set of libraries chosen.
FlatPak SDKs provide this.
Snap has an SDK as well, although note the singular, Snap is exclusively controlled by one company.
> We want to allow parallel installation of multiple OSes and versions of them, multiple runtimes in multiple versions, as well as multiple frameworks in multiple versions. And of course, multiple apps in multiple versions.
Flatpak can do multiple framework versions by default, and multiple application versions with command line flags.
Multiple different OSes hasn't happened yet, probably you'd want at least one non-experimental OS first...
> We want everything double buffered (or actually n-fold buffered), to ensure we can reliably update/rollback versions, in particular to safely do automatic updates.
OSTree does this.
> We want a system where updating a runtime, OS, framework, or OS container is as simple as adding in a new snapshot and restarting the runtime/OS/framework/OS container.
OSTree/Flatpak again.
> We want a system where we can easily instantiate a number of OS instances from a single vendor tree, with zero difference for doing this on order to be able to boot it on bare metal/VM or as a container.
No idea if this exists anywhere yet.
> We want to enable Linux to have an open scheme that people can use to build app markets and similar schemes, not restricted to a specific vendor.
Flatpak/FlatHub solve this; Snap does not because it's exclusively controlled by one vendor.
Snaps use squashfs, so instead of a hardlink farm, you have self contained fs images.
> Snap is similar, although it requires AppArmor, so can't run on many popular distros,
I'm not sure why you continue to repeat this factually incorrect information. AppArmor is not required. In fact, guessing you being a Fedora user, you can verify that by installing snapd from your distro repositories.
However AppArmor is used for providing confinement of snaps (aside from sandboxing which requires the same interfaces in the kernel as flatpak or docker do).
> and I think there's only one runtime that you're required to use, so no "mix and match".
Currently there are runtimes based on Ubuntu 16.04, 18.04 and upcoming 20.04. There's some work to support freedesktop runtimes, but it got stuck on defining metadata for describing security profiles of apps using given runtime.
> FlatPak SDKs provide this.
> Snap has an SDK as well, although note the singular, Snap is exclusively controlled by one company.
Wouldn't call it an SDK, but there's a tool called snapcraft to build snaps in a convenient manner. You could just as well use 'snap pack' or mksquashfs directly.
> I'm not sure why you continue to repeat this factually incorrect information. AppArmor is not required. In fact, guessing you being a Fedora user, you can verify that by installing snapd from your distro repositories.
I'm sorry, what I meant to write is, it doesn't run equally well on many distros; you get a first-class or second-class experience depending on which LSM your distro has chosen to use. This is not the case with Flatpak, because it doesn't use any LSM for sandboxing.
Although I've heard there were attempts to enable LSM stacking in the kernel, which would provide a way to solve the problem; do you know the current status of that?
Anyways, I would not expect to have LSM stacking enabled in Fedora or RHEL anytime soon after it lands in mainline, mostly due to political reasons, thought I'd very much like to be surprised when the day comes.
https://facebookmicrosites.github.io/btrfs/docs/btrfs-facebo...
2015: https://news.ycombinator.com/item?id=9477612
Discussed at the time: https://news.ycombinator.com/item?id=8251288
I guess the only thing you need is total control of all major distros to change sentiment.
it reminds me of the Clearcase version control system. You would run `cleartool setview <viewname>` to enter the namespace of that "view". Like Las Vegas, everything you did within that view would stay in that view.
This was before containers and VM's became widespread. It felt a like magic, because you could run one command and see a totally different build and tooling environment. Under the hood it meant that running `make` would pull in objects from other teams who used exactly the same view as you (so like ccache but across the network scaled to N-engineers).
Most companies have thrown clearcase out when svn/git came along because poor administration practices could push license / storage costs through the roof (and it was never cheap to begin with).
We have git/mercurial,... today which I'm quite happy for. But considering how old all these ideas are, it's a shame FOSS is still stuck in the stone age in this regard.
[1] https://www.ibm.com/support/knowledgecenter/SSSH27_8.0.1/com...
Another thing that ClearCase was already great at, 20 years ago, was providing a file system cache for C and C++ builds, providing a mechanism to only compile translation units that had outdated object files across all developer team.