Interesting. Anyone familiar with it? I'm curious if it's container per application, or container per distro supported. Having never run a system like this, how do you manage files? If I have samba from Debian and nginx from Arch, how does one edit their respective configs and such?
Not vanillaos, but bluefin/silverblue. You can manage many user level programs via toolbox or distrobox in a similiar way as a regular distro, but the packages are installed in your $HOME. You can still use regular containers as well too. Although you can still add extra system level stuff if you desire via layering (or soon via systemd sysext)
Ive read the source code of apx (the package manager) and its basically just a distrobox wrapper. From what I understand, It creates a container for each distribution and packages are installed in that container.
Not much familiar with it, but familiar with the concept: it's a fail.
If you want a comfy modern system goes for NixOS/Guix System on top of zfs. Safety came from design, not from isolation. Comfort came from autonomous power not quickly import third parties blob and so on. Such distros are like btrfs vs zfs. A tentative to denied the fact a model needed by some actors is total crap.
Filesystem is automatically mounted to the container. I believe this one use LXC through distrobox, Fedora silverblue does the same with toolbox which used podman.
If you want to give it a try you can install distrobox or toolbox right now on any distro, this is not unique to immutable readonly distros. This allows you to separate system / userland package management and have access to say, pacman on a fedora for example.
For firefox I've found it extremely annoying. When the snap updated in the background:
1. Pinning the desktop icon to the toolbar is annoying ... the desktop file is in a snap folder that keeps changing, so would disappear.
2. It showed the "you need to restart firefox in X days" notification frequently; I wasn't able to find a way to stop it reshowing the notification after dismissing it.
3. Updates caused firefox to crash if the underlying snap folder gets removed.
4. Keeping profile information across the snaps is not easy; I've had updates reset the profiles to the base.
For IntelliJ IDE I've not had issues thus far. I'm not sure if this is because it hasn't been updated enough to encounter the above issues.
Snaps use a compressed disk image, which is uncompressed and mounted the first time the process is run after startup. All of them are slower on first startup than their non-snap counterparts.
And unmounting these during shutdown can cause the shutdown to take a minute longer than expected. That may have been a bug that was fixed since I found out about it, though.
I've recently moved my laptop over to openSUSE MicroOS (specifically Kalpa, the KDE variant). It shares a bit of philosophy with Vanilla OS. However, in many ways it's almost unrecognizable as a Linux/Unix system.
The way it works (in my incomplete understanding) is that the root filesystem (running on Btfrs), specifically /usr and /var, are actually a read-only image. You can write changes to it, but each time you do, you have to rewrite the image. Each time you boot, you boot that immutable image.
This allows for easy automatic updates. And if it fails to boot after an update, it merely rolls back to the previous working image (crowdstrike take note). This seems to work well provided you don't have to modify the image too much. I installed fprintd, kdeconnect, and wine, and it's still doing OK.
The user applications are almost all Flatpaks. This works well most of the time, but not always. I was a heavy flatpak user before, and I will say that a number of Flatpsk bugs I've run into on other systems, I've not had on Kalpa, so perhaps its Flatpak implementation is better. The biggest issues I have are Flatpaks not being able to communicate with each other as easily as native binaries can.
If you don't have a Flatpak, you can always try to run it in DistroBox. This works..... OK....provided it's a userspace app. But if it's a userspace app, why not run the binary directly? Where distrobox really shines is for running .debs or .rpms on a non-native system. But those are gradually going away thanks to Flatpak anyways. Distrobox does have a fake root mode. I consistently run into boubdary issues with it on Kalpa. I was able to run software in distrobox that required root, and it technically ran, but it couldn't use any audio devices.
Overall, I find Kalpa (and MicroOS) very interesting. There are still edge cases where they break, but I was easily able to work around everything.
I use a docker image as my whole development environment and the experience is similar. I can do whatever I want inside the container, and all will gone after I restart it. For the changes I like it, I edit the dockerfile and build a new image. When I get a new machine, I just install docker and pull the image and start coding.
Also found MicroOS interesting. At first I felt limited but more and more I started to like the flow of doing things and being more pragmatic about my system.
In the end though, some of the cons of such a system started being too much and I went back to Leap. It was close to winning over me however.
I have also a laptop on silverblue, which is the immutable version of fedora. Most of the time usage is identical but once in a while you encounter an edge case that is easily solved on regular fedora workstation where you have to find extra weird workarounds to simple stuff.
Also while flatpak applications are usually sandboxed, they very often need access to your files to be useful. You are either limited to a small subset of flatpaks on fedora flatpak repo, or the whole uncurated flathub shithole where quality and security is totally random.
Also running a flatpak from the command line using `flatpak run tld.organizationname.appname` is a bit annoying. Toolbox, distrobox and containers solve some of the stuff but are also clunky way to do stuff that you would just do normally on a regular linux system.
All in all I will keep it on one of my laptops as I am curious about how it will evolve in the future and it is a decent OS for non tech / family use where your typical usage is to open a browser and a handful of gui apps so I can easily lend it to my kids and partner without worry.
Fedora Kinoite (like Silverblue, but with KDE in place of Gnome) user here. I agree with most of your points, though I've actually grown to appreciate the Distrobox/Flatpak workflow once I grew accustomed to them, to the point where I have no interest in returning to a traditional distro.
I've even started using Fedora CoreOS VMs as the basis for containerized service installs on my home network (Pihole, etc.).
To be fair, I do have quite a few packages layered on top of the base distro, as well. From memory: many admin tools that require "real" root, KVM virtualization support, RPM Fusion packages to enable hardware video decode, Mullvad's VPN client, tmux, vim-default-editor, a few font packages, Emacs, and a few basic development tools like cmake and make for the benefit of Emacs package installs.
The only problem I've ever had with layering is that once in a while I have to wait a bit and retry an update because newer package versions from the base image haven't yet made it out to the main RPM mirrors.
Oh, and Flatpak automatically symlinks "flatpak run" wrapper scripts to /var/lib/flatpak/exports/bin, so, assuming ~/.local/bin is in your PATH,
> The way it works (in my incomplete understanding) is that the root filesystem (running on Btfrs), specifically /usr and /var, are actually a read-only image. You can write changes to it, but each time you do, you have to rewrite the image. Each time you boot, you boot that immutable image.
Sounds like a glorified LiveCD distro. (In fact, it's almost exactly a LiveCD with 'persistence' once you account for other parts of the filesystem that are not 'immutable', such as /home/ .) Not sure what's supposed to be so innovative about this.
Ehm... The old OpenSolaris (even before IllumOS) have had "boot environments" based on zfs clones, on GNU/Linux NixOS/Guix System offer "generations" with a network of symlinks to obtain the same result. This distro honestly sound like btrfs: an answer of those fanatically convinced their model is right and others really different are wrong.
I state this because btrfs/stratis vs zfs "a rampant layer violation" (for those who remember) was the perfect example of a devs reactionary cohort who try, fails and even fails to recognize their failure, trying to state that anything new is bad, their old beloved way is the best.
Honestly package managers, boot process and installers are RELIC from another era and most fails to understand that. Declarative systems on top of modern storage like zfs (witch is modern since the others are stuck in the '80s even if born after zfs) or hammer (DragonFly BSD log-based fs like the old Linux nilfs2, but with much less of it's garbage collection issues and much more useful than "modern" ffs used on Android and alike) are "the future" since more than a decades but most try to ignore that sticking with relics and some try to denied they are the future creating monsters from stratis to docker, passing through snap/appimage/flatpacks etc all do denied a substantial change who simplify ENORMOUSLY anything just because such revolution will empower users and operations instead of the service industries and devs under Silicon Valley mode dummy-Toyoda-alike managers slavery command.
If you do not understand try to visualize how simple is to build a custom ISO with NixOS/Guix System, describe an easy to replicate system, orchestrate your hosts and handling their storage with zfs send-able snapshots. If you succeed you'll understand that AT HOME, on much smaller iron, you can do more than a k8s deploy, with much more resilience and much less effort. Now try to imaging it on scale: how many companies will remain on someone else computer (aka the cloud) with such systems common and spread?
Well, the result is kind of similar, but the way of doing it is different.
IIRC OpenSolaris (and other derivations) had a writable root but did ZFS snapshots on changes, kind of similar as Ubuntu was (crudely) doing with ZSys for a while. I would rather use zfsbootmenu in Linux.
In this case (and other similar distros), it has a read only root, and does overlays over it and/or running things inside containers and namespaces. That also allows sandboxing.
I haven't thought much about what system I like more. ZFS is easier to me, of course, as I already use it and don't have to change my workflow ¯\_(ツ)_/¯
> IIRC OpenSolaris (and other derivations) had a writable root but did ZFS snapshots on changes,
Not exact, there is SUN zfs auto-snapshot service, but that's for another purpose, a time-machine alike setup at a certain point in time even integrated with nautilus with a right-click menus to see the $volRoot/.zfs/snapshots/* of a specific file/folder, the package manage do clones so you can create different roots and keep or dispose them independently as long as you want. GNU/Linux scripts to wrapper grub-install and package managers try to mimic that but that's not on feature parity at all. Yes, in OpenSolaris/IllumOS the root was/is not read-only, but if you manage the system with IPS it's nearly that, "only" lacking respect of NixOS/Guix System the declarative configuration since IPS (the package manager) was a classic manual one.
NixOS/Guix System lack the zfs integration, using a network of symlinks to mount a read-only root out of /{gnu,nix}/store.
Doing something like containers, even if done "the right way" (meaning without wasting resources, something not easy on x86 and AFAIK not done by anyone so far) it's pointless because it's a false "security" model typically led to far less safe systems. Even if done properly like OpenSolaris/IllumOS zones, who do not waste much resources at least for storage thanks to zfs dedup, they only waste ram/cpu, it's not a good practice.
A safe system is one designed to be safe in every bit, not one were you create "shells", compartments to "keep something potentially unsafe in a sandbox". All have tried this approach have failed.
IMHO for personal workstations immutable distros are a solution in search of a problem.
In 3 years using Fedora (which hasn't a reputation for being a stable distro) I once had a bad kernel that prevented my Framework laptop from booting (solved by blacklisting said kernel). All my other Fedora machines were fine.
Why would I need an immutable distro if even Fedora is stable enough? Heck i could use Debian or a RHEL clone and never have to worry about stability.
For me it's exactly the opposite: I like numbered versions and I hate this Ubuntu/Debian/appleOS idiocy with naming versions like barking bore or lucrative lynx or Samara or any other crap like this where you need to constantly google the whole naming series to get the idea which one is newer.
Nothing is new under the sun. Reminds me of the good ol' CoreOS (the original, not the fedora-flavored, although it also applies), Container-Optimized OS from GCP and the likes of the new wave with Talos and Bottlerocket.
My only wish is that immutable filesystem, read-only rootfs and most of the system with just a FEW exceptions, atomic upgrades/rollbacks would be more and more widely encouraged, discussed and adopted.
Just eliminating the worry of root partition running out of space is a life changer. From the ground up any "traditional" linux distro will force you sit and design how to partition the disk sooner or later. The guys above address this issue (and a huge bunch of others) for you.
I’ve made a /data partition for many years now taking the majority of the disk. Root then has not needed more than thirty GB or so. (With kernels in the efi system partition, and home in data even less. I used to make it 32gb.)
The immutable root seems like it'd be troublesome. How do you end up with workable little dev environments on this sort of immutable root setup? NixOS has a similar benefit without immutable roots, and even allows for development shells per project which is... absolutely fantastic.
35 comments
[ 2.7 ms ] story [ 82.2 ms ] threadUser filesystem is totally transparent.
If you want a comfy modern system goes for NixOS/Guix System on top of zfs. Safety came from design, not from isolation. Comfort came from autonomous power not quickly import third parties blob and so on. Such distros are like btrfs vs zfs. A tentative to denied the fact a model needed by some actors is total crap.
If you want to give it a try you can install distrobox or toolbox right now on any distro, this is not unique to immutable readonly distros. This allows you to separate system / userland package management and have access to say, pacman on a fedora for example.
No loss here. I'd consider it a benefit in fact.
For firefox I've found it extremely annoying. When the snap updated in the background:
1. Pinning the desktop icon to the toolbar is annoying ... the desktop file is in a snap folder that keeps changing, so would disappear.
2. It showed the "you need to restart firefox in X days" notification frequently; I wasn't able to find a way to stop it reshowing the notification after dismissing it.
3. Updates caused firefox to crash if the underlying snap folder gets removed.
4. Keeping profile information across the snaps is not easy; I've had updates reset the profiles to the base.
For IntelliJ IDE I've not had issues thus far. I'm not sure if this is because it hasn't been updated enough to encounter the above issues.
And unmounting these during shutdown can cause the shutdown to take a minute longer than expected. That may have been a bug that was fixed since I found out about it, though.
The way it works (in my incomplete understanding) is that the root filesystem (running on Btfrs), specifically /usr and /var, are actually a read-only image. You can write changes to it, but each time you do, you have to rewrite the image. Each time you boot, you boot that immutable image.
This allows for easy automatic updates. And if it fails to boot after an update, it merely rolls back to the previous working image (crowdstrike take note). This seems to work well provided you don't have to modify the image too much. I installed fprintd, kdeconnect, and wine, and it's still doing OK.
The user applications are almost all Flatpaks. This works well most of the time, but not always. I was a heavy flatpak user before, and I will say that a number of Flatpsk bugs I've run into on other systems, I've not had on Kalpa, so perhaps its Flatpak implementation is better. The biggest issues I have are Flatpaks not being able to communicate with each other as easily as native binaries can.
If you don't have a Flatpak, you can always try to run it in DistroBox. This works..... OK....provided it's a userspace app. But if it's a userspace app, why not run the binary directly? Where distrobox really shines is for running .debs or .rpms on a non-native system. But those are gradually going away thanks to Flatpak anyways. Distrobox does have a fake root mode. I consistently run into boubdary issues with it on Kalpa. I was able to run software in distrobox that required root, and it technically ran, but it couldn't use any audio devices.
Overall, I find Kalpa (and MicroOS) very interesting. There are still edge cases where they break, but I was easily able to work around everything.
In the end though, some of the cons of such a system started being too much and I went back to Leap. It was close to winning over me however.
Also while flatpak applications are usually sandboxed, they very often need access to your files to be useful. You are either limited to a small subset of flatpaks on fedora flatpak repo, or the whole uncurated flathub shithole where quality and security is totally random.
Also running a flatpak from the command line using `flatpak run tld.organizationname.appname` is a bit annoying. Toolbox, distrobox and containers solve some of the stuff but are also clunky way to do stuff that you would just do normally on a regular linux system.
All in all I will keep it on one of my laptops as I am curious about how it will evolve in the future and it is a decent OS for non tech / family use where your typical usage is to open a browser and a handful of gui apps so I can easily lend it to my kids and partner without worry.
I've even started using Fedora CoreOS VMs as the basis for containerized service installs on my home network (Pihole, etc.).
To be fair, I do have quite a few packages layered on top of the base distro, as well. From memory: many admin tools that require "real" root, KVM virtualization support, RPM Fusion packages to enable hardware video decode, Mullvad's VPN client, tmux, vim-default-editor, a few font packages, Emacs, and a few basic development tools like cmake and make for the benefit of Emacs package installs.
The only problem I've ever had with layering is that once in a while I have to wait a bit and retry an update because newer package versions from the base image haven't yet made it out to the main RPM mirrors.
Oh, and Flatpak automatically symlinks "flatpak run" wrapper scripts to /var/lib/flatpak/exports/bin, so, assuming ~/.local/bin is in your PATH,
fixes your annoyance.Sounds like a glorified LiveCD distro. (In fact, it's almost exactly a LiveCD with 'persistence' once you account for other parts of the filesystem that are not 'immutable', such as /home/ .) Not sure what's supposed to be so innovative about this.
I state this because btrfs/stratis vs zfs "a rampant layer violation" (for those who remember) was the perfect example of a devs reactionary cohort who try, fails and even fails to recognize their failure, trying to state that anything new is bad, their old beloved way is the best.
Honestly package managers, boot process and installers are RELIC from another era and most fails to understand that. Declarative systems on top of modern storage like zfs (witch is modern since the others are stuck in the '80s even if born after zfs) or hammer (DragonFly BSD log-based fs like the old Linux nilfs2, but with much less of it's garbage collection issues and much more useful than "modern" ffs used on Android and alike) are "the future" since more than a decades but most try to ignore that sticking with relics and some try to denied they are the future creating monsters from stratis to docker, passing through snap/appimage/flatpacks etc all do denied a substantial change who simplify ENORMOUSLY anything just because such revolution will empower users and operations instead of the service industries and devs under Silicon Valley mode dummy-Toyoda-alike managers slavery command.
If you do not understand try to visualize how simple is to build a custom ISO with NixOS/Guix System, describe an easy to replicate system, orchestrate your hosts and handling their storage with zfs send-able snapshots. If you succeed you'll understand that AT HOME, on much smaller iron, you can do more than a k8s deploy, with much more resilience and much less effort. Now try to imaging it on scale: how many companies will remain on someone else computer (aka the cloud) with such systems common and spread?
IIRC OpenSolaris (and other derivations) had a writable root but did ZFS snapshots on changes, kind of similar as Ubuntu was (crudely) doing with ZSys for a while. I would rather use zfsbootmenu in Linux.
In this case (and other similar distros), it has a read only root, and does overlays over it and/or running things inside containers and namespaces. That also allows sandboxing.
I haven't thought much about what system I like more. ZFS is easier to me, of course, as I already use it and don't have to change my workflow ¯\_(ツ)_/¯
Not exact, there is SUN zfs auto-snapshot service, but that's for another purpose, a time-machine alike setup at a certain point in time even integrated with nautilus with a right-click menus to see the $volRoot/.zfs/snapshots/* of a specific file/folder, the package manage do clones so you can create different roots and keep or dispose them independently as long as you want. GNU/Linux scripts to wrapper grub-install and package managers try to mimic that but that's not on feature parity at all. Yes, in OpenSolaris/IllumOS the root was/is not read-only, but if you manage the system with IPS it's nearly that, "only" lacking respect of NixOS/Guix System the declarative configuration since IPS (the package manager) was a classic manual one.
NixOS/Guix System lack the zfs integration, using a network of symlinks to mount a read-only root out of /{gnu,nix}/store.
Doing something like containers, even if done "the right way" (meaning without wasting resources, something not easy on x86 and AFAIK not done by anyone so far) it's pointless because it's a false "security" model typically led to far less safe systems. Even if done properly like OpenSolaris/IllumOS zones, who do not waste much resources at least for storage thanks to zfs dedup, they only waste ram/cpu, it's not a good practice.
A safe system is one designed to be safe in every bit, not one were you create "shells", compartments to "keep something potentially unsafe in a sandbox". All have tried this approach have failed.
https://vermaden.wordpress.com/wp-content/uploads/2018/11/nl...
In 3 years using Fedora (which hasn't a reputation for being a stable distro) I once had a bad kernel that prevented my Framework laptop from booting (solved by blacklisting said kernel). All my other Fedora machines were fine.
Why would I need an immutable distro if even Fedora is stable enough? Heck i could use Debian or a RHEL clone and never have to worry about stability.
"50% power reliability is enough for anyone, I sometimes don't have to gather firewood"
Its hard to imagine never gathering firewood to heat your home in that reality.
All of this to say that you can make more assumptions and enable things that were not possible before with better reproducibility.
I eschew complexity wherever practical. There's so much complexity in modern life, especially for tech people.
Right now mutable Linux is absolutely fine for me, but I'd like to thank all the people that are alpha testing some of the tech I'll adopt later ;)
My only wish is that immutable filesystem, read-only rootfs and most of the system with just a FEW exceptions, atomic upgrades/rollbacks would be more and more widely encouraged, discussed and adopted.
Just eliminating the worry of root partition running out of space is a life changer. From the ground up any "traditional" linux distro will force you sit and design how to partition the disk sooner or later. The guys above address this issue (and a huge bunch of others) for you.
I’ve made a /data partition for many years now taking the majority of the disk. Root then has not needed more than thirty GB or so. (With kernels in the efi system partition, and home in data even less. I used to make it 32gb.)