I use it for a home server. Zones provides a secure way to have (on one physical machine with one physical network interface) some stuff you can only get to on the local network and some things you can get to over the public internet and some things via internet if you have the right ssh key. Each contained natively from each other. Crossbow firewall provides a nice way to contain traffic securely as well. ZFS let me set up two big external usb drives as a raid array, the resulting zvol can (iirc) have multiple filesystems for use by multiple zones although I only use it from one right now for the lan only zone. That zone shares via SMB to my network so I can use it for backups and media streaming.
I’ve been able to do almost everything in native zones. I had a bhyve zone set up to run a photo related GitHub code base that really needed Linux.
SMF is a joy to use for services and package management with pkgsrc is great. The whole thing just feels very thoughtfully put together.
You can probably achieve all this on Linux with docker and the right iptables (or whatever succeeded it) config I imagine? But on smartos I am using facilities that are integrated deeply into the os going back like 20 years now. I also just prefer the old sun stuff.
> SmartOS is a "live OS", it is always booted via PXE, ISO, or USB Key and runs entirely from memory, allowing the local disks to be used entirely for hosting virtual machines without wasting disks for the root OS.
Does anyone know if something like this is possible with Proxmox? I've got three servers I'm thinking of setting up as a small cluster and would like to boot them from a single image instead of manually setting PVE on each. Ansible or salt is an option but that tends to degrade over time.
wait this seems totally awesome? i hadnt remembered until reading the comments now that this was a joyent thing, and that somehow it has largely disappeared despite seeming like an awesome way to do all sorts of things.
I was intrigued by the idea that in the Manta object store you could schedule computations on the storage nodes. However I am not sure how much improvement that brings in practice. Any practical experience with this?
I am a huge fan of SmartOS. Back in the 2010s (around 2012), I was advocating its use in production at a small startup I worked. The SunOS kernel, ZFS, zero install, immutable core, convenient way to manage containers and VMs together - all of this looked great on paper, especially containers.
In reality, I ended up running almost everything in VMs. The only thing worked well natively was nginx. MongoDB, Mysql, even our php backend (some libraries) had issues, unfortunately.
A year ago, I considered SmartOS again as a home lab driver, and no success again, Linux just has better support: drivers, pci passthrough, etc... and now with containers+vm through Proxmox or anything else. You can even run a k8s+kubevirt with zfs practically out of the box as a complete overkill though.
You can get some of that with IncusOS (https://linuxcontainers.org/incus-os/introduction/), which includes ZFS, immutability and manages both containers and VMs. I haven't used the OS yet, but have been enjoying Incus + Ubuntu.
Not really ... "IncusOS is built on top of Debian 13". It has some similarities (immutable, ZFS, containers & VMs etc), but it isn't a derivative of Solaris.
I’m confused by the wording “without wasting disks for the base OS” - I wouldn’t normally consider this a “waste”, would anyone else? There are big downsides to running off of a USB key all the time unless I’m missing something
The biggest downside of running off of a USB key, is that it's super unreliable.
How exactly does it make any sense to use ECC memory and ZFS RAID for error correction and redundancy, but then rely on the modern floppy disk for the OS itself?
SmartOS has grown a web UI in the last couple years, too. I haven't gotten around to trying it out on my last remaining SmartOS homelab box. I enjoy the CLI tooling very much. For some, though, the web UI might be worthwhile: https://docs.smartos.org/web-interface/
Glad to see SunOS/Solaris still alive in some form in the Open Source space. Had heard of Illumos and OpenIndiana, but didn't know about SmartOS. It's definitely won me over with the NetBSD pkgsrc package manager... Very nice.
I ran a fairly large SmartOS with project-fifo.net as the controller for a number of years . We also had the amusing time with Pluribus networks switching too. When they started they were also a Illumos based switchos . Bhyve on illumos came out of Pluribus and happened partly due to a number of FreeBSD users heckling the "SUN GODS" over why the official kvm port was a piece of shit .
In any case in the 6 years os smartos we never had any dataloss from failed disks, sure fifo and smartos had their warts but lx-zones works amazing well and i think we got Garrett D'Amore to go back to BSD land for some time . In the end we had to jump to VMWARE when Heinz gave up on fifo.
Immutability and reproducibility is great. Depending on unreliable and antiquated hardware, like the USB key sticks, is not.
Who exactly has the environment where you can add, let alone promptly repair/replace, USB key sticks, on your server? Or run PXE when you have just a single server? How exactly do you do that in Hetzner or OVH? Let alone any other service where you get just a single dedicated server or two.
So, we're big enough to have our own quarter-rack in a collocation facility, let's do PXE. Now you have to have a whole separate infrastructure server, just for your other servers to be able to boot properly? (And how exactly does that server itself boot?) Plus, have an extra infra server for redundancy?
Sorry, but this is the reason noone would use SmartOS. You can't build a fortress on such a shaky foundation.
It's simply out of touch with the target market. At least with FreeBSD or OpenBSD, you known it'll just work™ on any single server, as long as serial console access is available, which is standard-enough. Going against the mainstream of Linux is already hard-enough, there's no reason to make it any harder.
SmartOS sounds like a lot of work, for negligible or even negative benefit.
There's zero good reasons why any machine with 450GB+ of zfs-backed redundant storage, needs to rely on USB keys or networking, in order to function properly. There's a reason Samsung's Joyent entirely abandoned and divested of SmartOS, because this sort of over-engineered mentality, simply doesn't compute. It prevents all sorts of usecases, and even with a growth mindset, still prevents the organic growth from a couple of servers to a rack and more.
24 comments
[ 524 ms ] story [ 450 ms ] threadJoyent, the company behind SmartOS, was since acquired, and I don’t usually see anyone talking about SmartOS nowadays.
Is anyone on HN using SmartOS these days?
I’ve been able to do almost everything in native zones. I had a bhyve zone set up to run a photo related GitHub code base that really needed Linux.
SMF is a joy to use for services and package management with pkgsrc is great. The whole thing just feels very thoughtfully put together.
You can probably achieve all this on Linux with docker and the right iptables (or whatever succeeded it) config I imagine? But on smartos I am using facilities that are integrated deeply into the os going back like 20 years now. I also just prefer the old sun stuff.
Does anyone know if something like this is possible with Proxmox? I've got three servers I'm thinking of setting up as a small cluster and would like to boot them from a single image instead of manually setting PVE on each. Ansible or salt is an option but that tends to degrade over time.
lots of other stuff will do the "boot from single image" part...say...https://fogproject.org/
https://apidocs.tritondatacenter.com/manta/index.html
Code + issues are active under https://github.com/TritonDataCenter (smartos-live, illumos-joyent, triton, etc.), and docs are at https://docs.smartos.org/.
SmartOS is released every two weeks, and Triton is released every 8 weeks -- see https://www.tritondatacenter.com/downloads
And Triton object storage will have S3 support in the next release!
[edit: removed semicolon from link!]
In reality, I ended up running almost everything in VMs. The only thing worked well natively was nginx. MongoDB, Mysql, even our php backend (some libraries) had issues, unfortunately.
A year ago, I considered SmartOS again as a home lab driver, and no success again, Linux just has better support: drivers, pci passthrough, etc... and now with containers+vm through Proxmox or anything else. You can even run a k8s+kubevirt with zfs practically out of the box as a complete overkill though.
How exactly does it make any sense to use ECC memory and ZFS RAID for error correction and redundancy, but then rely on the modern floppy disk for the OS itself?
In any case in the 6 years os smartos we never had any dataloss from failed disks, sure fifo and smartos had their warts but lx-zones works amazing well and i think we got Garrett D'Amore to go back to BSD land for some time . In the end we had to jump to VMWARE when Heinz gave up on fifo.
snarl/howl/chunter
https://project-fifo.net/ https://www.arista.com/en/support/pluribus-resources
Who exactly has the environment where you can add, let alone promptly repair/replace, USB key sticks, on your server? Or run PXE when you have just a single server? How exactly do you do that in Hetzner or OVH? Let alone any other service where you get just a single dedicated server or two.
So, we're big enough to have our own quarter-rack in a collocation facility, let's do PXE. Now you have to have a whole separate infrastructure server, just for your other servers to be able to boot properly? (And how exactly does that server itself boot?) Plus, have an extra infra server for redundancy?
Sorry, but this is the reason noone would use SmartOS. You can't build a fortress on such a shaky foundation.
It's simply out of touch with the target market. At least with FreeBSD or OpenBSD, you known it'll just work™ on any single server, as long as serial console access is available, which is standard-enough. Going against the mainstream of Linux is already hard-enough, there's no reason to make it any harder.
SmartOS sounds like a lot of work, for negligible or even negative benefit.
There's zero good reasons why any machine with 450GB+ of zfs-backed redundant storage, needs to rely on USB keys or networking, in order to function properly. There's a reason Samsung's Joyent entirely abandoned and divested of SmartOS, because this sort of over-engineered mentality, simply doesn't compute. It prevents all sorts of usecases, and even with a growth mindset, still prevents the organic growth from a couple of servers to a rack and more.