367 comments

[ 3.7 ms ] story [ 330 ms ] thread
Use the best tool for the job I say. If it’s BSD for you or your project then so be it. If it’s Linux then use that.
And use this article to figure out whether BSD is better than Linux for you or your project, right?
I was just trying to get ahead of the inevitable flame war that will ensue regarding this topic.

I didn’t know about some features in BSD though so I did find the article useful.

As a longtime user of both FreeBSD and various Linux distros, I agree with a lot of points regarding the coherence and ergonomics of FreeBSD’s design. However, I don’t generally run FreeBSD anymore, despite enjoying using it more, simply for the reason of security. I don’t feel (and maybe this is just a feeling) that security is a first class citizen on FreeBSD the way it is in Linux distros like Fedora or Ubuntu. Things can be slow to be patched and the defaults aren’t always particularly secure. I really miss ZFS though. Btrfs just isn’t as nice to work with.
Really? Here I thought FreeBSD puts security before everything. This comment is surprising to me.
Yeah I don’t think one or the other is inherently better but Linux definitely gets more users and is more “tested” so things like drivers and software will have better support.
Why not use ZFS on Linux?
It will never be in-tree, short of an act by Larry Ellison.

Depending on an out-of-tree module can get pretty crappy on Linux. The upstream kernel will break you a lot. If your distro is on top of keeping your out-of-tree feature working then great. It gets messy whenever that is not the case and/or suddenly you want a driver or feature only available in a newer kernel.

Filesystem and Distro/OS are part of one choice imo.
Larry Ellison can't relicense ZoL - they'd have to start from scratch porting the code from Solaris
I think people typically mean that Larry Ellison could come out and say there are no issues with incorporating CDDL code into a GPLv2 code base. Ideally in a document signed by him and lawyers at Oracle given to the Linux Foundation. If that happened I don't think people would have an issue adding ZoL to the kernel.
There are no issues on the side of CDDL anyway.

Larry can't change the fact that GPL is the source of incompatibility - the general interpretation is that GPLv2 forbids code that has stricter licensing terms than GPLv2, and CDDL happens to have that - in form of patent grant.

It's also why GPLv3 code has a license issue with Linux kernel, which is GPLv2.

They can relicense Oracle-owned code under GPL. As just one example.

There are probably other options, too.

Yes, and it would require starting the port from zero, as I have said.
Nah, it's the same code. There would be no need to create a new derivative work.

It's a pipe dream anyway, of course.

It's not the same code - OpenZFS has substantially diverged from ZFS v28 which was last common version between OpenZFS and Solaris, that's one thing.

Another is that all the porting work in ZoL that isn't already GPLv2 is CDDL - and Oracle is not the copyright owner. In fact, the copyright owners would not want to change the license to GPLv2, unless maybe it was done as dual license.

I was being a bit flippant rather than literal with the Ellison comment. I think I read a similar joke on an LKML thread or similar once.
Opinions and dogma about licensing is one reason as well as a sense that ZFS isn't actually better than ETX4. I'm not saying I agree with that, only that it's why ZFS and Linux don't mix from what I can tell.
It would appear canonical disagrees.
ZFS and EXT4 have different use cases.
ZFS and EXT4 on LVM have a lot of overlap. A decade or so ago I made a comparison between ZFS, BtrFS, and EXT4 on LVM [0] that may be of interest.

[0] https://rkeene.org/projects/info/wiki/BtrFS

The checksumming/error-correction of ZFS is big difference, and the main reason I moved some of my file systems to ZFS (even with versioned backups, on EXT4, I suffered data loss due to corruption/bitrot).

On the other hand, for a simple set-up on a lower resource system, EXT4 has lots of advantages.

A good, controversial read is available at [0]. I don't agree with his conclusions entirely, but the data he presents is interesting. And also paired with [1] to give space overhead and MTBF figures for various RAID configurations (where RAID-Z ~= RAID5, RAID-Z2 ~= RAID6)

[0] https://www.jodybruchon.com/2017/03/07/zfs-wont-save-you-fan...

[1] https://rkeene.org/projects/info/wiki/221

From experience, as I said, I had bitrot on EXT4, with carefully versioned backups, but it didn't help, since the bitrot was just faithfully propagated through all the backups (I didn't have 5+ year backups - who knows when the bitrot occurred).

And I haven't experienced file corruption in the 5+ years I've used ZFS. Of course, ZFS isn't a magic bullet that solves all ills, but I've been very happy with it. I still, of course, have multiple backups of important data, but now I'm more confident about the integrity of those backups.

I wasn't aware that anyone thinks ZFS is no better than ext4. Surely the interest in developing Btrfs and Bcachefs, with features obviously meant to compete with ZFS, would suggest that mostly people think it is? And you also have Canonical adopting ZFS outright, and Red Hat working on improving XFS as an alternative file system.
Been a FreeBSD user for very long time besides Linux Slackware distributions since kernel 1.0. Started with Walnut Creek CD distributions. In early years it was much better than Linux with its fast networking stack and filesystem. Also it gives a sense of conceptual integrity as distribution is not just kernel but all the utilities, they work together in unison. But than it also hamper innovation due to design by committee vs the best tool for the job.

Now with time and various flavours of Linux from NixOS, GuixSD, Gentoo, LFS, Slackware, arch, Debian, Ubuntu, CentOS, OracleOS etc., it is in many case a better choice than FreeBSD.

In most benchmarks and hardware support it has surpassed FreeBSD. I still hope FreeBSD continue to grow and innovate, but it’s not a viable choice in many areas of modern computing. Hopefully it can catch up. It still is a very viable choice in building network devices, but there also with modern hardware support Linux is giving it a tough competition.

I still use FreeBSD, but my majority production workload now is Linux with some firewall and networking related stuff in FreeBSD and openBSD.

I did try portable FreeBSD based version like dragonfly and pcbsd. But now I don’t use them that often.

As a counter to this, if you're working on network software that uses a lot of bandwidth and a lot of open connections, BSD is a bit more stable than Linux, so it's generally a better choice for that at least.
This hasn't been a significant issue for a while now, to the point that Linux is used on cache appliances from the FAANG (though Netflix is notable for using FreeBSD in that role).
This is dangerous advice. BSD networking stack may have been superior at some point but that hasn’t been the case for many years now.

You are very likely to run into issues trying to do filtering and/or routing at anything >1Gbps on commodity hardware (ie. e1000 nics).

Pf is pretty but it’s not worth trading stability for.

Did Juniper stop using FreeBSD recently? Also, Force10's FTOS has used NetBSD as a base certainly less than "many years now". Force10's L3 switches are at 1GB, 10GB and 40GB speeds IIRC.
The Force10 speeds are most likely not directly due to NetBSD, but from custom or specialized silicon.
Isn't 1Gbps the highest throughput of such hardware?
FreeBSD network stack has always remained rather stable. In terms of performance Linux caught up and even overtook at some point, but lately Netflix did a lot of performance improvements for FreeBSD. So, your information could be outdated.
No citation on such a big claim?
I would be totally on board with using FreeBSD as my primary server OS .. if it had a container engine. There was a port of Docker to FreeBSD, but it's ancient and unmaintained (before Docker did their big modular refactor).

In theory, you could implement a lot of the Docker Engine API for FreeBSD and on the backend use jails instead of namespaces/cgroups and ZFS if you wanted implement the same layering. Some of the security and cgroup limits may not be implementable on FreeBSD, but with the Linux binary emulation layer, you should still be able to run a lot of containers without modification (unless they need certain filecaps or privileges).

I had OpenBSD on a laptop once, but without 802.11ac support, you'll either have to downgrade the Wi-Fi card in newer laptops or use a USB adapter.

The fact that the Docker port was allowed to die is a pretty good indicator of how little interest there is in using FreeBSD in the industry, particularly in cloud environments.
"jails" are the native solution for FreeBSD. You wouldn't set up a FreeBSD server to run docker.
Most companies would rather use Docker than jails and that alone is a reason to avoid BSD.
That's fine. I've been running various Linux distros since 1993 and FreeBSD since 1996. Each has its strengths and weaknesses.
And how can I run OCI-compliant image straight to the jail? Docker isn't just a namespaced runtime.
It's simple: You shouldn't run Docker images on FreeBSD.
Wrong answer. This means you can't run modern cloud-native apps on BSD.
Tell that to companies like Netflix: https://www.phoronix.com/scan.php?page=news_item&px=Netflix-...

Not all applications require Docker.

Last I checked, FreeBSD is only used in their CDN appliances and Netflix is deploying their services in containers on Linux.
As I said, Linux and FreeBSD have their strengths and weaknesses. I use both, and definitely wouldn't deploy certain types of applications under FreeBSD.
You are probably still right but MIT license means that you just won't see what industry uses. Industry has used free BSDs and not contributed in code or money nearly to the extent that it has for some Linux flavors.
The license has nothing to do with that. GPL2 doesn't require industry to contribute back either. I decline to speculate on why industry adopts Linux.
A bit of it had to do with the SCO lawsuits, it had a freezing effect on BSD and it's adoption numbers dropped way behind Linux after that and never really recovered.
That wasn't SCO. It was an earlier set of lawsuits by AT&T because 4.4 BSD had evolved from AT&T unix.
Your right, It's been a while and I got the two mixed up. Funny how time can play tricks on memory. It was the AT&T lawsuit. SCO was Linux. Sorry for the misinformation. IIRC Linux had come out a few years before the BSD lawsuit, but I do recall a time when they where neck and neck and BSD started dealing with the legal issues and it just seemed like Linux keep on gaining traction after that.
From what I understand, this was a question and was answered on other forums.

FreeBSD jails are supported by HashiCorp's Nomad [1], and their Consul + Vault are in FreeBSD ports

The tool mentioned in the article, Bastille, would be used to install popular software into jails.

Also ansible works well for pushing software into FreeBSD. There is even an ansible role for nginx that works with FreeBSD that's maintained by nginx team.

FreeBSD jails do not insist to have 'one service per jail', shell access is fine too. A jail is pretty well functioning 'compartment' within an OS.

Jails work very well with ZFS, one elegant feature, is that one can create a 'base jail', create a zfs snapshot with it, and then copy it over (using zfs) to any new jail created. Sort of like a base template. [2] This makes spawning new jails with base software install rather quick.

I do not know if FreeBSD has some interesting Linux features like io uring.

On the other hand, Netflix is achieving an impressive 200 Gb/s serving video traffic [3] using FreeBSD

- - - Just wanted to mention, that OpenBSD is different than FreeBSD, different kernel, many different user space programs, different team, different primary goals, different hypervisors that they have built-in, etc.

Same statement for Dragonfly BSD, same for NetBSD.

With regards to issue with OpenBSD 'on a laptop once'

I am using 5G wifi running OpenBSD 6.6 on a laptop, not using USB adapter. The driver I am using is iwn0. Works great. Perhaps we have different laptops...

[1] https://nomadproject.io/docs/drivers/external/jail-task-driv...

[2] https://www.cyberciti.biz/faq/how-to-configure-a-freebsd-jai...

[3] https://people.freebsd.org/~gallatin/talks/euro2019.pdf

I wonder if FreeBSD Jails never really caught on that much because of the name.
I'm not sure about that. Outside of tech, nobody cares what the technology is called. Inside... we've been happily freezing process groups, killing frozen children, and reaping the dead ones. Objecting to jails would be silly at this point.
What’s your laptop?
hp EliteBook 2560p. 12gb ram, 120 gb ssd.
The FreeBSD answer to Docker is to not use it. Just go back to the old way of running services on bare metal or VMs. It's not a bad idea, but other people who worship refinement and perfection will get mad at you for it.
Given as jails started with FreeBSD, no, I would say that running services on the bare metal is not the FreeBSD answer.
If you use the built in tools, FreeBSD leans towards telling you to just install ports/pkgs and then start them jailless. Your machine becomes a pet experiment in short order.

I also find it weird that each jail has its own copy of all of userland. It's a lot easier to use macOS sandboxing, certainly it saves more disk space. And I feel like I'll forget to update them.

Yep. I’m not sure I agree about user land but jailing everything is definitely not the default. It’s there if you want it and know what you’re doing, though.
"I would be totally on board with using FreeBSD as my primary server OS .. if it had a container engine."

FreeBSD is the original container engine - although, for the sake of completeness, we should count Solaris "zones" and call it a tie ...

I think there is a cultural and temporal connotation of Docker - and the use-cases that it typically enabled - that makes it seem very different than 'jail' but make no mistake: jail did all of those things and more, many years earlier.

The VPS, as we currently think of it, was built first[1] on top of FreeBSD and jail.

[1] JohnCompanies announced beta availability of "server instances" on 303 and cDc mailing lists in mid 2001. Someone else coined the term "VPS" a bit later ...

Being first doesn’t matter. Docker became the industry standard and jails did not.
According to Wikipedia, FreeBSD jails were available in a release version in 2000. Linux had Virtuozzo in 2001, explicitly for the use case we would call VPS today. Solaris didn't have zones until 2005.
It's mainly that the Docker interface is easier to use, in fact I don't know of any Jails frontend that comes close.

Also, at this point it would need to support the same image format and be very obviously better to see wide adoption.

Docker in 2020 is, for 95% of the use cases [1], nothing but a better packaging format than .tar.gz.

I'm sure you could throw away the isolation bits and it would still be as popular.

[1] Totally scientific figure, of course.

You might want to have a look at Bastille, a “system for automating deployment and management of containerized applications on FreeBSD”.

https://bastillebsd.org/

Slackware lines up more with the original philosophy this article talks about. I like taking Slackware in either stable or -current and running it in this article instead of Debian ... If you wanna run Linux with a distro that hasnt changed its overall philosophy, structure, or hell even core software you should be looking at Slackware again. 15 will likely be out this year.
(comment deleted)
I've been a long time OpenBSD, FreeBSD and Linux user (I use Windows for mainly for work though). I've given up with the BSDs tbh. While I like that I can understand these systems. Linux is just easier to get things working on. There is just more information now out there for Linux as it is the defacto *nix out there. So it may well be that BSD has many technical merits, they tend not to be all that useful when you are trying to get stuff done.
It's been said a million times- FreeBSD is an operating system, Linux is a kernel. ("GNU/Linux" is a strange beast- kernel+libc or what?). It's a tough direct comparison to make especially when you consider things like documentation (which FreeBSD is often cited as being exceptional)
> ("GNU/Linux" is a strange beast- kernel+libc or what?)

Kernel plus userspace: the standard command-line tools, compilers and some additional stuff. Such as GNOME, for instance (it's considered a part of the GNU project).

It's a little weird to not put some of these headliner features at the top of the list. Like ZFS (in-tree, on the default installer, without licensing issues or hostile team dynamics).

On the other hand there are things in this list which are just the FreeBSD equivalent of something Linux has in some form. The ports tree is good, and I like that you can customize options etc., and now pkg is pretty good, but are you really going to miss that if you have apt or similar?

Aren’t Linux and freebsd using the same zfs fork at this point though?
I am not following closely enough to know when that makes it into a stable FreeBSD release, but yes it will be.

Except for one point: the zfs on Linux work is not in the mainline kernel, and won't be. A strict interpretation of your comment would actually be "no" - can it really be said that "Linux is using it" in that case? You need out-of-tree kernel support which is very difficult to maintain.

I find some points... debatable. For example software:

> Normally when you install software on a Unix operating system you find and download the software.

According to the title, this is about choosing FreeBSD over Linux, not over Unix. That means the normal way to install is "<package_manager> install <package_name>". For the end user this is just like ports without having to compile anything. Once you don't have the package or the port available, you end up in the same situation: download the source and compile (which is more likely to work on Linux OOTB simply due to popularity).

> Poudriere is a utility for creating and testing FreeBSD packages. It utilize the FreeBSD jail system to set up isolated compilation environments.

In Debian-like systems you can use pbuilder in the same way. Each package is built in the same clean choot. I suspect RH has something similar?

> bhyve

They don't list anything technically better about bhyve. Why would I use this over xen or kvm?

> FreeBSD has three different firewalls built into the base system: PF, IPFW, and IPFILTER, also known as IPF.

Ok. Linux has iptables and now nftables/bpfilter. It's also got queuing, stateless and stateful features, protocol tracking, etc. CARP can be done in userland, or via other solutions like linux-ha/stonith. What are the reasons to choose FreeBSD here? (also, mentioning 3 firewalls after being excited about well-designed system that just comes together rather than being glued together from third-party software just sounds weird)

> BSD init

They haven't listed what's actually technically better about it than in systemd. (not subjectively)

> Jails

Jails as described in the article, have the same features as lxc. Lxd provides zfs-as-a-backing-store option as well.

> FreeBSD has over five hundred system variables that can be read and set using the sysctl utility.

Ok. On my Fedora system "sysctl -a | wc -l" gives 2093.

> For example, on top of the geom_mirror module an encryption module can be added, such as geom_eli to provide a mirrored and encrypted volume.

So how is it better than lvm with its layers dm-raid and dm-crypt?

I get that people are excited about FreeBSD and I'd love to learn what are the technically better sides of it. But this article really doesn't list that many.

> They haven't listed what's actually technically better about it than in systemd. (not subjectively)

And even to avoid running headlong into this troll pit: Linux distros without systemd exist. It's dishonest to compare FreeBSD to Linux and then only focus on a subset of Linux distros.

I chose systemd since BSD's init has some points over the classic init used in linux previously. But any of the modern choices which actually manage processes (upstart, systemd, whatever you prefer) score a few technical points. Definitely didn't intend to single out systemd as better in this case.
> And even to avoid running headlong into this troll pit: Linux distros without systemd exist. It's dishonest to compare FreeBSD to Linux and then only focus on a subset of Linux distros.

Are any of them mature enough to depend on for serious work? I looked at moving back to Linux a few months ago, but all the non-systemd distributions seemed to be small projects and many of them looked to have schismed, collapsed, or both.

Void is a small project, but absolutely rock solid. It's BSD inspired and systemd free. I like it a lot.
I can also vouch for Void - I've used it on my primary laptop for about 4 years. It's a solid distro, with a great community. Void is very configurable, and it's easy to contribute to the project.

It's also unlike a number of 'systemd free' distros (many of which are essentially "Distro X, except without systemd") because its non-use of systemd is not out of some hate for systemd, but because the developers decided after using systemd (Void adopted systemd for a time really early on) that it conflicted with other goals of the project (like offering a musl libc flavour alongside a glibc flavour).

Here's the thing. I would run: Ubuntu, Debian, Centos or Red Hat on my laptop, desktop and also on any server that I run.

I wouldn't run any small distro in AWS.

Have to agree here. I get a lot of benefits professionally from running Linux personally. I've been a full time desktop linux user over 10 years, and I'm often considered the "linux wizard" because I'm known for SSHing into servers and fixing obscure errors that others have looked at for hours with no success. My secret, is that I've run into most of the common problems (filesystem permission issues, including ACLs, selinux, low memory/OOM, conflicting dependency versions, etc) before and already mostly know how to fix them. The time I did on Arch was particularly helpful here. It was a hard first week or so but I learned more about linux during that two weeks than any other time.

Anyway the reasons I say that: These days I only run distros locally that I would run in production. I get intimately familiar with the intricacies of the distro that way and managing the servers comes natural to me because of that.

Alpine and AntiX, I use them in VMs because they are lightweight, so far no issues.
What kind of apps do you run? A variety of languages/platforms?

Some stuff runs great in Alpine. Other stuff is subtlety but catastrophically broken in Alpine (such as database servers/clients and others that depend highly on locale for determining encoding).

Agreed. I thought Alpine was superb until I ran into an issue mounting NFS shares that seem to be mountable just fine on every other Linux machine I've got lying around. No idea what's wrong with Alpine's NFS.
> What kind of apps do you run? A variety of languages/platforms?

Mainly Python, some issues with Musl, but nothing too bad. Anyway, these days I tend to mainly use AntiX.

I know some developers use Gentoo with OpenRC for their daily work machines.
Slackware is still here looking at you guys
Slackware is the one Linux I keep coming back to after trying out something else. As always, it just works (and no dependency checking in its package managers is a blessing compared to dealing with some issues in Ubuntu or Debian; for others there's always pkgsrc, by the grace of NetBSD).
Ah, "serious work". That well-known absolute thing that is the same for everyone.
None of those are beyond hobby level, though.
> According to the title, this is about choosing FreeBSD over Linux, not over Unix.

"Unix" hasn't referred to a specific operating system since the 1980s. FreeBSD and Linux are both Unix OSes: different flavors of it.

That is my point though. Not every Unix system has a package manager. But almost every Linux distribution does.
That's even one of they key attributes that makes differences between distributions: the package manager. Other major ones include the init system, and whether the distro is rolling release or not.
Linux isn’t really a UNIX, though.
Isn't Linux an anagram (no not an anagram but I can't come up with the right word) for Linux Is Not UNix?

https://acronyms.thefreedictionary.com/LINUX

Backronym?
No, that’s GNU with the “not UNIX” recursive acronym. Linux was named by the guy who first hosted Linus’ OS and was an amalgamation of “Linus” and “UNIX” (“nix” was a popular suffix for platforms of that type back in those days so “Linux” was a play on the authors name in the style of that naming convention).

If I recall correctly, Linus originally wanted his OS to be called “Freenix” because he thought naming it after himself was a little vein.

For all intensive purposes, it is.
I think the expression might be "for all intents and purposes".
I hole-hardedly agree, but allow me to play doubles advocate here for a moment...
Well, if you want to be precise … Linux is a kernel.
And the OS is called GNU/Linux.

/me ducks

> That means the normal way to install is "<package_manager> install <package_name>".

This excludes a lot of non-free software, which is included in ports.

> Ok. Linux has iptables and now nftables/bpfilter. It's also got queuing, stateless and stateful features, protocol tracking, etc. CARP can be done in userland, or via other solutions like linux-ha/stonith. What are the reasons to choose FreeBSD here?

Documentation, in my experience.

> They haven't listed what's actually technically better about it than in systemd. (not subjectively)

From experience, documentation and bugs (mostly due to age, but still absolutely relevant). I have about a billion questions about why systemd is built the way it is but I suppose that's a matter of taste and who paid to build it.

> Jails as described in the article, have the same features as lxc.

Documentation as always with the linux/bsd comparison. In addition, LXC—in particular cgroups—is still quite young and unproven. The major benefit here is the docker interface enabling easier deploys and an image specification format, which freebsd still lacks.

Honestly, the documentation alone is enough to move me over to BSD—linux is basically the wild west, and you often discover undocumented behavior (is it a bug? is it intended? ask developer xxx on the mailing list.)

> This excludes a lot of non-free software, which is included in ports.

This depends completely on your choice of a distribution. There's nothing about package managers that makes a difference in this case.

For example Arch, especially with AUR will include all non-free software you want. Ubuntu offers them. Debian/RH have them one config edit away. Recently snap/flatpak also provide those to all distros - whether they're using free-only repos by default or not.

> In addition, LXC—in particular cgroups—is still quite young and unproven.

LXC was introduced over a decade ago. Cgroups even before that. Cgroups themselves are enabled per-session and per-service in pretty much every modern linux deployment for years now. There's less time between jails and cgroups than between cgroups and current day. Cgroups were added to linux in the same year as zfs was added to freebsd. (2007)

I'm not sure what you see as young or unproven about them.

LXC was only introduced in 2007. I wouldn't call ZFS a good comparison as I would also still consider it rather young in terms of basic unix technologies—depending on your needs, you could easily use some other simpler but older (and likely less buggier) software.

With respect to cgroups versus lxc specifically, I don't think the approach is well suited for security reasons—the surface area for bugs and vulnerabilities is much larger than with the jails approach.

> the surface area for bugs and vulnerabilities is much larger than with the jails approach

It would be great if someone wrote an article with technical details about it. I see this repeated very often, but if you actually go looking for details... it's hard to find any justification.

I can’t say I have a decent justification off hand myself—this was the impression I had when digging through a comparison of the two softwares about a year back. Cgroups appears to have a much more detail laden implementation, but this could be explained away with my lack of Linux internals.
The thing that always impressed me with FreeBSD was how it was a complete integrated system, and not a group of miscellaneous parts in loose formation.
>> bhyve

>They don't list anything technically better about bhyve. Why would I use this over xen or kvm?

Isn't it actually worse? There doesn't seem to be AMD support:

>Hosting Linux guests or FreeBSD guests with more than one vCPU requires VMX unrestricted mode support (UG)

Does AMD have UG or does this mean no AMD support for multi-processor guests?

With Meltdown and Spectre, doesn't this mean that VMs on linux AMD systems provide much better performance, and even better performance per dollar?

> does this mean no AMD support for multi-processor guests

I dunno about the details but I've got a Debian VM with 6 vCPUs running on Bhyve, FreeBSD 11.3, on my Threadripper 1920X.

AMD doesn't need unrestricted guest support, it has always had it.
> also, mentioning 3 firewalls after being excited about well-designed system that just comes together rather than being glued together from third-party software just sounds weird

I noticed this too. One of the main *BSD benefits people always tout is that it has "one true way" of doing things - yet having 3 different ways to do firewalling is somehow a great thing when we're talking about FreeBSD?

ipfw would be that one true way on FreeBSD, pf on OpenBSD
Last time I checked, FreeBSD'd pf is quite older than OpenBSD's and also FreeBSD doc says it has diverged from the OpenBSD's pf.
technical merits, can be plenty. however how does freeBSD integrate with the rest of the I wanna get things done ecosystem of people n software. Can I easily install it on a laptop such as a thinkpad and have a plug and play experience. Can I easily update and find packages | software I need. if Open source software approached things from the usability view, we wouldn't be beholden to corporate interests. I personally, use KDE on a thinkpad
FreeBSD marketing:

  - FreeBSD is like Linux but <>
  - It is more <> than Linux
  - It is less <> than Linux
  - It has more <> than Linux
  - It has less <> than Linux
  - Linux LINUX Linux LINUX LINUX
OpenBSD marketing:

Look, cute fish.

> OpenBSD marketing:

OPENBSD IS SECURE. ARE YOU SECURE? ARE YOU WORTHY? OPENBSD: SO SECURE WE DO NOT NEED YOU!

(comment deleted)
FreeBSD and Linux have pretty cute mascots too.

Puffy is exclusive to OpenBSD. Tux and Beastie are more generic.

Also:

"Early versions of OpenBSD (2.3 and 2.4) used a BSD Daemon with a halo, and briefly used a daemon police officer for version 2.5. Then, however, OpenBSD switched to Puffy, a blowfish, as a mascot." [0]

[0] https://en.wikipedia.org/wiki/BSD_Daemon

Edit:

You're absolutely right that there seems to be some kind of power fantasy among the FreeBSD crowd about crushing Linux.

Just do an image search for "BSD Daemon" and see the g(l)orious fan-art.

I run Ubuntu as my daily driver primarily for its broad 3rd party application support. As such I tend to run Ubuntu on servers as well, so I'm usually in familiar territory. I have a smart friend who extols the benefits of FreeBSD, but I haven't ever given it a serious go. Is there a canonical/non-hacky way to run debian packages on FreeBSD? Like can I run Slack and Zoom? Steam?
id rather see this on openbsd
Linux is a Boeing (a 737 Max, if you ask certain people), OpenBSD is a Cessna.

If you do a direct comparison, OpenBSD would lose in terms of benchmarks, hardware support, software support, features, etc.

But OpenBSD is more a philosophy, a particular way to approach computing that works for some people. OpenBSD is basically a continuation of Unix from the 80s, where the admin/user (no distinction in reality) is expected to do some work from the command line. Thus it's a great user experience. The config files tend to have the same syntax, and it's human-friendly syntax, not some XML or YAML crap designed for a tool. Things are kept simple so human brains can understand them.

But the trade-off is basically features, performance, hardware and software support. It's a bitter pill but, again, it works for some people.

I use OpenBSD full time on my router, but I tend to switch back and forth on laptop and servers (with Debian).

When Linux is a Boeing 737Max, then FreeBSD is a Airbus A380 and OpenBSD is a A-10 Warthog. xD
Practical reasons to choose linux over freebsd

1. People use linux more than freebsd

Nothing's more important than this.

Sounds straight from a 90's Microsoft marketing department, I'm glad not everyone listened back then.

More users is definitely a feature though.

tbh microsoft was the best choice in the 90s.

the only reason linux became useful is because it became popular.

only for very particular, and peculiar, senses of 'best'
For some domains, But certainly not all, Windows was arguably a better choice, but even then it was more Windows NT vs Novell than Unix.

I don’t recall Microsoft ever getting much traction in the racks of ISPs or in “big iron” installations, not until the late 90s... and even then it was tenuous.

Microsoft was a terrible choice for IP networking, for example. Changing the IP address of a server would require a reboot.

The majority of the work I did in the 90s was on HPUX and DGUX, but also a bunch of others. There was a lot more OS diversity back in the day.

Might as well at not choose to do anything then. Just use whatever everyone else does.

We need people trying out better ideas even if they aren't as popular or easy.

I have dabbled in BSDs (mainly OpenBSD) but can't stick with any of them because Docker doesn't work without virtualizing Linux. If I have to have a separate setup for "real work" versus hobby work, then it's a non-starter for me; I don't have enough time or brain capacity.

I like the idea of BSDs as this self-contained system (like the polish of Apple's walled garden but without the tyranny), but it is too far behind the cutting edge.

I don't think the parts about ZFS being "better integrated" on FreeBSD are really true any more.

For one thing, ZFS-on-Linux is now the canonical, upstream OpenZFS implementation - https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSOnLinuxN...

For another, Ars Technica recently reviewed FreeBSD 12.1-RELEASE and found poor ZFS integration in the installer and in mount(8):

> Eventually, I realized that FreeBSD's mount command didn't really understand my ZFS root filesystem. -- https://arstechnica.com/gadgets/2020/04/not-actually-linux-d...

Compare that with the relatively streamlined ZFS-on-root install support built into Ubuntu: https://arstechnica.com/information-technology/2019/10/a-det...

(I'm sticking with ext4 until kvdo, bcachefs, or maybe tux3 is mainlined.)

> (I'm sticking with ext4 until kvdo, bcachefs, or maybe tux3 is mainlined.)

I haven't heard of kvdo before - thanks for bringing it up!

I'll add stratis-storage to this mix too. (Although I'm contributing to the bcachefs development and really want it to succeed)

why not use zfs, since it does the things that kvdo, bcachefs, and tux3?
Only one reason of Oracle law suite sword hanging over neck. Even if canonical gives guarantee they will fight with Oracle, if a small company is dragged by oracle in a court room it's hard to fathom the expenses.

So yes there is a need for zfs alternative. So far stayed away from it and using btrfs which works well, except for some RAID limitations.

Fortunately the license Sun applied means any lawsuit would need to attack something much more fundamental, like the very idea of interoperability, as they don't have option to "claw back" any portion of the OpenZFS codebase.
Given that they are currently fighting that an API should be copyrightable, doesn't make me confident that they won't do this.
If they do, GPL is fucked anyway. And not just GPL.
ZFS is not released under GPL, if it is then it won't be a problem. So there is a need for an alternative and BTRFS works well for me, no need of license encumbered ZFS, not sure of others.
My personal reason is that zfs.ko entirely bypasses the kernel page cache, preferring its own internal cache (the ARC). Reimplementing kernel subsystems is a recipe for interoperability nightmare (c.f. the NVIDIA binary driver). For instance, this causes unnecessary memory pressure when you have a mix of ZFS and non-ZFS filesystems (e.g. SMB mounts).

Additionally I curmudgeonly have yet to come to terms with the blatant layering violation of ZFS being a combined block-device-manager and filesystem. But the benefits (file-level RAID behaviour) of that are indeed clear.

> I curmudgeonly have yet to come to terms with the blatant layering violation of ZFS being a combined block-device-manager and filesystem

It has both advantages and disadvantages. It’s a trade-off. ZFS decided it was worth it, and so did btrfs.

Sure they committed these crimes, but those are the reasons why ZFS is better than the alternatives.
> Eventually, I realized that FreeBSD's mount command didn't really understand my ZFS root filesystem.

I couldn't find this direct quote in your link. But I'm using FreeBSD with ZFS on a few machines and I don't understand this comment.

You can absolutely boot with ZFS on root. The installer lets you do it out of the box. Including with geli for encryption.

And you can use mount(8) to mount a ZFS partition, I have done that all the time when booting from recovery media and mounting my root pool. This gets a little confusing at first because most uses of mount(8) require a device name, and mounting ZFS takes a pool name, so you may have to zpool import first, but once you get over that it's fine.

The quote appears in the article, on page 3 after some lazy-loading (do you have Javascript enabled?)
Ah ok, this is about trying to remount read-write. Yes, that requires zfs set readonly=off on FreeBSD. [vs. -o rw with mount(8)].
Actually zfs mount -u / works just fine.
But I think the complaint is that it doesn't happen through mount(8).
The Linux kernel devs hate ZFS and actively sabotage it even backporting changes specifically to break ZFS to LTS branches.

It's fine if you use something like Ubuntu where it's officially shipped with the distro. If not, be prepared to apply kernel patches whenever the kernel team takes a snipe at ZFS.

I'm not trying to be political here, just my day-to-day experience working with several Linux machines running large zpools. It's still worth it compared to the alternatives but if I could use FreeBSD I would.

The Void team does a good job with managing ZFS as well (it's in the official repos). The unofficial ZFS on Arch is generally ok as well (though Arch's management of kernels is weird and a bit tricky).
Do you have an example of a commit where they did this? Just curious as to how (no need to explain why, I get it)
See https://github.com/openzfs/zfs/issues/8259 for details.

While they didn't remove it to sabotage ZFS, they don't seem to care about it ruining ZFS performance on Linux.

Of course they have no obligation to make ZFS on Linux users happy, still a bit of a bummer tho.

It looks like Linux kernel devs just don't care about ZFS, like they don't care about many out-of-tree kernel patches/modules.
There's absolutely no evidence for this. On the other hand, there's some evidence that Sun made ZFS licensing explicitly non-compatible with the GPL, so as to not "sabotage" Solaris.

Linux has other efforts going on in this area, including Btrfs and Bcachefs - there's also ZoL, of course.

Take a look at any of Greg Kroah-Hartman's comments on patches that screw up ZFS.

One of the biggest ones was removing exports of kernel functions that ZFS used for encrypted datasets: https://marc.info/?l=linux-kernel&m=154689892914091

I'm fine with this sort of stuff in general assuming it doesn't affect LTS, but this specific patch was backported to LTS kernels even though it has nothing to do with security or bugfixes. I had several zpools I had to apply custom kernel patches on the machines for no other reason than because somebody is mad at licensing.

Bcachefs isn't even stable enough to use in production. Btrfs is okay, and I've used it before, but it's also a minefield of killing your data depending on what features you use. If you use all the features you need to make it on par with ZFS you may as well dump your data in the trash.

ZFS 'just works' and none of the features are incomplete or have asterisks attached that compromise data integrity. Yeah it has architectural problems, doesn't do stuff the Linux way, and has a license that makes devs angry. It's also never lost a byte for me in production for almost a decade of use.

I know the Linux devs don't owe me anything and I'm not really angry or anything about this. It's just a fact that it's frustrating to have to deal with political problems just to run a server.

You're just ignoring what I said above. Nobody is out to screw ZoL, but the kernel community doesn't care enough to maintain compatibility with it.

Why, as a kernel dev, would you care if there's no possibility of it ever being mainlined because of decisions Sun explicitly made to be hostile towards your project?

It's like trying to cozy up to someone who just punched me, not happening.

Bcachefs isn't yet up to parity for sure, but it is stable enough for everyday use and doesn't corrupt you data like Btrfs sometimes does. A stable foundation that gets expanded upon daily.

> I don't think the parts about ZFS being "better integrated" on FreeBSD are really true any more.

ZoL is not in the mainline Linux kernel, therefore it isn't even "integrated". Linux 5.x introduced some backwards incompatible changes which broke ZoL also triggering a response from Linus.

https://www.phoronix.com/scan.php?page=news_item&px=Linus-Sa...

For these reasons I'm also going to stick with ext4 on Linux. In the meantime we run appliances and servers which run ZFS on FreeBSD without any issues. SmartOS is another choice to run ZFS but I don't have any experience using it. Maybe I'll try it as an alternative to Proxmox.

How do e.g. the sections about bhyve and Bastille describe Technical reasons to choose FreeBSD over GNU/Linux? The describe features, but they tell me nothing about why those might be better than counterparts in the Linux world.
Wow I see no mention of no systemd in FreeBSD here.

I love how unified the freebsd maintenance and just general system utilities are. The documentation doesn’t lie. I’ve never had a bsd box stop booting because of a wonky system change either.

> Wow I see no mention of no systemd in FreeBSD here.

The author wrote about it in the "part 1" article linked in the introduction.

I started using FreeBSD a few months ago for a Tor relay, trying to promote OS diversity. I had read these types of articles about how FreeBSD is so much cleaner and well documented than Linux, and I wanted to try it out.

Honestly, I found it seriously lacking. It always seemed to me like most of those article writers had poor understanding of Linux, and now it seems to me like they have poor understanding of FreeBSD too.

For example, this article, like many, touts the high quality of FreeBSD documentation. I found it to be similar to Linux documentation: a few excellent pages, mostly good pages, some mediocre pages, and lots of undocumented stuff. People (including me) often complain about the quality of Linux networking documentation, which is unquestionably terrible, but I found that FreeBSD isn't really that much better overall. Better in some places, worse in others, often you have to give up and go find the source code.

Another thing often touted is the centralized development of FreeBSD making it more organized. This article betrays itself here by explaining that FreeBSD has three different firewalls, one originating locally, one forked from OpenBSD, and one forked from some dude. Now, to be fair, Linux is far from clean here: between ipchains, iptables, nftables, and bpfilter, Linux has gone through a lot of change. At least with Linux though there are at most two practical options at any given time, and there is a clear forward progression. With FreeBSD, I spent several hours trying to figure out the differences between each, which wasn't helped by the documentation problems: manual pages being fragmented and incomplete and the handbook being woefully out of date on this subject.

It also seems like a lot of the hyped FreeBSD features are actually far behind Linux features. As other commenters have mentioned, Linux has device-mapper, which is similar to GEOM, and is used for LVM and modern RAID setups. Jails came before containers, but Linux namespaces are now far more powerful than jails, which aren't that much more than chroots. bhyve is nice, but lags behind QEMU/KVM. Poudriere is fine but most Linux distros have chroot builders too now.

In conclusion, I don't find FreeBSD particularly attractive for any of the reasons that I hear often touted.

Any Linux or BSD distro could achieve best-in-class documentation for new users by simply adding example usages for most commands to the top of their man pages. Man pages tend to read more encyclopedias than practical guidance, and FreeBSD was no exception last I looked.

Command xyz

Common usage: xyz -s source_ip -d dest_ip:port

The Arch Wiki is pretty damn good in my opinion. I don't see how FreeBSD's documentation is better, if "better" = more useful and up to date.
Slightly off-topic here, but you might like the tdlr project[1]! It's community-maintained, example-focused documentation for common command-line tools. Currently, there are over 1700 pages, and it's very easy to contribute new pages or improve existing ones, because it's all in simple markdown on github. (For example, [2] is the source for the tar page.)

[1] https://tldr.sh/

[2] https://raw.githubusercontent.com/tldr-pages/tldr/master/pag...

It's sad to see that FreeBSD rehashes the same arguments they used at least since 2005 (I actually couldn't believe that the article is written in 2020) and completely ignore all the innovation that Linux has undergone since then. My favourite:

KMS (Kernel Mode Setting) makes graphical UI a first-class citizen and Linux a solid desktop foundation. Did FreeBSD finally catch up here?

Cgroups and namespaces together with a COW filesystem makes Linux a solid foundation for containers. I used jails back in 2009 and, honestly, its vision was incomplete. For n jails you needed n copies of the userspace. You couldn't use lo0 property, etc etc. Jails felt both too restricted and insufficiently isolated from one another. Has anything changed since then?

I really wished FreeBSD stopped glorifying its past and was realistically looking at its current position in the OS landscape. Until that changes, FreeBSD will be both a lacking desktop OS and a lacking server OS for my daily work.

>The kernel and base system is completely separated from the third party applications and base system configuration goes into /etc while all third party configuration goes into /usr/local/etc. Everything you can configure and everything you can tune or setup is very well documented in the man pages.

One thing I find ironic about BSDs is that their top billing feature is being a "complete integrated (base) operating system" rather than a collection of software maintained by different projects, but at the same time, they're almost always discussed as server/poweruser operating systems rather than desktop operating systems. This seems completely backwards to me.

If I'm a sysadmin running a server, I probably know what I'm doing and feel confident replacing parts of it. I likely studied this and relative to a typical person I'm much more likely to find the way it works to be interesting. By contrast if I'm a desktop user there's a very good chance I don't know what I'm doing and don't really want to. It seems like the second situation is the one where you benefit more from having a wall up that separates the system from the userlibs.

There have been fleeting attempts to make desktop-forward BSD distributions over the years, including PC-BSD and a few short-lived initiatives by the NetBSD project, but it never gets off the ground. Is this a missed opportunity?

What usage this article is talking about? For a server? For a desktop environment?

For servers, me and my colleagues use Linux, and not being able to use Docker is a tough call, our workflow is very Docker-centered. I know that Jails exist, but it's not just about containment, it's about packaging, building and deploying too, and having ready-made tools (and a lot of existing Dockerfiles) for that.

For desktop, I use macOS, and I'm not willing to change if it worsens the experience, which I suspect it will do.

I wonder what would be the killer use for FreeBSD today? I'd be eager to try, since I'm eager to try all kinds of stuff, but I fail to find any place I could easily stray from my current OS combination.

This seems more so like a list of differences, without any details as to what makes them different, let alone which is better...
If Linux were a railroad, FreeBSD would be a model railroad club. Nice stuff, in a way, and very similar to the real thing, but lacking a certain industrial usefulness.
Not at all. FreeBSD is without doubt on equal footing with Linux in terms of industrial usefulness, assuming the features you seek are available. Large companies use FreeBSD in production for mission critical apps but are generally less vocal about it.
Whatevery floats your boat. I use linux, docker and kubernetes all in the cloud. I'm getting more distant from underlying OS everyday.

BSD is a fine operating system. There is nothing wrong with it. Linux is a good operating system and gets the job done. Whatever works.

I think what is more concerning is that we really haven't had a big leap in hardware, OS, and system design for sometime. When are we going to have the guts to make the next step?

I’d argue NixOS is a huge step that a lot of people still have to catch on to (to be fair: that includes more contributors to improve documentation and ergonomics, but it is already valuable and workable as it is)
Working on this!

Check out the NixOS for existing sysadmins workshop over at https://github.com/ghuntley/workshops/tree/master/nixos-work...

If you want a TL;DR overview of NixOS then start here https://github.com/ghuntley/workshops/tree/master/nixos-work...

Something I wonder about after reading your overview - software from outwith the package manager.

It's quite often that the package manager in the Linux distro I'm using has an old version of Nginx, PHP, or whatever, and so I have to build my own, or at least use an alternative package source.

How is this kind of thing handled with NixOS config? (if it is)