98 comments

[ 0.24 ms ] story [ 153 ms ] thread
Neither. FreeBSD
For infra stuff (storage, firewalls, maybe loadbalancers) maybe -- but unless it can run k8s it's not going to displace linux anytime soon..
I'd love to use FreeBSD, but Docker still seems to not work and I depend on Docker a lot.
I actually like Windows as my dev machine and writing server software i would use less *nix-y dependencies to make most of my local development/debugging time and usually having mainly BSD as target but Linux working (with few dependencies coming from Windows both usually works once one is done since the diff to Windows is always bigger leading to fairly stable code).

Now however with WSL2/Docker enabling easier development I'm tempted to take taco-bell style shortcuts, it'll go quicker in many ways but also reduce portability to other platforms like FreeBSD (that while a favourite overall ends up behind in total if i start taking these shortcuts).

CentOS Stream defeats the entire purpose of CentOS, and is a completely different thing. So I think the title is the wrong question.
I think the poster is trying to make a point that it might be more pain for now to move over to a real community project like Debian, but in the long run you'd be less at the whims of IBM/RH overlords using you as beta tester (CentOS Stream) or at the mercy of them trying to handicap new "freeloading" projects (RockyLinux,etc).
I agree, CentOS was mostly used as a stable distro. Now it's the dev branch of RHEL. And it won't even benefit from being the dev branch, there is no way RedHat is releasing security updates in it before RHEL gets them.

So all updates and bugfixes will be pushed to CentOS Stream first, but they'll go out of their way to circumvent this for security patches. Just because of their paying customers.

Makes sense but it's kinda funny.

The OP is mostly about being a Debian fanboy.

It doesn't even consider that Fedora uses pretty much the same level of packages that Debian has and offers stable release upgrades for 6 years now.

Because Fedora has no such thing as LTS. But is that really necessary if you treat it like a rolling release?

I'm of course a RHEL fanboy and I love stuff like SElinux and yum/dnf too much to go back to Debian.

The OP is a Debian developer / Project Leader [1], maintaining quite a few packages, not some rando "Debian fanboy".

[1] https://wiki.debian.org/highvoltage

I didn't say random, you added that. Clearly you'd have to enjoy what you do to be project leader.
I think Debian is the sweet spot for release cadence--stable is supported for ~3 years, and there is a new stable released ~2 years. This cycle avoids the RHEL issue of a "rolling turd of backport fixes"/trying to figure out WTF version you're really running for packages, and the Ubuntu issue of not knowing exactly what's coming in the next release (users can easily deploy Debian Testing to validate the next version). Finally, the breadth of Debian's packages are outstanding, and Debian's backports repo gives you an easy, "blessed" way to use newer package versions without impacting your existing system.
Not advocating for Ubuntu, but there's also "devel" "release" where you can see the current state of next release. How's Debian Testing different from that?

Edit: it's "devel" not "develop"

Debian Testing comes in two flavors. Rolling and non-rolling. You can either always stay in the rolling state for Desktop and non-critical application or, get to the bandwagon early and end-up with stable release eventually.

To keep on rolling, you can use the keywords (stable, testing, unstable) in sources or use release names to stop rolling when something releases (buster, stretch, etc.).

The nice thing about using release names is the annealing effect during development. First major version bumps stop (feature freeze). Then minor version bumps stop (RC state / full freeze). Then one day you read that Debian is released. Congrats! A last apt-get and you've arrived in stable.

While I like this structure, it is confusing at first and can be hard to explain to stubborn managers.
Isn't that the same as Ubuntu devel?

You can always stay on devel, or switch to a stable release when (or preferably few days before) it's released.

I don't know. I started using Debian 15+ years ago and never switched to another distro :)

Had to reinstall my 8 year old installation to switch to 64 bit, and never reinstalled the new one.

These days there's a crossgrader that can switch from 32-bit to 64-bit -- and apparently, if you're feeling lucky and know what you're doing, from amd64 to arm64, assuming you're going to take the disks and move them to a new system.
You probably meant "Debian* comes in two flavors", but as you explained yourself, there are actually three: * stable (non-rolling) * testing (rolling) * unstable (rolling)

What's interesting is that close to a freeze, "testing" is quite stable already, so pinning to its name (so bullseye as of now) allows you to keep this rolling state until it converges to the next stable.

I understand he meant something different: that you can use Testing in two different ways:

1) by explicitly referring to "testing" in your apt configuration. Then it's rolling forever.

2) by referring to the current "testing", so for example "bullseye". Then you're temporarily on testing, rolling for a while until the day bullseye becomes the new stable, and then you're on stable too. I often do this a bit ahead of a new stable on my personal laptop, to check what's coming with the new stable. I guess it's probably quite common.

Yep. That's what I tried to say. :)
Debian comes in five suites actually: oldstable, stable, testing, unstable, experimental. There's oldoldstable but it's practically dead but, it's there.

You can manage these suites in two different modes: rolling (suite based) or non-rolling (codename based)[1].

As you also noted, testing has a sinusoidal rolling speed. It starts rolling slowly when a stable is released. Then rolling gets faster. Then slows down (feature freeze) and stops (RC/Full freeze). Then that version becomes stable. Testing separates and whole cycle starts again.

Testing never becomes unusable. Unstable has this risk. Instead testing is always tried to be kept in a semi-rc state. Any breaking bugs is a barrier for unstable to testing migration. As a result, testing can be used as a desktop system without any big drawbacks.

[1]: Since unstable is always codenamed sid and experimental has no codename, they always roll. Similarly since stable is rolled from release to release, it doesn't quite roll but backports and volatile allows it to creep forward in a controlled manner.

Debian testing doesn't get timely security updates so it's inappropriate to use on production servers.

Source: https://www.debian.org/security/faq#testing

I know. However, back then testing also was getting security fixes with 24h deadline but, since Debian got much bigger, it became unfeasible and abandoned, no biggie.

I always deploy current stable to any production server and I keep my desktop systems in rolling testing.

Ubuntu is based on Debian unstable, and if https://en.wikipedia.org/wiki/Ubuntu#Background is still up to date, they freeze packages from Debian unstable before releasing, which looks pretty similar to Debian Testing getting packages from Unstable that have not proved too buggy.

So you basically have this ordering more or less in terms of stability and how dated the packages are:

From older (and more tested) to newer (and less tested): Debian Stable | Ubuntu | Ubuntu devel <-> Debian Testing | Debian Unstable

Ubuntu devel <-> Debian Testing meaning that their orders depend on their respective phases, e.g. Debian Testing will soon freeze (prior to the next stable release) so in that time Ubuntu devel will probably be more up-to-date by virtue of being based on unstable. The rest of the time there's automatic migration from Unstable to Testing provided no blocking bug is discovered, so I'd expect packages in Debian Testing to be more recent.

Honestly I think the main differences are the default experience, but if you know which packages you want to use, Debian and Ubuntu should be pretty similar.

Debian also has single reboot, guaranteed stable-to-stable update.

We use CentOS at the office for obvious reasons but, my personal servers are always Debian since 2005 or so. Had no problems whatsoever.

Debian Testing is also a great, semi-rolling desktop distro. May be not the latest and greatest but, it's very very dependable.

What exactly are the "obvious" reasons (to you/your office) to use CentOS?
I'm an HPC administrator. Scientific computation and HPC ecosystem is built upon RedHat infrastructure. First ScientificLinux, then CentOS. To be able to run scientific software and project related middleware with ease, you need a RH-Style distro. That distro is CentOS.

(a little more insight is here: https://news.ycombinator.com/item?id=25349371)

... What do you mean by "RedHat Infrastructure"? What kind of packages are available for RHEL but not for say, Debian?
Nothing sinister. It's not about availability of packages. It's more like people develop and verify on RedHat based distros (because ScientificLinux was RedHat based). Some examples:

  - Software is released RPM first.
  - Software certification done on RedHat and CentOS (compile / verification).
  - Library versions present in RedHat and CentOS are used in development.
  - Documentation is written RedHat first.
You can do everything on Debian. Possibly something would be hardcoded and you need to work around or re-compile it. Or, it works without hiccup but, developers can't provide support since they didn't test on Debian.

Scientific software is a very different beast. Its accuracy and precision sometimes depend on the mood of a newborn butterfly in Zanzibar. I develop such software so, I know a little bit about the peculiarities.

Since RedHat also has paid support, RH based distributions are considered more sound and mature, because if something breaks, RedHat can fix. It's no different for Debian but, RedHat was able to fly an engineer for NEC Corp. to reproduce and fix a GCC bug on-site. System in question was Itanium based IIRC.

That isn't it, it is just that tools are built for RH-flavors first, and used on RH-flabored distros.

I'm sure you can get this stuff working on Debian. I'm also sure it is much more of a PITA to do so, just because that's not where the eyeballs are.

I also work in HPC, and our current cluster runs Ubuntu. We're a bit of an outlier though, as you say CentOS is the most common platform.

A lot of the ML/DL stuff is developed on Ubuntu first, causing occasional headaches when deploying on CentOS, similar to traditional HPC except the other way around. Oh well..

As ML gets more popular, and now this, maybe there's a chance of breaking the RHEL (rebuild) monopoly in HPC.

Oh, hi! It's interesting to see clusters running on different operating systems.

Seeing Ubuntu on the scene is nice. From what can we see, our users can run their ML/DL loads relatively effortlessly on CentOS for some time.

Moreover, with GPU support on Docker, we guess that people will migrate to containers to run their loads so distribution will become irrelevant in the future.

Do you use IB on your nodes? If yes, is it easy to deploy the kernel modules?

> From what can we see, our users can run their ML/DL loads relatively effortlessly on CentOS for some time.

Yes, containers are a god-send for those apps that require some specific version of a specific distro.

> Do you use IB on your nodes? If yes, is it easy to deploy the kernel modules?

Yes, we do. We use Mellanox OFED, which supports Ubuntu, so it's no more difficult than on CentOS. Perhaps even slightly easier, since minor Ubuntu version updates don't break the kernel internal ABI's so there's no need to wait for all your out-of-tree kernel stuff (MOFED, Lustre, whatnot) to be updated before you can upgrade.

> I think Debian is the sweet spot for release cadence--stable is supported for ~3 years, and there is a new stable released ~2 years.

Security fixes are for 5 years.

That's including volunteer LTS support, which only applies to an unspecified subset of packages. Full support is ~3 years, per parent comment.
The LTS team mostly consists of developers paid by Freexian, who are funded by various large and small companies.

https://wiki.debian.org/LTS/Team https://wiki.debian.org/LTS/Funding https://www.freexian.com/services/debian-lts.html

The LTS project supports most packages and the unsupported subset is specified, with the debian-security-support tool able to report unsupported packages that are installed:

https://salsa.debian.org/debian/debian-security-support/-/bl... https://packages.debian.org/stable/debian-security-support

(comment deleted)
Personally I will advocate for Flatcar Linux in my organization for any on-metal deploys.
So why not Ubuntu instead of Debian?
Snaps
How does that prevent you from using Ubuntu as a server with docker or something else of your choosing?
Remove snaps:

  sudo apt-get remove snapd
  sudo apt-get remove lxd-agent-loader # 20.04
The key to keeping them uninstalled is to pin the packages so they don't get "upgraded," which results in them being reinstalled.

  sudo apt-mark hold snapd
  sudo apt-mark hold lxd-agent-loader # 20.04
If you want to go the next step in cleanup:

  sudo umount /var/lib/snapd/snaps/*.snap
  sudo rm -rvf /var/lib/snapd/*
  sudo rm -vf /etc/systemd/system/snap-*.mount
  sudo rm -vf /etc/systemd/system/snap-*.service
  sudo rm -vf /etc/systemd/system/multi-user.target.wants/snap-*.mount
  sudo rm -vrf /snap/*
There's apt-get purge or apt remove --purge to do those manual steps.
I guess it is because Ubuntu is also corp-controlled (Canonical). Personally I use Ubuntu server. But it is much easier to migrate from Ubuntu to Debian than from CentOS to Debian.
Question from a current Ubuntu user that I also asked myself:

Why Ubuntu instead of Debian?

There are definite advantages in running the same thing on servers as devs are running on laptops. (And some disadvantages.. but the balance ends up on the plus side)
Why do people run Ubuntu on PCs?

It's an everchanging, relatively buggy project, without upgrade support, and full of those weird choices (unity, phoning home) it tries to impose onto every user.

Commercial ISVs that target the desktops/laptops of gamers, consumers, and trendy cloud-focused startup devs are more likely to pick Ubuntu than Debian as a supported OS if they don't bother to support both. And Ubuntu makes it easier to trade off software freedom in favor of just having things work out if the box, though both provide ways to do so.
I have had similar odds of stuffing working out of the installer on Ubuntu and Debian.

The main difference is that in Ubuntu the fix usually involves removing some program that is messing with everything. While in Debian it usually involves adding some program that isn't installed by default for some reason.

(Debian would gain a lot by making an installer with the backports kernel, by the way.)

Heh, yeah. It sounds like you prefer Debian in general, and so do I - not only am I a Debian developer too, I'm typing this on my personal laptop running Debian. That said, sometimes I pick Ubuntu in a work context to be less of an oddball, since my work contexts tend to be the trendy cloud-focused tech startups where Ubuntu is comparatively common and I want to direct my day-job sysadmin energies toward my employers' infrastructure instead of my laptop.

Despite that, at the company I'm helping to found as CTO, I may need to set a default of Windows in the short term: the story for using MS Office and screen sharing / other fancy features of Zoom on Linux isn't great, web dev on Windows is getting surprisingly good with things like WSL and VS Code, we don't already have a fleet of Intel-based Macs to use, and for now the new M1-based Macs are likely to demand workarounds for the CPU architecture on which we can't spend the necessary time at this extremely early stage of our corporate lifecycle.

Macs will probably become a viable option for us again in a couple of years, and Debian / Ubuntu will probably be viable from day 1 for any technical employees who don't need the commercial software I listed above.

What makes you think it's without upgrade support? Upgrades are supported, and work well IME.

Ubuntu today is much less opinionated than before in desktop use, since they switched from Unity to the same Gnome that most other distributions are using.

That can be interpreted either way. For people in the trendy cloud startup world that we often focus on here on HN, Ubuntu is probably more popular in the server world than Debian, but overall I suspect it's exactly opposite.
Then use Debian on laptops as well? Both Ubuntu and Debian run on a wide range of hardware, so I don't see that as an argument in any direction.
Unless it's changed a lot recently, Ubuntu support for new laptops is better because Debian has slower release cycles & generally doesn't fast track fixes for newish laptops in drivers. Also there seems to be a bigger new laptop user community on Ubuntu so you can get better information on which laptops work 100% and workarounds for the rest.
Possibly because of some commercial proprietary software support. Typically they support RH, now sometimes Ubuntu LTS (trending up). Vary rarely Debian, unfortunately.
With Ubuntu, anything not in the main repo (universe, etc) doesn't get security updates.

From personal experience, I've broken way more ubuntu installs on a release-upgrade than I have on debian (zero).

Universe may receive security updates. There is just no guarantee since they are not supported by Canonical.
I use arch for my desktop (cutting edge libraries and api!) and debian for my servers (never fails).

Ubuntu doesn't fit for me

Debian provides security support for way more packages and way longer.

Also, Ubuntu adds some bloated software of dubious quality.

Looked at a server ssh I had open to an 1804 ubuntu, and on my desktop

First thing /etc/update-motd.d/50-motd-news does is source /etc/default/motd-news if it exists

Only if that file sets "ENABLED" to 1 will it do anything

  [ "$ENABLED" = "1" ] || exit 0
I haven't knowingly deleted that file, not sure how it opts in. Perhaps this line in my preseed removes

  popularity-contest popularity-contest/participate boolean false
Notably though, with base-files 10.1ubuntu2.10 and

  $ dpkg -L base-files | grep motd-news
  /etc/update-motd.d/50-motd-news
  /lib/systemd/system/motd-news.service
  /lib/systemd/system/motd-news.timer
There's a mention on the prerm about removing that conffile.
Debian runs old kernels.
Debian provides up-to-date kernels in the stable-backports repository. You get a choice of what to run. And nowadays even the 'old' stable kernel is a LTS release so there's no support issue.
The missing piece here is drivers. Anyone who sells servers will make sure there are functional drivers for RHEL. Using a RHEL clone means you can always be sure to find drivers for your servers. Using a RHEL clone as the core of your product means more or less the same thing. Can the same thing be said for Debian?
The solution to this problem is either dead simple, or impossible (depending on your specific field of work, I guess): You do not buy or deploy hardware that doesn't have mainline support at the time of your specific Debian release's kernel version (or its backports variant, if need be).

$prevjobcompany lives by that unwritten rule for more than two very profitable decades by now, and I recommend anyone considering operating a Linux-based OS to adopt it.

> You do not buy or deploy hardware that doesn't have mainline support at the time of your specific Debian release's kernel version (or its backports variant, if need be).

There are glacial changes, but scientific instruments are often still shy on Debian or even Ubuntu :(

Do you have the source for these drivers? It probably isn’t too difficult to make them compile for a slightly different kernel
Hah, I wish! If it has a source, then it will probably already be on mainline and the issue will be moot. These are at least a bit free-er than some other instruments which rely on Windows (and maybe not even 10 or 64-bit at that).
If these are Linux kernel drivers then it sounds a bit like they might be GPL violations?
I don’t think this is a real problem that people encounter in 2020 unless they’re using some truly exotic hardware.
> unless they’re using some truly exotic hardware.

We're one of those (who uses, not who manufactures)! But in most cases it is not a question of Linux distribution and more of is it Windows (hopefully not XP) or not :(

> Can the same thing be said for Debian?

It can't. Not just drivers, firmware updates for BIOS among other things. Ubuntu is getting there but it isn't considered as seriously by all manufacturers.

> firmware updates for BIOS among other things

There's a growing distro-agnostic solution here, LVFS fwupd https://fwupd.org/

Too much of discussion has been focused on the system itself but aren't we forgetting the most important thing - all the packages that organizations depend on, some of them build in house with no resources to port to another system. It isn't as easy as everyone makes it to be, switch to Debian - ya right, rebuild the whole infrastructure - good luck with that.
I have been using Debian GNU/Linux since 2006 (Debian 3.1 Sarge). The primary reason for choosing Debian back then was the large number of packages Debian had in its repositories. Over these 18 years, I have used Debian for frontend servers, backend servers, personal laptops, personal virtual machines, team virtual machines, etc.

What I have realized in these years is that its stable branch is extremely stable. I have never ever faced any issue with dependency management or package installation with the stable branch. Yes, the packages can be sometimes a couple of years old but the whole system is exceptionally stable. I believe it sets a benchmark for stability.

I used Debian on my home PC/laptop/PC from potato (2000ish), then testing, upto about 2008, but moved to Ubuntu LTS in 2008 for hardware purposes. On my desktop ran 08, then rebuilt to 12, then rebuilt to 16. Last night I tried doing an upgrade from 16 to 20 (via 18). Took about an hour to go through both.

A couple of niggles so far 1) nvidea drivers broke, had to install the latest version with apt 2) Sound wasn't working from user accounts, pulseaudio complaining about unable to access the files. lsof showed timidity had snd files open too, and removing that seemed to do the job 3) clusterssh reported a perl issue - for some reason I had a ~/perl5 folder in my home directory

vlc looks weird now too, but overall not too bad, and will last another 4 years. Not sure if this desktop will last beyond 2024 - it's a 2015 HP Z440 with a Quadro K2200 graphics card.

Over time I've seen bad things with ubuntu -- changing networking, breaking logging by forcing systemd, etc. Ultimately though you can only delay change, but like Cnut, there's no much you can do about the incoming tide. Having breaking changes every 4 or 5 years is the price to be paid.

Server wise we never upgrade, reboot, pxe boot, build the latest, then run the installation commands. Had various issues over the years with updating internal packages (apache 2.2 to 2.4 changes come to mind)

I've always had a personal preference for debian/ubuntu, but in practice I've had to work on applications that ultimately had to run in highly regulated environments where RHEL was the only approved platform. Having the ability to have lots of testing environments running CentOS without huge licensing fees was a win. I don't know the specifics of pricing but I hope RHEL can ease the burden this may place on startups etc by offering a generous free non-production licenses if they don't already.
How much of a boost will this change by CentOS give to efforts to get Docker running on FreeBSD? I get the impression that many here are open to adopting FreeBSD but that Docker support is a non-negotiable requirement.
I’m a big fan of FreeBSD (in fact, I was planning to install CentOS on a new storage server, but I’ll probably go back to FreeBSD instead). But, I imagine anyone wanting to run Docker on FreeBSD would want to be able to run Linux-based containers. In which case, you’d end up with a Mac like situation — you could have command line Docker communicating with a Docker daemon running in a Linux VM.

If that’s what you’d need, then why not just run a Linux distro to start with?

If you are looking for a Debian-based release for the desktop, take a look at Pop!_OS. If you can look past the appalling bad taste in naming, it's great. The software "shop" GUI makes it very easy to pick between .deb or flatpak (where available), but of course you can do it all from the CLI.

It has sane defaults (if you are OK with Gnome) and easy installation of (non-free) nVidia drivers / CUDA / tensorflow. Steam works out of the box.

It's as close as any OS (Mac/Windows) or other linux distro I've used to being ready to go from the first login, and staying that way with minimal fuss.

I can also vouch for POP. I run it on my laptop and my family's computers as well.
I suppose I'll jump on this bandwagon as well.

It is a distro that feels like a full OS instead of a random conglomeration. My only negative with it is it uses Systemd in general, and Systemd boot in particular. If you are fine with that, it is just about the perfect distro.

(I prefer open-rc as an init system/service manager, I recognize that Systemd makes sense for Pop.)

Steam is included in the Debian non-free repos, so it ought to work in Debian as well. In general, Debian (Stable or Testing) makes a fine desktop distro, aside from the well-known issues wrt. proprietary drivers.
I'm fairly sure that Pop!_OS is Ubuntu based rather than Debian based.
Having lived through the transition from “using expensive commercial tools that run on expensive commercial UNIX” to “using expensive commercial tools that run on RHEL” I think I can tell you exactly why Debian was not chosen as the successor to expensive commercial UNIX.

First of all, the general transition to Linux was started by engineers, customers of UNIX, who were tinkering with Linux. We weren’t software engineers and not super interested in how Linux worked, just what we could do with it. Debian was a chore to install, in the way of getting things done. Red Hat, on the other hand, was easy to install. Red Hat was the distro of choice for experimenting with Linux.

The other side of it, the business side, is that RHEL met the need of, “see boss, this is fully supported, just like UNIX.” That had to be a thing before these types of businesses would transition from UNIX to Linux. Only after that transition would front-line engineers be brave enough to sneak a RHEL clone like Centos into the mix. Sadly, Debian just didn’t stand a chance in this situation.

Fast forward to today and now you see front-line engineers hacking Ubuntu LTS distros to look enough like RHEL for the commercial tools to run on it. You still have to keep a RHEL box around to reproduce issues before going to support with issues in the commercial tools, but that’s much cheaper than a whole fleet of RHEL boxes.

> Debian was a chore to install, in the way of getting things done. Red Hat, on the other hand, was easy to install.

Debian was a relative chore to install in the pre debian-installer days. But d-i has been a feature of Debian since Sarge (v3.1, released in 2005) and it brings Debian to parity with other distros.

Being fully-invested in one distro in the first place seems like an old-school approach to me, and I'd assume that, with distros like Alpine and BusyBox (maybe even throw macOS in there), it's more common for most people to deal with multiple distros than they're giving credit for. I've more or less gotten used to the cycle of figuring out which launch system and package manager I'm dealing with for any given system and having Google/StackOverflow open for the things I forget.
I would argue Ubuntu is better suitted nowdays for servers vs Debian, mostly because it has support from a corporate and also because Debian once again is running older packages, notably libc / kernel.
Depends on what. For your Desktop Debian is not good, but bearable because of the variety of packages. Fedora still the best, or Arch if you want to break your machine every few months. Some even like Ubuntu or Mint, but those should be ignored nowadays.

But if you need a stable and secure server Debian or Ubuntu are a joke. There are no engineers with Debian, only lawyers and community managers. There your can use RHEL, Oracle Linux (with dTrace and ZFS), or SLES. Those do have the experts. For an unstable server CentOS Stream is fine.

As noted in other HN discussions, I have found Debian (a user since 1997) to have the highest quality of the distros I have tried (except maybe Devuan which is a close Debian derivative). Specific things worked reliably which seemed like they were inconsistent, broken and/or untried by their developers, elsewhere. And the Debian Policy makes packages and the system as a whole work well, more consistently, IMO.

(Though I mostly use OpenBSD now -- when I have to use a linux, it is debian or devuan.)

Edit: also, Debian is very easy to install, has extensive documentation, and (IIRC) commercial support is available. There are reasons that there are more eventual Debian derivatives than of any other distro.