43 comments

[ 2.8 ms ] story [ 116 ms ] thread
That's strange. CentOS 6.x is still very popular.
And shipping with kernel 2.6 and being 4 years old.
yeah it makes sense the 2.6 kernel is too ancient. They have kept it patched up, but upward and onward!
Yeah it's huge in the enterprise. Not everyone has migrated to 7.x with systemd yet.
Far from it, I would be surprised if even most RHEL installations ran RHEL 7 yet.
But it is very old, and does not support a lot of features that new software like docker needs. The enterprise release cycle is just too slow for software that is dependent on kernel features.
Compatibility with CentOS was a big driver to adopting Docker at my prior employer. It's benefits were made even more pronounced on semi-legacy systems like that.
Completely agreed. It has been a huge help in the migration of legacy systems to containers while underlying monitoring and config infrastructure is being updated to support systemd.
You can still use 6.x as a base layer inside the container, just not on the host OS.
For anyone else curious, Docker 1.8's target release date is August 4th, so basically in a month all support will be dropped for RHEL/Centos 6.x. Definitely a bit concerning to me.

I'm quite interested to see how the whole Open Container Project pans out, and I'm pleased by the prospect of it, just hoping they successfully iron out some of the "Docker wrinkles" that have shown over the course of it's lifecycle in the last few years. No 6.x support from the get go doesn't necessarily put a great taste in my mouth, so far.

Super worrying to me too. 1.8 marked the move to Open Container Project. I guess this means Open Container Project won't be supporting 6.x.

It would have been nice to get more than 1 month of notice.

(comment deleted)
Interesting sidelight: DoD uses RHEL almost exclusively, so this sort of implies that DoD has no interest in Docker as their container of choice.

Edit: to clarify, I was thinking the HPC systems are RHEL, and account for a fairly large number of RHEL licenses. Certainly the "enterprise" is outlook, sharepoint, and a ton of Win7 terminals.

Interesting. All the DoD systems we interface with run Windows. There may be a mixture, we just don't work with those systems.
Valid point. I guess I was thinking the HPC systems. Certainly every client-facing terminal is a Windows box. I don't know much about the network stack, but there's clearly a lot of Windows there too.
We're an external partner pulling data via a 24/7 Windows-based web service, a system of record. The three we work with are all Windows, but three is likely a blip in their system count.

All I know is it's weird how they and their vendors manage the systems and environments. Glad it isn't me!

This is terrifying on so many levels, but hardly surprising.
I'll just say that your intuition on this one is not correct.

Take a look at the GSA presentation with Booz Allen at DockerCon.

DoD is massive and there's a ton of different deployment requirements all over the place. On the systems I've worked on, they've ranged from old RHEL 4.x versions up to CentOS 7 now and also SunOS and Solaris and everything in between. There's been some movement away from COTS software in favor of just paying a contractor to do the support for you when you already pay them tens of thousands per week for development and support in the first place, but this is also tough to generalize. I can only say that DoD has been increasingly budget strapped in most R&D areas and has been clamping down hard on costs (in probably the wrong places as usual, but there's structural problems that keep DoD and federal sector in general from doing the most cost-effective purchasing).
OpenStack dropped support for CentOS 6 in its Kilo release, so this isn't too surprising.

CentOS 6 was released in 2011, which is 4 years old. A lot of features Docker uses have been highly improved since the 2.6 days, so it makes sense to drop support for operating systems which don't support some things.

Correction, openstack dropped python 2.6 support in its kilo release, you can of course run python 2.7 on centos6.x/rhel6.x (and I know companies doing this, including godaddy and yahoo) via a variety of manners, and then openstack will work for you again (until u can shift to centos7.x or rhel7.x).
I guess that Red Hat will have to keep supporting it with security updates then?
As per their support model for 6.x, but, in the description:

RedHat themselves only support Docker on RHEL7 (as stated by https://access.redhat.com/solutions/1378023, and confirmed by several people)

Right but this is odd considering they used to support it and only quietly updated their support document in May. If a fellow issue commenter on github hadn't linked to it I (and everyone else) would have missed it.
I don't know what they've said when publicly, but the intention of 6.x on their end related to support for Docker has always been best efforts in terms of their support practices. Docker support by Red Hat in RHEL7 is actually a compelling differentiation they use in the field to get their existing customer base to upgrade.

This conversation goes back to the moment we started talking about it before the announcement[1] in September 2013

1: http://www.redhat.com/en/about/press-releases/red-hat-and-do...

Docker isn't part of RHEL, it's only available on RHEL6 as part of EPEL. The EPEL package maintainers will, in theory, keep supporting the rather old version of Docker (1.5) in EPEL for another 5 years.
Not necessarily. EPEL is based mostly on volunteer effort. Package are supported only if there is active interest. If those mainters move on to something, Docker 1.5 will be orphaned and then removed from the repo.
I have a very limited understanding of how distributions work so this is probably a stupid question but why don't RHEL/CentOS update the Linux kernel they ship with their 6.x OS? Isn't the kernel supposed to never break user space apps?
They don't update whole kernel, but they backport features and bug fixes from newer kernels to the ones they ship.
RHEL focuses on stability over bleeding edge features. What this typically means is that you will have an older kernel / user land packages installed, then Redhat backports security patches into it's maintained ports. You can read about it here [1]. The end result is a fairly stable kernel and system OS. But, you have to live with an older feature set (think 2-3+ year old package with just security patches applied).

I've maintained lots of RHEL boxes as a sysadmin -- here's my take. This type of release cycle works pretty well, in that you can almost always apply patches without hosing your system due to "new features". One example of something this protects you against, is a package changing the expected config file format, or maybe the expected command line arguments. Another good example, would be a change to a kernel driver for some storage array, or network interface, that could quickly break lots of systems!

This type of release cycle also gives commercial software vendors a known target to develop against and sell support for. Say for example, that you are using an enterprise database like Oracle/Sybase/DB2, they will verify their product works against RHEL 5.x/6.x/7.x and give you commercial support, as they have a pretty good understanding of what you are running. But, if the kernel/packages are always moving targets, it would be near impossible to support your end platform, so this also adds to the backporting of security patches without adding features thought process.

Although, it is also a pain, in that you are typically way behind on cool things happening out there on other distros. So, there are pros and cons to it. But, if you are running a large mission critical DB cluster, you typically want things to be extremely stable, so for the target market (the enterprise) this works pretty well.

[1] https://access.redhat.com/security/updates/backporting

There are definitely upsides and downsides. The vast majority of our systems are RHEL6, because that's what a vendor of ours supports. Since we have the infrastructure around to support RHEL6 (puppet modules, vm templates etc), most of our supporting systems are also RHEL6.

If we need newer features or packages, we spin up a CentOS7 or RHEL7 image which still isn't bleeding edge, but is significantly newer than RHEL6.

It's nice to have a bit of stability for apps and services that require it - but there's always the option of bringing newer versions online for the services that need it.

As much as RHEL is super conservative, Fedora is the polar opposite. They'll wantonly break things, throw things out, completely overhaul how things integrate.

For example, in previous releases they've removed ifconfig, and lately, even yum got chopped.

It's like living a few years in the future. systemctl was there years ago.

The kernel isn't just about user space apps though - it's about interaction with hardware. Updating the kernel can have large (or subtle) impacts on the performance and characteristics of hardware, which in a production environment can be dangerous, hard to track down, and impact your application anyway.
The issue of bringing anything like Docker (app virtualization) to RHEL/CentOS/SLC/CERN/Rocks 6.x is supporting an old kernel, adding custom-compiled kernel modules or compiling a custom one. I had this exact conversation with some clients whom I did some hypervisors work with and whom recently sold their startup to one of (VMware, Citrix, Microsoft). In fact, about 1-2 years ago we joked they should've bought dotCloud (now Docker).

PS: Docker is a prime example of how a side feature can be instrumental to a pivot to something much larger.

Linux does introduce bugs, which change the interface, but also adding new features can change the way programs behave, eg if the program detects features at run time and uses them.
I am not surprised. RHEL 6 is getting long in the tooth!
I use cent OS 6.6 on a server because it is similar to the software our customers run. I also like using docker even though our customers do not. At this point I'm just going to install ubuntu server and use Vagrant so I can run what ever OS I need.
I don't know why they ever bothered.

People running RHEL are among the most conservative Linux users there are. We have some of that where I work. We have big production systems still running RHEL 5 (which RedHat still supports, and for those systems the less you change the less of your time they demand).

Anyone still using RHEL 6 is not really the sort of person who jumps onto the latest container or cloud frameworks the moment they are released. If they are into virtualization at all, they are probably using VMWare.

I have to argue with your point a bit. I actually prefer RHEL (CentOS) and if given a choice, I still run CentOS. It is what I started with, and now am most familiar with, but agree the package versions are super old and outdated. I do like yum more than apt, but probably again because it is was I'm most familiar with.
I agree. I'm more DevOps, (really, just a dev that is super comfortable in an ops position) and my go-to distro has always been Debian, but nearly every position I've had has been supporting applications on Ubuntu. It wasn't until my last position when we were acquired and thrown into a CentOS ESXi datacenter that I really started to love CentOS's release cycle. You won't get all the somewhat recent or cool features without feeling like you're treading into the territory of hacks and warranty-voiding (try running Xen on Cent or RHEL), but you'll also not dread keeping your servers up to date, or keeping on top of the latest logoed vulnerability sweeping the net. I'd guess that 99% of the standard LAMP and at least 80% of the common Rails, SQL, K/V almost-crud apps could benefit from CentOS.
Most people running RHEL and RH-based distros tend to want to stay behind the curve as much as possible for many (typically legitimate) reasons. When a rash of OpenSSL vulnerabilities showed up, some older distributions were not affected because the packages they were on were so old they hadn't picked up some of the code that had been patched in for over a couple years that was flawed.

But users and developers on these systems do want to have a chance to run some modern stuff like containers as well because there's a strong chance that they'll have to support their software for 10+ years potentially. On one system I worked on I saw that the architecture called for support for the same exact distribution for over 10 years - these are the kinds of places that will be paying Microsoft extra to support XP n years after it's gone EOL.