Kubernetes is opinionated in ways that Mesos is not, and fits together significantly better than Docker swarm does - And we're still only seeing the tip of the kubernetes iceberg.
To put it bluntly, Kubernetes benefits from all of Google's borg expertise while Mesos and Swarm are trying to reinvent the wheel, and there's a lot of chesterton's fencing going on - They're adding and removing features without understanding why those features do or don't exist in Borg.
Mesos requires you to run too much - There's barely a default scheduler. Swarm doesn't require you to run enough - There's barely equivalents of "Services" and it's all designed for plugging in packages without thinking of how they fit together. Kubernetes strikes a balance.
Extensions to Kubernetes like istio are what's next on the horizon, and good libraries to deal with the additional details in the k8s ecosystem (things like grpc-channel balancers where you specify a service and a mode and it automatically routes your requests and deals with failover and retry logic). We don't have a good Kubernetes Mapreduce framework yet, but it's gonna show up soon. There's a lot more as knowledge leaks out of Google.
There are several factors. Technical brilliance is one, but Kubernetes is of course not entirely alone there; DC/OS (which is what you usually would be talking about these days if you're saying "Mesos", since Mesos is just a scheduler and doesn't in itself provide most of the features you need) is also highly regarded. Swarm is not highly regarded, and often disparaged as an inferior imitator.
Another big reason is better marketing and evangelism. Kubernetes has a lot of companies betting on it and who are actively promoting it. It has the CNCF (Cloud Native Computing Foundation) [1], a suborganization of the Linux Foundation. While CNCF also includes Mesos, it was initially, I'm told, invented as a way to move Kubernetes out of Google, thereby commoditizing and democratizing it as an open-source ecosystem; Kubernetes was the first CNCF project.
It's a savvy move on Google's part. It's reminiscent of Sun's "standardizing" of J2EE, JMS, etc. back in the day, which in itself spawned a whole cottage industry of enterprise software and certifications. CNCF/Linux Foundation, of course, offer certifications, as well as events (KubeCon) and training programs, all of which involve marketing and evangelism.
So there's a whole engine behind Kubernetes which make the competitors (of which we have Hashicorp Nomad, DC/OS, Amazon ECS and to some extent CoreOS (at least their original cluster design) and Rancher) seem like children's lemonade stands by comparison.
But even with all of this momentum, none of this would work very well if Kubernetes weren't popular with developers and sysadmins. Kubernetes has been enthusiastically embraced for its excellent design, and so there's little pushback from below; when devs tell management they should be betting on Kubernetes, it's easy for them to say yes, but the opposite is also true. And since Kubernetes also works to commoditize containers, it ends up being one of those rare projects that appeal equally to developers, sysadmins and suits. The pointy-haired bosses at IBM and other big corporations realize they can leverage the platform for their own enterprise offerings.
So you could say that Kubernetes scales all the way down to the individual developers, and also all the way up to the highest levels of big corporations. It's the Linux of container orchestration.
I asked a representative of Cloud Foundry this year why they are switching from diego to kubernetes and he told me: "Three reasons: google, google, google."
I was not able to get out more information but am guessing google just put more money / manpower / advertisment into kubernetes than cloud foundry into diego.
9 comments
[ 69.2 ms ] story [ 904 ms ] threadTo put it bluntly, Kubernetes benefits from all of Google's borg expertise while Mesos and Swarm are trying to reinvent the wheel, and there's a lot of chesterton's fencing going on - They're adding and removing features without understanding why those features do or don't exist in Borg.
Mesos requires you to run too much - There's barely a default scheduler. Swarm doesn't require you to run enough - There's barely equivalents of "Services" and it's all designed for plugging in packages without thinking of how they fit together. Kubernetes strikes a balance.
Extensions to Kubernetes like istio are what's next on the horizon, and good libraries to deal with the additional details in the k8s ecosystem (things like grpc-channel balancers where you specify a service and a mode and it automatically routes your requests and deals with failover and retry logic). We don't have a good Kubernetes Mapreduce framework yet, but it's gonna show up soon. There's a lot more as knowledge leaks out of Google.
Another big reason is better marketing and evangelism. Kubernetes has a lot of companies betting on it and who are actively promoting it. It has the CNCF (Cloud Native Computing Foundation) [1], a suborganization of the Linux Foundation. While CNCF also includes Mesos, it was initially, I'm told, invented as a way to move Kubernetes out of Google, thereby commoditizing and democratizing it as an open-source ecosystem; Kubernetes was the first CNCF project.
It's a savvy move on Google's part. It's reminiscent of Sun's "standardizing" of J2EE, JMS, etc. back in the day, which in itself spawned a whole cottage industry of enterprise software and certifications. CNCF/Linux Foundation, of course, offer certifications, as well as events (KubeCon) and training programs, all of which involve marketing and evangelism.
So there's a whole engine behind Kubernetes which make the competitors (of which we have Hashicorp Nomad, DC/OS, Amazon ECS and to some extent CoreOS (at least their original cluster design) and Rancher) seem like children's lemonade stands by comparison.
But even with all of this momentum, none of this would work very well if Kubernetes weren't popular with developers and sysadmins. Kubernetes has been enthusiastically embraced for its excellent design, and so there's little pushback from below; when devs tell management they should be betting on Kubernetes, it's easy for them to say yes, but the opposite is also true. And since Kubernetes also works to commoditize containers, it ends up being one of those rare projects that appeal equally to developers, sysadmins and suits. The pointy-haired bosses at IBM and other big corporations realize they can leverage the platform for their own enterprise offerings.
So you could say that Kubernetes scales all the way down to the individual developers, and also all the way up to the highest levels of big corporations. It's the Linux of container orchestration.
[1] https://www.cncf.io/