I never see this mentioned, but to me the most important feature of Docker based CaaS (or PaaS) is that they are no longer purely for hosting custom software. The focus of Heroku style PaaS systems has been the flow of code to running software, but what about applications I have no code for?
I want to run GitLab, Mattermost, an API gateway, etc. With Docker based CaaS/PaaS, this is easy since apps can be easily packaged as containers that are treated the same as your own custom apps. This is only my opinion, but I think we are getting to the point where all server software can be run on something like Kubernetes and traditional virtualization becomes even more of a commodity.
Disclaimer, I work for Red Hat! We have both a CaaS (RHEL Atomic Enterprise Platform ie Kubernetes) and a PaaS built on top of it (OpenShift v3). Containers taking over the entire data center is my own opinion, but it would be great!
I am in complete agreement. I don't think it's a question of if, and more a question of WHEN everything in the data center is containerized. The benefits far outweigh the negatives (like educating your whole staff on a new workflow or set of tools), and it seems that ease of use when it comes to the ecosystem/tools/orchestration is really the only hurdle keeping it from getting there. We're working on making those "non-custom" software packages easy to stand up and use via http://containership.io/#marketplace
Awesome that you want to run GitLab in a CaaS. We want to make sure you can deploy all your software from GitLab to Kubernetes without using any glue. Work on this is being done by the Redspread team in https://gitlab.com/gitlab-org/gitlab-ce/issues/1300648
Yes, I've been following that! GitLab Deploy is also looking like it could be exciting. I was disappointed though that the merge request that added Kubernetes support to the GitLab CI MultiRunner was rejected. Right now Jenkins is usually the default choice for our customers that use OpenShift, but being able to dynamically provision testing containers would be an amazing feature.
That would be great, I'm not married to the implementation as much as the convenience of using the cluster manager we already have (kubernetes) to distribute builds.
I currently am using a VM with a docker runner, and there are two (somewhat obvious) issues:
* It requires root access to the machine to be able to spin up containers
* Every VM with a docker runner needs enough storage to handle every docker image used for builds, instead of centralizing the images. I really need to put in a cron job or something to clean these up.
Kubernetes should fix both of these, without introducing another technology in our environment like Docker Machine.
At this point it doesn't make sense to introduce support for another clustering manager. The GitLab Runner does support docker-machine (you can pretty easily configure it to create machines on-demand on any cloud provider). These machines will be pretty much self-managed. Soon the Runner will be also compatible with Docker Swarm, the implementation is already on separate branch.
From my perspective Kubernetes is great as platform for deploying applications, but I'm not so sure if it comes for distributing builds.
What is important that GitLab Runner is being constantly improved and at some point with more convincing (for example coming from You) we can get back to native Kubernetes executor :)
Glad to hear you're following that. Can you link to the Kubernetes merge request? I think it introduced too many dependencies but I'm happy to have another look.
I think you're looking for Chef or Puppet recipes, real configuration management tools which can flexibly deploy a complete stack, not some Bitnami toy which doesn't solve anything.
The limitations with most all container platforms is what they don't do major things which need to be addressed somehow in production systems:
- hard resource isolation (cpu, ram, network, IOPS)
- [something else obviously important here]
Containers are great for limited testing and local development, but for production they are like anti-type 1 hypervisors... more trouble that they're worth (docker instances still get stuck all the time as zombie processes, requiring a hard host reboot).
Worse, most of these containers are built by unknown persons, are unsigned and hosted publicly, so it's effectively running untrustworthy code however they decided to package and deliver it.
Finally, to add insult to injury, there's no reliable way to rebuild an image from scratch without some sources, whereas configuration management tools are intended to describe how to deploy apps and put systems into a desired configuration.
I actually am very experienced with Puppet, and I agree configuration management tools still have a (now narrower) space.
But most of your concerns are actually handled by what we are doing at Red Hat. OpenShift v3:
* applies SELinux policies to each running container
* can handle multiple styles of container upgrades (rolling, green/blue, canary, etc)
* includes an automated build process for automatically patching all of your containers when there is an update to their base image (including supported releases of RHEL containers that are kept up to date with errata)
* automatically reconfigures iptables as containers go up/dowm/migrate
* collects metrics on all running containers
* health checks for running containers to ensure they are not zombies
Since it is built on top of Kubernetes, the only configuration that you need to apply/maintain is the pod description, a short json document that describes what to run, what ports to expose, etc.
There are various reasons, but here are a few of the big ones:
1. Getting ahead of your costs before they start to shoot up as you grow. Check out this pricing calculator for an idea of what I mean: http://containership.io/#pricing
2. Cloud portability: Heroku runs in 2 AWS regions (us-east-1, eu-west-1). What happens if you want to host in a specific country, a completely different provider (maybe you have some free credits?) or even your own data center eventually? Not gonna happen with heroku.
3. More flexibility to run the types of databases, queues, or other open source software that isn't possible to run on Heroku.
When you go from $100 Dedicated Server with 10TB of bandwidth in average, to $900+ in AWS just for the bandwidth, well, you gotta think about it. Heroku puts a 2TB/month BW soft limit, that won't cut for some of us.
22 comments
[ 3.1 ms ] story [ 57.0 ms ] threadHere's both links to make it a little easier:
http://blog.containership.io/iaas-vs-paas-vs-caas-which-clou...
http://blog.containership.io/iaas-vs-paas-vs-caas-which-clou...
You mean ones without any employees or an office?
I want to run GitLab, Mattermost, an API gateway, etc. With Docker based CaaS/PaaS, this is easy since apps can be easily packaged as containers that are treated the same as your own custom apps. This is only my opinion, but I think we are getting to the point where all server software can be run on something like Kubernetes and traditional virtualization becomes even more of a commodity.
Disclaimer, I work for Red Hat! We have both a CaaS (RHEL Atomic Enterprise Platform ie Kubernetes) and a PaaS built on top of it (OpenShift v3). Containers taking over the entire data center is my own opinion, but it would be great!
If so, then the wikipedia disambiguation page (https://en.wikipedia.org/wiki/CAAS) redirects to a yet unwritten one:
https://en.wikipedia.org/wiki/Containers_as_a_Service
;)
GitLab CI Lead
I currently am using a VM with a docker runner, and there are two (somewhat obvious) issues:
* It requires root access to the machine to be able to spin up containers
* Every VM with a docker runner needs enough storage to handle every docker image used for builds, instead of centralizing the images. I really need to put in a cron job or something to clean these up.
Kubernetes should fix both of these, without introducing another technology in our environment like Docker Machine.
At this point it doesn't make sense to introduce support for another clustering manager. The GitLab Runner does support docker-machine (you can pretty easily configure it to create machines on-demand on any cloud provider). These machines will be pretty much self-managed. Soon the Runner will be also compatible with Docker Swarm, the implementation is already on separate branch.
From my perspective Kubernetes is great as platform for deploying applications, but I'm not so sure if it comes for distributing builds.
The problem with cleaning old images and caches is also addressed. You may be interested in checking out this application: https://gitlab.com/gitlab-org/gitlab-runner-docker-cleanup
What is important that GitLab Runner is being constantly improved and at some point with more convincing (for example coming from You) we can get back to native Kubernetes executor :)
GitLab CI Lead
https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_r... introduces many dependencies. Not sure how to solve it. Maybe something like https://github.com/deric/gitlab-ci-mesos ?
The limitations with most all container platforms is what they don't do major things which need to be addressed somehow in production systems:
- data backups / restores
- security practices / policies (good luck running SELinux)
- migrations to new versions
- security updates
- firewall rules
- monitoring & performance agents
- hard resource isolation (cpu, ram, network, IOPS)
- [something else obviously important here]
Containers are great for limited testing and local development, but for production they are like anti-type 1 hypervisors... more trouble that they're worth (docker instances still get stuck all the time as zombie processes, requiring a hard host reboot).
Worse, most of these containers are built by unknown persons, are unsigned and hosted publicly, so it's effectively running untrustworthy code however they decided to package and deliver it.
Finally, to add insult to injury, there's no reliable way to rebuild an image from scratch without some sources, whereas configuration management tools are intended to describe how to deploy apps and put systems into a desired configuration.
But most of your concerns are actually handled by what we are doing at Red Hat. OpenShift v3:
* applies SELinux policies to each running container
* can handle multiple styles of container upgrades (rolling, green/blue, canary, etc)
* includes an automated build process for automatically patching all of your containers when there is an update to their base image (including supported releases of RHEL containers that are kept up to date with errata)
* automatically reconfigures iptables as containers go up/dowm/migrate
* collects metrics on all running containers
* health checks for running containers to ensure they are not zombies
Since it is built on top of Kubernetes, the only configuration that you need to apply/maintain is the pod description, a short json document that describes what to run, what ports to expose, etc.
1. Getting ahead of your costs before they start to shoot up as you grow. Check out this pricing calculator for an idea of what I mean: http://containership.io/#pricing
2. Cloud portability: Heroku runs in 2 AWS regions (us-east-1, eu-west-1). What happens if you want to host in a specific country, a completely different provider (maybe you have some free credits?) or even your own data center eventually? Not gonna happen with heroku.
3. More flexibility to run the types of databases, queues, or other open source software that isn't possible to run on Heroku.
4. With CaaS you have root access to the servers powering your software, where Heroku is more of a black box. Remember what happened with (rap)Genius? http://techcrunch.com/2013/02/14/heroku-admits-to-performanc...