One thing I'd like to point out are OS upgrades, security patches or generally package updates. With docker I just rebuild a new image using the latest ubuntu image (they are updated very frequently), deploy the app, test and then push the new image to production. Upgrading the host OS also is much less of a problem because far fewer packages are installed (i.e. it's just docker and the base install).
This still doesn't say what you are doing. You update the base image, which is presumably something every Docker user does, then you "deploy the app".
What are you deploying? How much heavy lifting is your dockerfile doing? How much of the environment do you have to setup manually? How do you supply the app its static and dynamic data? How do you make the app accessible to users? How are you handling availability if your app crashes? Is the app distributed or load balanced in any way?
We have many images, and we build them in a CI setup using jenkins. I used to run jenkins inside docker and build images within that docker, but this turned out to be a problem. (Mainly more and more resources were used up until the disk was full.) Now it's just jenkins installed on the host building images, starting them and run integration tests.
I have learnt enough about Docker to know it's not something which solves any problem I have, but finding out concrete facts about what others are actually doing with it was one of the hardest parts of the learning process.
The official Use Cases page is so heavily laden with meaningless buzzwords and so thin on actual detail that I still feel dirty just from reading it. https://docker.com/resources/usecases/
Some of the answers are interesting. I agree with you that most of the companies they mention on their page are so big that it makes you think that Docker is a thing for big companies, and not relevant to what I do.
For my part, I use Docker for my personal home server to keep dependencies documented and keep everything cleanly isolated.
Currently I have 8 images running, and more sitting around that I spin up as needed, running everything from a minecraft server, to AROS (an AmigaOS re-implementation) , haproxy, a dev-instance of my blog, a personal wiki, my dev environment (that I use to run screen and ssh into for when I want to edit stuff - it bind mounds a shared home directory, but means I don't mess up to host server with all kinds of packages I decide to install).
It's gotten to the point where the moment I find myself typing "apt-get install" I pause to consider whether I should just put it in a Dockerfile instead. After all, if this is a package I'll need once, for a single program, the Dockerfile does not take much longer to write, and it saves on the clutter.
I have a draft blog post about how I'm using it I keep meaning to post - probably after the weekend.
At Lime Technology, we have integrated docker into our NAS offering along with virtual machines (KVM and Xen). Docker provides a way to eliminate the "installation" part of software and skip straight to running proven and tested images in any Docker environment. With Containers, our users can choose from a library of over 14,0000 Linux-based apps with ease. Docker just makes life easier.
I used Docker to solve a somewhat unconventional problem for a client last week. They have a Rails application that needs to be deployed in two vastly different situations:
* a Windows server, disconnected from the internet
* about 10 laptops, intermittently connected to the internet
Docker let us build the application once and deploy it in both scenarios with much less pain than the current situation, which basically consists of script to git-pull and over-the-phone instructions when dependencies like ruby or imagemagick need to be upgraded.
We run VirtualBox with a stock Ubuntu 14.04 image with docker installed from the docker-hosted deb repo. We use the Phusion passenger Ruby image[1], which bundles almost every dependency we needed along with a useful init system so we can run things like cron inside a single container along with the application. This makes container management trivial to do with simple scripts launched by non-technical end users.
The laptops are Macbook airs and were not running a VM at all. Instead, users had a script they could double click to launch the application in a terminal window.
Now the laptops run the VirtualBox setup and always have the application running in the background. Docker adds value by letting us distribute a much smaller amount of data vs sending out an entire VM image.
For the Windows server, we used to distribute upgrades by sending out an entire VirtualBox appliance image, which was usually around 3GB. Additionally, the operator would have to manually shuffle data between the old and new images. Now, we can ship out an saved Docker image (built with `docker save`) which cuts down on the amount of data transferred, and the final VM we shipped him knows how to upgrade the Docker container and shuffle the data automatically.
- Are you/have you tried using "FROM" and layering things in multiple images to reduce what you need to keep shipping?
- Anything stopping you from using a private registry? I'm running one at it seems to work quite well (with the caveat that it's annoying to have to specify the registry all the time).
- You talk about "shuffling the data". Does that mean you're not using volumes to keep the data separate from the container? If so, any particular reason?
1) We've talked about it but it's not a blocker so we haven't done it yet. Right now we're trying to reduce the number of layers we ship, since they seem to get big for no good reason.
2) We're using a private Docker Hub account to transfer images to the laptops. They have a script that the users can invoke that shuts down the container, updates, runs some initialization tasks (`db:migrate` + some other stuff) and then brings the container back up.
3) Yep, we're using mounted shared directories in both cases. Previously the laptops of course just stored everything on the local filesystem since they were running the app directly. I'm not 100% sure what the Windows server was doing, but I believe the operator had to move data from one share to another and run initialization tasks by hand.
It simplifies the building and deployment process for the application. You can install the new version of the application in a Docker Container and just push the changes of Container to all your clients. You can do the same thing for changes in the infrastructure which is needed to run your application. Just deploy the changes.
Docker also enforces immutability. With a VM there's always the temptation to manually fix any issue that arises, and if you don't have some bulletproof way to document that then you'll have issues when you go to recreate the environment on a new machine. Docker kind of forces you to solve the original problem via the dockerfile, which is what will spawn images for any future installs anyway.
Could you explain this more? I think my confusions stems from where the config comes from. Regardless of whether I have a bit-for-bit image or a vm created from a bunch of script commands, the immutability disappears when I apply the config.
So for my example If I have a role that specifies one instance of a a galera server. I have to config each one with the other servers in the pool. And each config will be dependent on the other server's config. So is Docker the first part (get the galera server instance running) and then there is some 2nd part that does the config so the instances in the cluster work together?
To your first question: for me it's a on-paper vs reality difference. On paper you're exactly right re "vm created from a bunch of script commands" will end in the same state.
The reality is that once the VM is built there is the temptation/opportunity to make ad-hoc changes for any variety of reasons. Those ad-hoc changes sometimes make it back into the official build process, but sometimes they get forgotten in the heat of the moment. With docker you can't do this...to make the necessary change you are also changing the official build process. No opportunity for the two to deviate.
Second question: Yes, that is my understanding (though not a use case I have atm).
Very good call out. For most use cases this actually turns out to be OK, but to reduce the surface area of this being a potential issue you could:
- Vendor dependencies (works to replace stuff like `go get` but probably not for apt packages etc.)
- Create a base image which handles the stuff you need to reach out to the network for (`apt-get install openjdk-6-jre` etc.) and is infrequently updated. Then the Dockerfile for the final application is `FROM me/myjava` and just does a few things that don't use the network like `ADD . /code`.
- Use `docker commit` instead of Dockerfiles for those steps (pretty gross IMO)
- Use CM in your docker build to install a very specific version of a package if you need (I'm not 100% sure this exists but it seems probable). This isn't perfect but tightens things up if you're worried about upstream breaking apt packages etc.
One of the goals of a new image format for Docker (this is 2.0 stuff) is to make the layers content-addressable by ID. That way, you will have a reasonable assurance that two Docker images constructed with the same Dockerfile in two different places will have the same IDs if they result in the exact same layers, and you will be able to see the point of divergence otherwise.
"- Create a base image which handles the stuff you need to reach out to the network for (`apt-get install openjdk-6-jre` etc.) and is infrequently updated. Then the Dockerfile for the final application is `FROM me/myjava` and just does a few things that don't use the network like `ADD . /code`.
- Use `docker commit` instead of Dockerfiles for those steps (pretty gross IMO)
- Use CM in your docker build to install a very specific version of a package if you need (I'm not 100% sure this exists but it seems probable). This isn't perfect but tightens things up if you're worried about upstream breaking apt packages etc."
These are some of the goals of ShutIt.
We had complex development needs due to technical debt, and dockerfiles simply didn't cut it, and I got frustrated with the indirection of chef/puppet/ansible. I also needed the several hundred devs in my company to get productive quickly, so transferring all the little bash scripts and storing it in docker was the path of least resistance.
It's out of date and heavily edited, but I talk about this here:
What value does adding a user add if you're already running a VM? By default, all that user adds is directory-access-controls. Docker provides isolation between processes on the system it is running. Executing a root exploit from inside a Docker container is not impossible, but it's also harder than "simply" being a user. Application-level security can also be improved significantly if an application requires multiple processes to run on a host. Docker can be used to restrict processes from accessing the network, etc. Nothing that couldn't be done without Docker by a sufficiently dedicated ops team, I'll admit, but Docker greatly simplifies and standardizes these mechanisms. That's especially true if you've adopted DevOps culture where developers have come to own more of the systems security.
I've used to to make an Ubuntu packaged app available on CentOS machines. The compile-from-source was a bit of a headache (lots of dependencies which also had to be compiled from source) so being able to deploy like this saved a lot of hassle.
I've used docker for process isolation at two companies now. In both cases, we were executing things on the server based on customer input values, and desired the isolation to help ensure safety.
In the first company, these were one-off import jobs that would import customer information from a URL they provided.
In the other, these are long-running daemons for a multi-tenant service, and I need to reduce the risk that one customer could exploit the system and disrupt the other customers or gain access to their data.
I have some other experiments in play right now in which I am packaging up various services as docker containers, but this is currently non-production.
At Shippable(shippable.com) we've been using docker for over an year now for the following use cases:
1. deploying all our internal components like db, message queue, middleware and frontend using containers and using a custom service discovery manager. The containerization has helped us easily deploy components separately, quickly set up dev environments, test production bugs more realistically and obviously, scale up very quickly.
2. running all the builds in custom user containers. This helps us ensure security and data isolation.
We did run into a bunch of issues till docker was "production-ready" but the use case was strong enough for us to go ahead with it
Deployment. We have a legacy application that would take about a day of configuration to deploy properly. With Docker (and some microservices goodness) we've reduced the deploy down to an hour, and are continually improving it.
We've been using Docker for YippieMove (www.yippiemove.com) for a few months now, and it works great.
Getting your hand around the Docker philosophy is the biggest hurdle IMHO, but once you're there it is a delight to work with. The tl;dr is to not think of Docker as VMs, but rather fancy `chroots`.
In any case, to answer your question, for us it significantly decreased deployment time and complexity. We used to run our VMs and provision them with Puppet (it's a Django/Python app), however it took a fair amount of time to provision a new box. More so, there were frequently issues with dependencies (such as `pip install` failing).
With Docker, we can more or less just issue a `docker pull my/image` and be up and running (plus some basic provisioning of course that we use Ansible for).
how do you do restarts when you update the app ? I assume you have to take the app server out of the server pool (remove from load balancer or nginx) and shut it down, then docker pull your image.
I'm doing deploys with ansible and its just too slow
Actually, we have Nginx configured with Health Check (http://nginx.org/en/docs/http/load_balancing.html#nginx_load...). Hence, it will automatically take a given appserver out of the pool when it stops responding. Once the node is back again, Nginx will automatically bring it back into rotation.
Also, we actually use a volume/bind-mount to store the source code on the host machine (mounted read-only). That way we can roll out changes with `rsync` and just restart the container if needed.
The only time we need to pull a new update is if the dependencies/configuration of the actual container change.
How do you deal with connections that are in progress to the app server? If you just take it down, you're potentially throwing away active connections.
Yes, that's absolutely true and something we're aware of. It would of course be possible to solve, but would increase the complexity by a fair amount.
It is also worth mentioning that it is a more back-end heavy service, than front-end heavy. Since each email migration runs isolated in its own Docker container, a given customer can generate 100s of Docker containers.
Hence, given the relatively low volume of users on the web app, and the fast restart time, the chance of throwing away an active connection is relatively low.
ok thanks. I have several app servers and I take them out of nginx server list, stop it gracefully, git pull and configure (slow, I want to get rid of this step), put it back in nginx servers, move onto the next one.
tedious, although my whole deploy-to-all-servers is a single command.
Yeah, that sounds pretty tedious, but I guess it could still be automated (but somewhat tricky).
Once CoreOS becomes more stable, we're looking to move to it. The idea is then to use `etcd` to feed the load balancer (probably Nginx) with the appserver pool. That way you can easily add new servers and decommission old ones.
We automated this pretty trivially at my last job using Fabric[0]. All we had to do was cycle through a list of servers and apply the remove from LB, update, add to LB steps. Removing from the LB should simply block until connections drain (or some reasonable timeout). It makes deploys take longer for sure, but avoiding the inevitable killing of user connections was worth it.
I will add that if you're using Docker (which we weren't) it might be easier to deploy a new set of Docker containers with updated code and just throw away the old ones.
We use the Google Compute Engine container optimised VMs, which make deployment a breeze. Most of our docker containers are static, apart from our application containers (Node.js) that are automatically built from github commits. Declaring the processes that should run on a node via a manifest makes things really easy; servers hold no state, so they can be replaced fresh with every new deployment and it's impossible to end up with manual configuration, which means that there is never a risk of losing some critical server and not being able to replicate the environment.
At Shopify, we have moved to Docker for deploying our main product. Primary advantages for us are faster deploys, because we can do part of the old deploy process as part of the container build. Secondly: easier scalability, because we can add additional containers to have more app servers or job workers. More info at http://www.shopify.com/technology/15563928-building-an-inter...
wvanbergen, forgive me for veering off topic. I'm planning on applying to Shopify (Toronto) as a software developer before the end of the weekend. Any advice you're willing to share?
MattyMc: sure. Primarily: be yourself, show what you are passionate about, and be willing to adopt change. When applying: cover letter > resumé, and try to stand out because we get many applications. Email me at willem at shopify dot com if you have any specific questions.
One of the main things I'm using it for are reproducible development environments for a rather complex project comprising nearly ten web services.
We have a script that builds a few different docker images that the devs can then pull down and get using straight away. This is also done through a dev repo that they clone that provides scripts to perform dev tasks across all services (set up databases, run test servers, pull code, run pip etc.).
It used to take a day to set up a new dev enviroment, now it takes around 30 mins and can be done with almost no input from the user and boils down to: install docker, fetch databases restores, clone the dev repo, run the dev wrapper script
This is approximately what I'm using it for too. I'm working on my MSc, and I'm using it to make reproducible experimental environments. Packages locked to specific versions, all required libraries installed, isolated from the rest of the system. Working pretty well in that capacity!
At UltimateFanLive we use docker on Elastic Beanstalk to speed up the scaling process. Our load goes from 0 to 60 in minutes, as we are connected with live sports data. Packages like numpy and lxml take way too long to install with yum and pip alone. So we pre-build images with the dependencies but we are still using the rest of the goodies on Elastic Beanstalk. Deploy times have plummeted and we keep t2 cpu credits.
We use Docker to set up our testing environments with Jenkins and install the application in it. Every build will be installed in a Docker Container automatically. The Container is used for acceptance tests. The Docker Containers are set up automatically with a Dockerfile. Its an awesome tool for automatating and deployment and used to implement the concepts of "Continuous Delivery".
We're using Docker for a few internal projects at Stack Exchange. I've found it to be simple and easy, and it just works.
We have a diverse development team but a relatively limited production stack - many of our devs are on Macs (I'm on Ubuntu), but our servers are all Windows. Docker makes it painless to develop and test locally in exactly the same environment as production in spite of this platform discrepancy. It makes it a breeze to deploy a Node.js app to a Windows server without ever actually dealing with the pain of Node.js on Windows.
Also, it makes the build process more transparent. Our build server is Team City, which keeps various parts of the configuration in many different hidden corners of a web interface. By checking our Dockerfile into version control much of this configuration can be managed by devs well ahead of deployment, and it's all right there in the same place as the application code.
Since we are still waiting for CoreOS + (flynn.io || deis.io) to mature. I modified our existing VMWare VM based approached to setup ubuntu boxes with Docker install. Where I then use fig to manage an application cluster, and supervisor to watch fig.
When its time to update a box, jenkins sshs in calls docker pull to get the latest, then restarts via supervisor. Any one off docker run commands require us to ssh in, but fig provides all the env settings so that I don't have to worry about remembering them. The downtime between upgrades is normally a second or less.
The biggest thing I ran into is that each jenkins builds server can only build and test one container at a time. After each one, we delete all images. The issue is that if you have an image it wont check for a new image. This applies to all underlaying images. We cut the bandwidth by having our own docker registry that acts as our main image source and storage.
* Sanity in our environments. We know exactly what goes into each and every environment, which are specialized based on the one-app-per-container principle. No more asking "why does software X build/execute on machine A and not machines B-C?"
* Declarative deployments. Using Docker, Core OS, and fleet[1], this is the closest solution I've found to the dream of specifying what I want running across a cluster of machines, rather than procedurally specifying the steps to deploy something (e.g. Chef, Ansible, and the lot). There's been other attempts for declarative deployments (Pallet comes to mind), but I think Docker and Fleet provide even better composability. This is my favorite gain.
* Managing Cabal dependency hell. Most of our application development is in Haskell, and we've found we prefer specifying a Docker image than working with Cabal sandboxes. This is equally a gain on other programming platforms. You can replace virtualenv for Python and rvm for Ruby with Docker containers.
* Bridging a gap with less-technical coworkers. We work with some statisticians. Smart folks, but getting them to install and configure ODBC & FreeTDS properly was a nightmare. Training them in an hour on Docker and boot2docker has saved so much frustration. Not only are they able to run software that the devs provide, but they can contribute and be (mostly) guaranteed that it'll work on our side, too.
I was skeptical about Docker for a long time, but after working with it for the greater part of the year, I've been greatly satisfied. It's not a solution to everything—I'm careful to avoid hammer syndrome—but I think it's a huge step forwards for development and operations.
Addendum: Yes, some of these gains can be equally solved with VMs, but I can run through /dozens/ of iterations of building Docker images by the time you've spun up one VM.
Yes, very much so. In fact, our goal is to have NixOS based containers. Right now, we're using Debian as the base image, and there's /no/ guarantee that the versions of software installed are consistent (since Docker caches based on the line in a Dockerfile, rather than what's actually installed).
With Nix, we can have version guarantees in all of our Docker images--including the cached images.
That depends on how the package is specified doesn't it? You can snapshot the full chain of versions with dpkg and explicitly specify them all. It should be too hard to wrap this into something like Gemfile.lock
Great. I have a major complaint about Nix. Their packages are built with all sorts of dependencies included. So you install mutt and you end up getting python. Or you install git and you also get subversion.
I understand all their philosophy, but they should allow for flexible runtime dependencies without the need for rebuilding packages. Perhaps with a second hash or something, to sign dependencies.
If you take the steps for, say, AWS, of building a new AMI for every role you have, then it's pretty much the same.
(but in my experience with building AMI's, that process is way too slow compared to Docker)
Docker becomes closer to declarative when you build static Docker images for every role and rebuild for every change and re-deploy. Even more so when your deployment is based on a tool like Fleet that declaratively specifies your cluster layout.
The point is to avoid ever having situations where you say "install package foo on all webservers". Instead you say "replace all webservers with a bit-by-bit identical copy of image x".
The benefit is that you can have already tested a container that is 100% identical, and know that the deployed containers will be 100% identical, rather than hope the commands you pass to the config tool handles every failure scenario well enough.
For configuration, we separate the application and the config files into two separate containers. The config files are provided through a shared volume to the application. This model is definitely odd. However, it's allowed us to decouple our application and our configuration and to swap out configurations. With this in mind, it's more declarative because we specify "run this application with this configuration unit" rather than "here's how you get yourself started". See the Radial project for our inspiration[1]
We've found that this approach has generalized so far. For example, setting up a Cassandra cluster is often a real PITA to configure since you need the seed IPs up front. Our configuration container manages the dance by registering and pulling the IPs from Consul (etcd would work fine too). Perhaps a bit of smoke-and-mirrors, but it's achieved being able to spin up a properly-configured Cassandra cluster without needing to manually specify who's in the cluster.
Great explanation. If possible, I'd love to see/know more about how the statistician training step was accomplished. I also work with many nontechnical folks and haven't found success getting training on docker to 'stick'.
The Docker folks get a point for bootstrapping a familiar user interface: git. Our non-dev coworkers are competent enough with git, and they felt comfortable drawing analogies between the two. They pull the image (from our private Docker registry), run the container, make some changes, build, run, repeat. Very similar to pull, check out, commit, etc in git.
The only pain I've had is the silly flags for 'docker run'. Ugh. Before I told them to make aliases, there were all sorts of complains when they forgot '-it' and '--rm'. I think '-it' should be the default, and possibly '--rm' as well, with switches to toggle them off. Oh well.
Most people seem to be using Docker with distro-based images, ie. start with ubuntu and then add their own app on top etc. Is anyone using more application-oriented images, ie. start with empty image and add just your application and its dependencies?
I'm doing this, but starting with the minimal 80MB debian:stable image and then using apt-get to grab my dependencies. The 80MB covers the dependencies for apt-get.
I believe the centos images are also minimal, but the ubuntu image starts out much larger.
We are running our app[1] in instances of Google Compute Engine. We installed Docker in those instances.
Our app is a bunch of microservices, some Rails Apps each one running with Puma as webserver, HAProxy, some other Rack app (for Websockets). We also use RabbitMQ and Redis.
All the components are running in their own containers (we have dozens of containers running to support this app).
We choose this path because in case of failures, just 1 service would be down meanwhile the whole system is nearly fully functional. Re-launching a container is very straightforward and is done quickly.
156 comments
[ 4.9 ms ] story [ 221 ms ] threadWhat are you deploying? How much heavy lifting is your dockerfile doing? How much of the environment do you have to setup manually? How do you supply the app its static and dynamic data? How do you make the app accessible to users? How are you handling availability if your app crashes? Is the app distributed or load balanced in any way?
And out of curiosity: is your testing of new images automated?
I have learnt enough about Docker to know it's not something which solves any problem I have, but finding out concrete facts about what others are actually doing with it was one of the hardest parts of the learning process.
The official Use Cases page is so heavily laden with meaningless buzzwords and so thin on actual detail that I still feel dirty just from reading it. https://docker.com/resources/usecases/
https://news.ycombinator.com/item?id=8324138
Some of the answers are interesting. I agree with you that most of the companies they mention on their page are so big that it makes you think that Docker is a thing for big companies, and not relevant to what I do.
Currently I have 8 images running, and more sitting around that I spin up as needed, running everything from a minecraft server, to AROS (an AmigaOS re-implementation) , haproxy, a dev-instance of my blog, a personal wiki, my dev environment (that I use to run screen and ssh into for when I want to edit stuff - it bind mounds a shared home directory, but means I don't mess up to host server with all kinds of packages I decide to install).
It's gotten to the point where the moment I find myself typing "apt-get install" I pause to consider whether I should just put it in a Dockerfile instead. After all, if this is a package I'll need once, for a single program, the Dockerfile does not take much longer to write, and it saves on the clutter.
I have a draft blog post about how I'm using it I keep meaning to post - probably after the weekend.
* a Windows server, disconnected from the internet
* about 10 laptops, intermittently connected to the internet
Docker let us build the application once and deploy it in both scenarios with much less pain than the current situation, which basically consists of script to git-pull and over-the-phone instructions when dependencies like ruby or imagemagick need to be upgraded.
We run VirtualBox with a stock Ubuntu 14.04 image with docker installed from the docker-hosted deb repo. We use the Phusion passenger Ruby image[1], which bundles almost every dependency we needed along with a useful init system so we can run things like cron inside a single container along with the application. This makes container management trivial to do with simple scripts launched by non-technical end users.
[1]: https://github.com/phusion/passenger-docker
Now the laptops run the VirtualBox setup and always have the application running in the background. Docker adds value by letting us distribute a much smaller amount of data vs sending out an entire VM image.
For the Windows server, we used to distribute upgrades by sending out an entire VirtualBox appliance image, which was usually around 3GB. Additionally, the operator would have to manually shuffle data between the old and new images. Now, we can ship out an saved Docker image (built with `docker save`) which cuts down on the amount of data transferred, and the final VM we shipped him knows how to upgrade the Docker container and shuffle the data automatically.
- Are you/have you tried using "FROM" and layering things in multiple images to reduce what you need to keep shipping?
- Anything stopping you from using a private registry? I'm running one at it seems to work quite well (with the caveat that it's annoying to have to specify the registry all the time).
- You talk about "shuffling the data". Does that mean you're not using volumes to keep the data separate from the container? If so, any particular reason?
2) We're using a private Docker Hub account to transfer images to the laptops. They have a script that the users can invoke that shuts down the container, updates, runs some initialization tasks (`db:migrate` + some other stuff) and then brings the container back up.
3) Yep, we're using mounted shared directories in both cases. Previously the laptops of course just stored everything on the local filesystem since they were running the app directly. I'm not 100% sure what the Windows server was doing, but I believe the operator had to move data from one share to another and run initialization tasks by hand.
So for my example If I have a role that specifies one instance of a a galera server. I have to config each one with the other servers in the pool. And each config will be dependent on the other server's config. So is Docker the first part (get the galera server instance running) and then there is some 2nd part that does the config so the instances in the cluster work together?
The reality is that once the VM is built there is the temptation/opportunity to make ad-hoc changes for any variety of reasons. Those ad-hoc changes sometimes make it back into the official build process, but sometimes they get forgotten in the heat of the moment. With docker you can't do this...to make the necessary change you are also changing the official build process. No opportunity for the two to deviate.
Second question: Yes, that is my understanding (though not a use case I have atm).
http://zwischenzugs.wordpress.com/2014/07/16/phoenix-deploym...
- Vendor dependencies (works to replace stuff like `go get` but probably not for apt packages etc.)
- Create a base image which handles the stuff you need to reach out to the network for (`apt-get install openjdk-6-jre` etc.) and is infrequently updated. Then the Dockerfile for the final application is `FROM me/myjava` and just does a few things that don't use the network like `ADD . /code`.
- Use `docker commit` instead of Dockerfiles for those steps (pretty gross IMO)
- Use CM in your docker build to install a very specific version of a package if you need (I'm not 100% sure this exists but it seems probable). This isn't perfect but tightens things up if you're worried about upstream breaking apt packages etc.
One of the goals of a new image format for Docker (this is 2.0 stuff) is to make the layers content-addressable by ID. That way, you will have a reasonable assurance that two Docker images constructed with the same Dockerfile in two different places will have the same IDs if they result in the exact same layers, and you will be able to see the point of divergence otherwise.
- Use `docker commit` instead of Dockerfiles for those steps (pretty gross IMO)
- Use CM in your docker build to install a very specific version of a package if you need (I'm not 100% sure this exists but it seems probable). This isn't perfect but tightens things up if you're worried about upstream breaking apt packages etc."
These are some of the goals of ShutIt.
We had complex development needs due to technical debt, and dockerfiles simply didn't cut it, and I got frustrated with the indirection of chef/puppet/ansible. I also needed the several hundred devs in my company to get productive quickly, so transferring all the little bash scripts and storing it in docker was the path of least resistance.
It's out of date and heavily edited, but I talk about this here:
http://www.youtube.com/ianmiell
and here:
http://ianmiell.github.io/shutit/
and on my blog:
http://zwischenzugs.wordpress.com/
In the first company, these were one-off import jobs that would import customer information from a URL they provided.
In the other, these are long-running daemons for a multi-tenant service, and I need to reduce the risk that one customer could exploit the system and disrupt the other customers or gain access to their data.
I have some other experiments in play right now in which I am packaging up various services as docker containers, but this is currently non-production.
https://groups.google.com/forum/#!topic/trezor-dev/5MCyweTY4...
1. deploying all our internal components like db, message queue, middleware and frontend using containers and using a custom service discovery manager. The containerization has helped us easily deploy components separately, quickly set up dev environments, test production bugs more realistically and obviously, scale up very quickly.
2. running all the builds in custom user containers. This helps us ensure security and data isolation.
We did run into a bunch of issues till docker was "production-ready" but the use case was strong enough for us to go ahead with it
Getting your hand around the Docker philosophy is the biggest hurdle IMHO, but once you're there it is a delight to work with. The tl;dr is to not think of Docker as VMs, but rather fancy `chroots`.
In any case, to answer your question, for us it significantly decreased deployment time and complexity. We used to run our VMs and provision them with Puppet (it's a Django/Python app), however it took a fair amount of time to provision a new box. More so, there were frequently issues with dependencies (such as `pip install` failing).
With Docker, we can more or less just issue a `docker pull my/image` and be up and running (plus some basic provisioning of course that we use Ansible for).
I'm doing deploys with ansible and its just too slow
Also, we actually use a volume/bind-mount to store the source code on the host machine (mounted read-only). That way we can roll out changes with `rsync` and just restart the container if needed.
The only time we need to pull a new update is if the dependencies/configuration of the actual container change.
It is also worth mentioning that it is a more back-end heavy service, than front-end heavy. Since each email migration runs isolated in its own Docker container, a given customer can generate 100s of Docker containers.
Hence, given the relatively low volume of users on the web app, and the fast restart time, the chance of throwing away an active connection is relatively low.
tedious, although my whole deploy-to-all-servers is a single command.
Once CoreOS becomes more stable, we're looking to move to it. The idea is then to use `etcd` to feed the load balancer (probably Nginx) with the appserver pool. That way you can easily add new servers and decommission old ones.
[0] http://www.fabfile.org/
its slow, but I go have a cup of tea while it works.
We use the Google Compute Engine container optimised VMs, which make deployment a breeze. Most of our docker containers are static, apart from our application containers (Node.js) that are automatically built from github commits. Declaring the processes that should run on a node via a manifest makes things really easy; servers hold no state, so they can be replaced fresh with every new deployment and it's impossible to end up with manual configuration, which means that there is never a risk of losing some critical server and not being able to replicate the environment.
We have a script that builds a few different docker images that the devs can then pull down and get using straight away. This is also done through a dev repo that they clone that provides scripts to perform dev tasks across all services (set up databases, run test servers, pull code, run pip etc.).
It used to take a day to set up a new dev enviroment, now it takes around 30 mins and can be done with almost no input from the user and boils down to: install docker, fetch databases restores, clone the dev repo, run the dev wrapper script
We have a diverse development team but a relatively limited production stack - many of our devs are on Macs (I'm on Ubuntu), but our servers are all Windows. Docker makes it painless to develop and test locally in exactly the same environment as production in spite of this platform discrepancy. It makes it a breeze to deploy a Node.js app to a Windows server without ever actually dealing with the pain of Node.js on Windows.
Also, it makes the build process more transparent. Our build server is Team City, which keeps various parts of the configuration in many different hidden corners of a web interface. By checking our Dockerfile into version control much of this configuration can be managed by devs well ahead of deployment, and it's all right there in the same place as the application code.
When its time to update a box, jenkins sshs in calls docker pull to get the latest, then restarts via supervisor. Any one off docker run commands require us to ssh in, but fig provides all the env settings so that I don't have to worry about remembering them. The downtime between upgrades is normally a second or less.
The biggest thing I ran into is that each jenkins builds server can only build and test one container at a time. After each one, we delete all images. The issue is that if you have an image it wont check for a new image. This applies to all underlaying images. We cut the bandwidth by having our own docker registry that acts as our main image source and storage.
* Sanity in our environments. We know exactly what goes into each and every environment, which are specialized based on the one-app-per-container principle. No more asking "why does software X build/execute on machine A and not machines B-C?"
* Declarative deployments. Using Docker, Core OS, and fleet[1], this is the closest solution I've found to the dream of specifying what I want running across a cluster of machines, rather than procedurally specifying the steps to deploy something (e.g. Chef, Ansible, and the lot). There's been other attempts for declarative deployments (Pallet comes to mind), but I think Docker and Fleet provide even better composability. This is my favorite gain.
* Managing Cabal dependency hell. Most of our application development is in Haskell, and we've found we prefer specifying a Docker image than working with Cabal sandboxes. This is equally a gain on other programming platforms. You can replace virtualenv for Python and rvm for Ruby with Docker containers.
* Bridging a gap with less-technical coworkers. We work with some statisticians. Smart folks, but getting them to install and configure ODBC & FreeTDS properly was a nightmare. Training them in an hour on Docker and boot2docker has saved so much frustration. Not only are they able to run software that the devs provide, but they can contribute and be (mostly) guaranteed that it'll work on our side, too.
I was skeptical about Docker for a long time, but after working with it for the greater part of the year, I've been greatly satisfied. It's not a solution to everything—I'm careful to avoid hammer syndrome—but I think it's a huge step forwards for development and operations.
[1]: https://coreos.com/using-coreos/clustering/
Addendum: Yes, some of these gains can be equally solved with VMs, but I can run through /dozens/ of iterations of building Docker images by the time you've spun up one VM.
With Nix, we can have version guarantees in all of our Docker images--including the cached images.
http://gregoryszorc.com/blog/2014/10/13/deterministic-and-mi...
`RUN apt-get install some_package=<version>` If you want a newer version, update the Dockerfile with the version you want.
I understand all their philosophy, but they should allow for flexible runtime dependencies without the need for rebuilding packages. Perhaps with a second hash or something, to sign dependencies.
In a Dockerfile you specify a base image and commands, may often invoke a config management tool.
How is using Docker more declarative?
(but in my experience with building AMI's, that process is way too slow compared to Docker)
Docker becomes closer to declarative when you build static Docker images for every role and rebuild for every change and re-deploy. Even more so when your deployment is based on a tool like Fleet that declaratively specifies your cluster layout.
The point is to avoid ever having situations where you say "install package foo on all webservers". Instead you say "replace all webservers with a bit-by-bit identical copy of image x".
The benefit is that you can have already tested a container that is 100% identical, and know that the deployed containers will be 100% identical, rather than hope the commands you pass to the config tool handles every failure scenario well enough.
http://zwischenzugs.wordpress.com/2014/08/09/using-shutit-an...
http://zwischenzugs.wordpress.com/2014/09/15/using-shutit-an...
We've found that this approach has generalized so far. For example, setting up a Cassandra cluster is often a real PITA to configure since you need the seed IPs up front. Our configuration container manages the dance by registering and pulling the IPs from Consul (etcd would work fine too). Perhaps a bit of smoke-and-mirrors, but it's achieved being able to spin up a properly-configured Cassandra cluster without needing to manually specify who's in the cluster.
[1]: http://radial.viewdocs.io/docs
The only pain I've had is the silly flags for 'docker run'. Ugh. Before I told them to make aliases, there were all sorts of complains when they forgot '-it' and '--rm'. I think '-it' should be the default, and possibly '--rm' as well, with switches to toggle them off. Oh well.
As we deploy on low power devices the minimal overhead of docker is crucial for us.
I believe the centos images are also minimal, but the ubuntu image starts out much larger.
Our app is a bunch of microservices, some Rails Apps each one running with Puma as webserver, HAProxy, some other Rack app (for Websockets). We also use RabbitMQ and Redis.
All the components are running in their own containers (we have dozens of containers running to support this app).
We choose this path because in case of failures, just 1 service would be down meanwhile the whole system is nearly fully functional. Re-launching a container is very straightforward and is done quickly.
[1]: https://dockerize.it