We use a similar setup for some of our services at edX. It is worth noting that you may need to adjust your development behaviors/environment somewhat to accommodate Docker.
For example, PyCharm supports using Docker Machine but not the new Docker for Mac. This means you won't necessarily have access to PyCharm's code inspection and remote debugging if you solely use Docker. This has resulted in my having a virtualenv on my host machine as well as the Docker setup, which is not ideal.
Once the tooling is better, things will be great; however, we aren't quite there yet.
yeah, the tools have some way to go, I did have issues with virtualenvs like you mentioned but given how fast the docker ecosystem has developed, I expect them to make it there soon enough.
I wanted to learn some things a while back (can't remember what, I think it was laravel and angular) and set about creating a dev environment for it. Whatever I started out to learn fell by the wayside and building a shit-hot, easy to use, modular, containerised dev environment became my soul focus. I'm getting ready to open it up to the world any day now[1].
I'll post about it once I've ironed out a few more quirks. It makes life so much less unpleasant in so many ways.
As a workaround we set up an alias for opening the file in VIM and saving it (without making any changes) that would run from within the container. Like the following:
I don't think you'll see these problems with the new Docker 1.12 for Mac (no docker toolbox), which brings the experience on the Mac much closer to what you see on Linux boxes.
If you're running something that doesn't do live code reloading, it won't work without restarting whether or not you're running it in a container. If you're running something that does do live code reloading, it should work just the same inside a container, assuming you have the volumes mounted.
The code does change inside the docker container when it's mounted as a volume, you just have to watch for it. We (at http://www.duecourse.com, a mostly node + Microservices shop) use nodemon for this
We currently just don't mount node_modules at all, and run npm install when we build the image, and have to re-build whenever we change package.json. It sounds bad, but once we've got something working we tend not to need to add new dependencies, it's normally just code changes from there on out.
As an example, here's a docker compose for one service:
ports:
- "20003:20003"
volumes:
- ./APP/src:/opt/app/src
- ./APP/test:/opt/app/test
command: npm run dev
`run dev` is the thing that calls nodemon, which reloads whenever src or test change. Otherwise, everything like node_modules exists inside the image.
I just want to add that your Docker "containers" are just processes on the host, and don't actually exist outside of the containerized process' lifetime. Your container is not a VM. So restarting a container to pick up code changes is just saying that you're restarting your process to pick up code changes. That's why you'll need to use something like pm2 if you want files watches and hot-loaded. The other comments are great, I just wanted to give some insight into why.
This is why I just run all of an app's dependencies in a Docker container and connect the app to the Docker containers. Then I don't have to brew install all this crap.
ah, overlapping use of the word 'service' here. I meant the internal server service/process (e.g: uwsgi) if it doesn't watch for changes.
I think restarting an entire container for such small-scale changes is a bit much. Personally, I often resort to using a development server on the active project most of the time, it's a dirty solution with the consequence being that I've negated some of the advantages of my setup - if your server software is fast at restarting, you could throw in some sort of watcher.
> webapp declares an env_file from which to pick up env vars. This is important because we configure a lot of our applications with environment variables so that the same docker image can run in different environments, taking inspiration from 12-factor apps.
Keep in mind that your environment variables can be learned by inspecting the image. So if you make that image public, those envs are visible. As far as i understand, this is analogous to committing your passwords to source control and should be a consideration.
Huh? I mean, sure, you can probably figure out the _names_ of the environment variables expected by the code of the image, but the image itself doesn't have the values for those variables.
Environment files are—unless I'm missing something—a run-time consideration. You provide the an env-file you have locally to connect to your development images, etc., and could deploy (possibly) the same image elsewhere, with a different env-file. But the env-file values are not baked into the image.
They _can_ be baked into the image via build-args or the "ENV" Dockerfile syntax, but as your Dockerfile is committed to git normally this has the same dramas, of course! You're completely correct, but I wanted to mention that it was possible :)
Build-args and ENVs are not the same thing. Build-args use the ARG syntax and are baked in during docker build. ENVs are not baked in; the values are passed in during docker run or docker exec.
You are correct, they are not the same thing; however they can be combined to bake in an ENV to an image as a default value for a given ENV key, controlled at "docker build".
I was just starting to play with this last night. Frustrating couple of hours because it turns out that IntelliJ's Docker plugin isn't yet compatible with Docker For Mac, and in a way that isn't JetBrains' fault. You have to do some kind of weird workaround with socat.
Semi-related, but I had issues with Docker Compose and database volumes for web-development work (where the database volume-only container to allow for persistence across containers would be recreated, thus losing the data that was there). This prompted a tiny wrapper around some patterns we were using, exposed via a CLI tool and JSON file:
Even more unrelated: it's written in Nim, and a big refactor is coming along that works with the Docker socket directly (either the AF_UNIX or TCP socket, depending on what's available)
If you're sharing your docker-compose.yml file with other developers, this will get annoying really quickly. Normally, I'd place the docker-compose.yml file in the root of my project, and then replace "/path/to/my/code/on/my/dev/machine" with ".". That way, it doesn't matter where other devs keep code on their machines.
In the case where you have a compose file that builds multiple projects (See: double time!), keep the compose file in one git repo, and then include the code for the other two projects (webappone and webapptwo) as submodules.
27 comments
[ 3.3 ms ] story [ 66.2 ms ] threadFor example, PyCharm supports using Docker Machine but not the new Docker for Mac. This means you won't necessarily have access to PyCharm's code inspection and remote debugging if you solely use Docker. This has resulted in my having a virtualenv on my host machine as well as the Docker setup, which is not ideal.
Once the tooling is better, things will be great; however, we aren't quite there yet.
My dev env is deteriorating after 2 years use and I'll set up a new one this weekend.
Probably going to use NixOS, but also looking into Docker.
I'll post about it once I've ironed out a few more quirks. It makes life so much less unpleasant in so many ways.
[1] yeah we'll see how that goes
One thing to note is that volumes can act weird at times if using docker-toolbox (I know the Blog Post was about using a Linux host).
https://www.virtualbox.org/ticket/14920
As a workaround we set up an alias for opening the file in VIM and saving it (without making any changes) that would run from within the container. Like the following:
This is the part I find unacceptable for a development environment. Is there a way to have live-reloading of docker instance somehow?
This has been one of the great challenges for me. Nothing seems to work well with volume mapping.
We currently just don't mount node_modules at all, and run npm install when we build the image, and have to re-build whenever we change package.json. It sounds bad, but once we've got something working we tend not to need to add new dependencies, it's normally just code changes from there on out.
As an example, here's a docker compose for one service:
`run dev` is the thing that calls nodemon, which reloads whenever src or test change. Otherwise, everything like node_modules exists inside the image.`rerun --background -- "foreman start"`
It is super simple and works well for us
I think restarting an entire container for such small-scale changes is a bit much. Personally, I often resort to using a development server on the active project most of the time, it's a dirty solution with the consequence being that I've negated some of the advantages of my setup - if your server software is fast at restarting, you could throw in some sort of watcher.
Keep in mind that your environment variables can be learned by inspecting the image. So if you make that image public, those envs are visible. As far as i understand, this is analogous to committing your passwords to source control and should be a consideration.
Environment files are—unless I'm missing something—a run-time consideration. You provide the an env-file you have locally to connect to your development images, etc., and could deploy (possibly) the same image elsewhere, with a different env-file. But the env-file values are not baked into the image.
https://github.com/girvo/dup
Even more unrelated: it's written in Nim, and a big refactor is coming along that works with the Docker socket directly (either the AF_UNIX or TCP socket, depending on what's available)
In the case where you have a compose file that builds multiple projects (See: double time!), keep the compose file in one git repo, and then include the code for the other two projects (webappone and webapptwo) as submodules.