HN Explorer

Ask HN: Alternatives to Vagrant for development environments?

135 points by purephase ↗ HN
I've been using Vagrant for about 5-6 years now to automate the setup of development environments. It was a great tool for spinning up environments that were similar to production and easily shared across team resources that jump in and out of projects.

The number of issues we're dealing with daily with Vagrant is rising. The productivity boost it provided is entirely gone and the support is taking a substantial amount of time.

We love Hashicorp and, I don't mean this to be a slight against them, but I'm curious what alternatives people are using to:

- Automate building dev environments (Linux-based) with dependencies etc. - Ensure that your development environment is similar enough to production to reduce/eliminate potential issues moving to production.

I have done some googling around, but it still appears that Vagrant is the top dog in this space. Willing to accept that it's us and not the tool, but issues with NFS sharing, port collisions, support for Fusion, deprecations with legacy Chef setups and more is just more than we can handle right now.

Bonus points for tools that can be re-purposed for automating deployment as well (Terraform). Parity between development and other environments is important.

106 comments

[ 2.8 ms ] story [ 138 ms ] thread
phamilton
docker compose works well for us. We pull down all our prod docker images and can get a local cluster running with one command.
scaryclam
Can you elaborate on any of these issues? I've been using vagrant for a similar amount of time and don't find that there are really that many problems with it, nevermind to find a daily increase with them. There are certainly alternatives, but if you're having issues with vagrant to the extent that that your question implies, then you're likely to have have issues with those as well.
sebringj
If you mount to a local folder using docker-compose up, its so simple and quick for programming. Vagrant seems heavy compared to that and I develop on vagrant frequently. I have a mac and have the latest docker for mac installed. Very easy.
webo
There are other tools your team need as well: git, nvm, rvm, awscli, terraform, kubectl, etc. Vagrant helps with pre-installing all of that, whereas docker is mostly for running specific containers.
Terr_
Well, why not put those in docker too? The container you develop locally with doesn't have to be the same one that eventually ends up in production.

Here's a little demo designed for editing+testing a PHP library even if your local machine doesn't have anything more than Bash, Docker, and a text editor.

https://github.com/DHager/docker-php-library-demo

sawantuday
Put those tools in docker containers and replace original command with alias for docker run 'command'.
sebringj
I'm not in devops so I probably shouldn't give advice. I'm just thinking of the time I had to help with a Wordpress plugin and it was pretty easy using Docker. The company I contract with uses Docker in production and they are a very well known host so I would imagine you would have all the tools you would need but I don't know specifics.
hullsean
As others have said docker. Faster, more lightweight. ‍️
nunez
and also a terrible option for non 12 factor apps (i.e. anything with hard dependencies on the underlying system or other apps, which many do)
lenlorijn
Why would it be a terrible option for those cases?
sethammons
In our case, we had difficulty with docker due to logging to syslog in addition to stdout and stderr. We have shared volumes and persistent data on disk, and we have test cases where we kill child or parent processes, but if the parent process was pid 1, the container would die. We've addressed all of that in different ways and develop in docker with docker compose. The biggest pain now is that sometimes containers stop being able to reach one another, requiring a restart of docker for mac, and, for some reason, each time we up a container, it requires internet and VPN access to check for the latest images and will just hang without it, making offline development practically impossible for code that needs to run in docker. That limits what I can work on when on the train.
[deleted]
(comment deleted)
[deleted]
(comment deleted)
jldugger
> deprecations with legacy Chef setups

test kitchen supports a lot of various platforms, including vagrant, AWS, and openstack.

krakensden
Test kitchen isn't really a substitute, it doesn't do the plumbing (ports, random folders) you want for a dev environment.
akulbe
Doesn't test kitchen just call vagrant by default?
jonaf
Nix has a tool called nix-ops that will provision vms on virtual box (you can use containers powered by libvirt as well if you want). It's cool because you can use functional programming concepts and it takes a lot of the support headache away. The option to provision vms is really nice in some cases, which Docker just doesn't give you. But you can also use containers, just like docker, with reproducible builds, which means you can effectively test deployments without ever spending a dime on real hardware.
labdsf
Looks like you may want to try Vagga [1]. It is Linux-only, but you said that you want Linux-based environments. It is for development environments by design, so it is way easier to setup than Vagrant. Even if it does not solve your problems (which you have not specified), at least you will spend less time maintaining it.

[1] https://vagga.readthedocs.io/

berdario
Nix

https://jakob.gillich.me/post/2016-03-22-self-contained-deve...

https://ariya.io/2016/06/isolated-development-environment-us...

You can easily automate provisioning and deployment with Nixops:

http://container-solutions.com/step-towards-future-configura...

But if you need to share/deploy what you worked on, in some place where you might not have Nix available, you can just as easily create Docker containers out of your Nix expression:

http://lethalman.blogspot.co.uk/2016/04/cheap-docker-images-...

[dead] BuuQu9hu
Nix changes the game.
atmosx
HM, I use Vagrant daily to switch among Linux distros and run tests. Nix has a steep learning curve and although the community seems to be great, it is small and getting help might be problematic.

Enforcing NixOS to a team in my view will lead to a severe productivity downturn in the short-medium run.

Nix is much older than people think, I am not it will be around in 5-10 years though. I believe that Ansible, Puppet, Chef, etc are way simpler and more valuable market-wise if you are not into Haskell already...

bluejekyll
I don't know about that. The concepts of Nix answer significant deficiencies in the tools you mention. Those tools all assume that you need to build and operate systems, which seems like a lot of effort and overly complex, when the trend is to make all systems throwaway.

I've been following Nix for a while, and don't yet have an arena where I can play with it, but the concept is exactly what I think the industry needs. The biggest challenge that I see with nix is that it is basically a new OS and will require a while before many people are comfortable adopting it.

Also, it's not Haskell, it's got it's own minimal functional language, Nix, right?

thomastjeffery
Nix is fantastic. I'm running NixOS right now, and I can happily tell you that it is, by far, the best operating system I have ever used.

That being said, Nix sucks. Documentation is scattered, deprecated, and incomplete. I still don't understand some of the most basic things with NixOS, because I cannot find documentation.

Take nix-shell for example: You can make a shell.nix file in a directory to describe the environment you need, then simply nix-shell in that directory to get a working environment. It's great! I understand the concept, and use it regularly, but apart from some very tailored examples, I don't even know how to describe a simple environment with one package!

sumedh
I guess most people would just say Docker. Personally after using docker, I just cannot go back to Vagrant.
atombender
While I agree that Vagrant isn't 100%, the biggest source of our issues with Vagrant have actually been with VMware Fusion. VMware has lots of issues with managing its state. 8.x has a long-standing issue with port forwarding rules getting stuck, so some of us downgraded to 7.x just to get productive again.

The whole reason we migrated to VMware, about 3 years ago, was that we were using Virtualbox and being really bothered by the issues with it, things like corruption of files accessed through the shared folder mounts, that weren't being addressed. It's exasperating how brittle this VM stuff is.

Personally, I'd love a slimmer Linux-only solution based on Xhyve or Apple's Hypervisor.framework. I don't really want a Vagrantfile, I just want a low-level tool that I can drive from a wrapper, which we need anyway. Both Docker for Mac and Minikube now uses Xhyve, with great results.

We're slowly migrating our stack over to Kubernetes. Building with Docker for Mac is great, and Minikube seems really promising.

ledgerdev
As far as I know, docker for mac isn't actually usable for dev work if you use mounted volumes. Basically it pegs CPU and makes disk access 10-150 times slower than native which makes mounted volumes impossibly slow. A personal example would be building a jar on a macbook native took 11 seconds, takes docker for mac 9 minutes. See https://github.com/docker/for-mac/issues/77 https://github.com/docker/for-mac/issues/668 and a slew of other reports that can all be traced back mounted volume performance.

Moral of the story I moved to linux.

cygned
Using Docker for Clojure and JavaScript development here. I actually don't have these issues. There is a performance loss, sure, but not nearly as much as you describe it.
ledgerdev
I would say just check out the threads for the sort of experiences people are reporting across pretty much every stack from rails to symphony. In my case above it was actually jvm clojure building an uberjar, probably worst case scenario. REPL was usable because it didn't do a any mounted volume access, just network access.
cygned
I noticed that lein-ring with autoreload is magnitudes slower. We now turn it off for microservices not being worked on.

Anyway, I have read a lot about those problems and I hope it gets better at some point - especially for Docker on Windows, which is a pain in the neck for our developers (who now work with a Linux VM running Docker; ...)

silentOpen
Many developer use cases do not demand heavy bind mount file access. For example, incremental builds of static languages typically work fine.

docker/for-mac#77 is related to "bind mount" performance but docker/for-mac#668 is virtual block device performance. Have you posted a reproduction workload that demonstrates the performance problems you are having?

Docker 17.04 includes a `cached` bind mount flag that relaxes consistency of the mount in exchange for significantly reducing guest-host roundtrips especially for inefficient workloads that repeatedly access the same files.

edsrzf
Could you please link to some docs for the cached flag you mentioned? I can't find anything about it.
yallop
Documentation for 'cached' is on its way. There'll be a blog post up soon with a friendly introduction, and some representative benchmarks. In the meantime there's a drier but more detailed specification of the behaviour of 'cached' waiting in a pull request here:

  https://github.com/yallop/docker.github.io/blob/d9a57867/docker-for-mac/osxfs-caching.md
lobster_johnson
We're seeing the same issue. We've been using shared folders with VMware for a long time, which are also slow (but not this slow).

There's a case for keeping the Docker environment stateless and expendable (wipe to restart), and keep certain things (PostgreSQL databases, for example) on the host machine. Currently, that's impossible because of the aforementioned issue.

I haven't tried the "cached" flag since it's still in beta, however.

henridf
Minikube is working great for us. Makes it easy to run something that's pretty close to a production stack on a dev machine.

Typically with locally-built (dev) images rather than those from the registry that CI writes to, but other than that the k8s manifests are the same.

jitl
We've had the same experience with Vagrant. Vagrant is a very general tool that makes it easy to do some fairly complex things, but that generality adds significant overhead. I counted THREE different SSH connection systems in Vagrant the last time I was trying to optimize it.

We eventually wrote our own tool that duplicates the smallest possible Vagrant featureset.

We use SSH for control, Unison for real-time file syncing, and Chef for "provisioning" configuration of the remote developer environment. Our tool requires only SSH access to some host -- so we provision an EC2 instance for every engineer, and then connect it to our tool; but we could use local Docker or even a Vagrant-managed VM. We reimplemented Chef Solo provisioning, with 100% backwards compatibility with our existing Vagrant Chef recipes, which was really easy! We leverage SSH's ControlMaster for persistent connections.

Our tool does the same things that we were using Vagrant for, but takes 1/3 of the time for typical tasks. Where Vagrant would take ~6 minutes to provision a host, our tool takes ~2 minutes using the exact same Chef recipes. We managed to shave a full 30 seconds of mystery time off between when `vagrant provision` would start, and when Chef would begin it's run.

I encourage you to think about cloud developer environments in general. We got big wins in dev productivity by moving our devenvs off engineer laptops.

praneshp
Do you have any engineers that take public transportation every day? My current company has a dedicated team to maintain a developer VM that runs on our Macs, and that has made my life much better. I don't need internet access to write and test code (except to google/SO, of course). I can also afford to take the slow train (Caltrain is twice as slow if you take the slowest one).

If your commuting employee size is low, I guess it's not important enough problem to solve. I (and a few others here) would love to see a blog (or repo) showing off your tool!

jitl
We still support a local Virtualbox/Vagrant VM for the "offline" case, but that's the very environment that we're actively migrating away from. Realistically speaking the thought of an "offline" environment is impossible - our users need regular updates of our repositories, which requires a network connection; and of course SO/Google is a necessity :-). Also, at our scale we're running into the limits of Apple's machines. There's just not enough RAM for all of (Jetbrains Something), (Local VM with tons of services), (some ARM emulator), (40 Chrome tabs), (Slack). We can't provide a good local dev experience going forward. Between MiFi and MOSH (a UDP, mobile optimized SSH "alternative"), I think we can support the transit use case without using a local VM.
adrianN
Is it normal to start working before you arrive at the office? Does the commute count to your working hours? If I were to commute by public transport, I wouldn't work for my employer during that time, I'd do something that I enjoy, read a book or work on my own projects.
humanrebar
To each his own, but I would appreciate the option to be professionally productive from anywhere.
praneshp
I like to have the choice. Sometimes my wife and I try to cook something complex for breakfast, and I take the train 1.5 hours later, which also goes about 25 mins slower. I replace my usual book reading with work, so I can leave around the same time I usually leave in the evening. I also work all the way on the commute home, so I leave early enough to help cook/climb and still be in bed at 10.

Also, I enjoy what I do at work, and love the company I work at. I don't really care about where I do that work from, or if I worked 2 hours (a week) over my working hours. Actually I make it a point to read code from other parts of the company over the weekend because I find that more useful than pointless side projects.

khazhou
If you're working on the train that's work. Better to catch up on work email or whatever than waste the commute hours each day. Remember the US is not strict about work hours for salaried employees like EU.
[dead] khazhou
If you're working on the train that's work. Better to catch up on work email or whatever than waste the commute hours each day. Remember the US is not strict about work hours for salaried employees like EU.
roadbeats
You can try Happy Hacking Linux with a custom post-install.sh; http://kodfabrik.com/happy-hacking-linux

It'll get you a very productive and fast Linux setup based on Arch Linux and Xmonad, letting you automate the setup process by auto-linking your dotfiles and running post-install.sh if exists. This way you can spin up development environments really fast. For example, I turned an old desktop computer to a ready-to-use development machine in 30 minutes :)

subsection1h
OP specified "development environment is similar enough to production", and you recommended "Linux setup based on Arch Linux and Xmonad". Does not compute.
SteveNuts
We've moved to an automated build system of using Packer to make pre-configured boxes for vagrant actually. It works well, and a lot faster than running the provisioners on a blank box.
mastazi
For us, the biggest problem is that Vagrant, on Windows machines, doesn't run well on Hyper-V (it has network limitations e.g. you can't force a static IP for the machine).

So we have to keep using VirtualBox which is slower.

Another consequence of this is that in order to use VirtualBox you need to completely disable Hyper-V, so on Windows machines our devs are forced to use Docker Toolbox (instead of Docker for Windows which is faster but based on Hyper-V).

(Note: we are a small team with little experience both in Docker and Vagrant so we welcome suggestions: if you thing we're doing something wrong please leave a comment).

extrapickles
As others have mentioned, use Docker Compose instead of vagrant. That way you don't have the overhead of learning two different systems (compose and the Docker command line are very similar).

If it takes more than 20m to get a new dev machine setup from a fresh OS, you probably haven't pushed enough of your environment into images.

Don't be afraid to ignore the rules about what and how much goes into a container as those are for production images (eg: customer facing). It's prefectly fine for this usecase to treat it more as a mini-vm than the 'process on steroids' you use for production.

cryptarch
I'm currently using docker-compose for work on a SaaS product written in Java, it's quite a joy to work with.

I am struggling with slow image builds, but that's more of a problem of "not having a build server", "not having a Docker registry set up" and "using a lot of 3rd party online build dependencies".

nickstefan12
we don't run docker in production, but docker-compose has been awesome for our local dev environment. We can get new engineers new computers going in 20 minutes!

Much easier than vagrant or running everything manually.

acejam
Docker compose with locally mounted volumes.
sawantuday
Recently found this[1] docker compose based alternative to XAMPP or WAMP/LAMP installation. Seems pretty good alternative to quickly setup local environments plus can be replicated in production if you are happy with docker. As many machines/containers can be started as needed and data can be easily shared from local file system. Raync based one liners can be setup as well for data sync across machines.

[1] https://github.com/cytopia/devilbox/blob/master/README.md

aub3bhat
Docker compose its unbelievably good.
alrs
Vagrant is a symptom of MacOS. Move to Linux, and you get native containers for free.
BozeWolf
And after moving to linux start an "Ask HN" topic "what tools can replace macos tools on linux".

Your comment would be: "not having all the tools available is a symptom of using linux as a desktop os".

hatsix
I get that this is a troll comment, and I shouldn't feed it... but seriously, what tools does macos have that linux doesn't? (I type this on a macbook pro) I have to add a ton of things to OS X in order to get it to behave in a sane manner... utilities to manage windows, utilities to re-enable the volume keys when I connect to an external display, utilities to enable alt-tab-like behavior, a utility to take screenshots that have sortable timestamps -- so the latest screenshot is always at the top, homebrew, which lets me install software, GCC... the list goes on... that's not even development-specific utilities.

All of the development-specific tools are cross-platform. I have yet to use a tool that isn't available on Linux, and our entire development staff are running OSX full-time.

BozeWolf
Just open an ask hn for it;-) I will vote it up!
aaronbasssett
Sketch, Keynote, Paw

I'm a developer, but I also talk at community groups and conferences. I know there are alternatives (Inkscape, Impress, Postman) but they're all missing one or more features I use frequently, or are just not as polished.

That being said I'll probably make the change to Linux for my many development machine at some point and keep an Air or something lightweight as a conference/presentation machine.

yupyup
Please don't take this the wrong way but the most important thing in a presentation should not be the graphical aspect of the slides, it should be the content, specifically how you can communicate with the audience with what you're saying verbally.

I have seen awesome talks and it didn't matter if the slides were not pretty.

cagataygurturk
I recommend you to take a presentation training from domain experts (mainly physchologists) and you'll understand how wrong you are.
brianwawok
Experts that think ever meeting needs a PowerPoint? No thanks.

A Google doc outline and some demo is enough for most dev presos.

cagataygurturk
Well, like you know how to tell your problem to computers, those experts know how to tell your problem to human beings. And higher you are in your career, more you have to speak non-tech people that pays your salary and needs to be convinced about your next idea.
brianwawok
That assumes one's goal is to go into soul sucking middle management at a large tech company. For many of us, that is not our goal.
khazhou
Yes but effective communication is NOT about visuals or font choice. An idea presented clearly and concisely is key.
xaedes
Visual presentation plays a big role in a .. well.. _presentation_.
hatsix
Yeah, I can see how those three are more polished. I guess I just don't use them... I was blissfully unaware of them, as the alternatives are good enough...
minusf
perhaps it's the same set of tools but up to date.

compare the versions from your brew list with the packages in your vagrant box for the most popular Linux distributions..

i always find it funny that on BSD's i use latest versions of software developed on Linux, while on Linux (Ubuntu, mint, Debian, centos) they are years behind.

zeroxfe
Big ones for me are Google Drive and Evernote.
ceejayoz
> what tools does macos have that linux doesn't?

I'd personally strongly miss Tower (for Git), Transmit (SFTP/AWS S3), Sequel Pro, Postico, and the iOS Simulator.

im_down_w_otp
We're recreating the functionality of the "Vagrantfile" through Behave (python cucumber framework) scenarios calling down into the step definitions to provision things through libvirt.

It unifies both the development environment setup and the system integration testing setup/execution into the same set of tools. It's also helpful in setting up esoteric environments through QEMU like QNX on ARM during much of the testing before extending it to getting real hardware in the loop.

We also use the Ansible Python API to keep from having to re-implement a bunch of orchestration primitives from scratch.

mitchellh
As the creator of Vagrant, I just wanted to say thank you for using the software. 5-6 years is no joke! I'm glad/hopeful that the project had a net positive impact for you during that time. :)

I won't attempt to hijack this thread or use it for personal motives to convince you to stick with it. I understand technologies change, requirements change, and choices change. The only thing I ask is if you can find the time to email me (mitchell@hashicorp.com) with your pain points so we can look to see how we can improve things in the future.

And the only other comment I'll make is that at HashiCorp we have full time staff on Vagrant and have continued to drill down issue counts through the year. So if you are a Vagrant user: fear not, it is something we continue to deeply care about.

Thank you!

russell_h
First off, thanks for your work on Vagrant. I've had my frustrations with it, but over the years it has been an invaluable tool.

That brings me to this: I wish I could pay for Vagrant - not just the VMWare Fusion plugin, but a fully supported configuration of your choosing. I would happily migrate our dev environment to that configuration, and spend spend several hundred dollars per user per year year on Vagrant if I thought it would work more reliably as a result.

arekkas
we switched to a docker environment and the benefits are immense, starting from bootstrap to boot time. Docker-compose makes it easy to pull 3rd party deps that just work, instead of configuring them in one VM. Of course docker has it's downsides, but maybe vagrant could incorporate docker and start moving away from the one-VM principle?
brianwawok
Seems docker compose is the slightly better way to go.

I don't docker for my actual code , but it's handy for the local DB and redis type problems.

Actual language changes best way I think. Java and Python I tend to just run from jetbra ins with docker having all the dependencies up.

purephase
Mitchell,

It may not have been obvious in the original question but, believe me, I wholly appreciate the work that has gone into Vagrant and it truly is a great piece of software that we have happily supported over the years (with the Fusion plugin).

I believe that some of the challenges are not necessarily with Vagrant, but the combination of distros, build tools (Chef / Ansible -- we're still with Chef) various cookbooks / versions vs. roll your own and, perhaps most important, supporting Fusion/Parallels and Virtualbox with all of their idiosyncrasies. Add to that the introduction of "new" technologies such as containers etc.

The question was not in any intended as a slight against Vagrant, but more of a question of how people are handling their development environments today and if Vagrant is still a big part of their processes or not.

We still love Vagrant and it's promise as well as other Hashicorp products (we use Terraform and are in the process of switching to Vault) so we believe in the tools. Just had a rough 6 months or so of supporting Vagrant in our development processes (for some reason -- could be us!) and thought I'd ask the community.