I was surprised to see growing ecosystem and community. I though that docker would take mainstream in virtualization, so main usecase for vagrant would be when you really need a virtual machine.
Docker isn't virtualization, and AFAIK has never really made any moves in that direction outside of small, local dev (eg running a local VM on a non-Linux host to get a local dockerd-on-Linux available).
I'm late to using Vagrant, hands-on. I used it years ago, but didn't build my own Vagrantfiles. It seems to be a very nice programmatic wrapper around using, primarily, VirtualBox's Unattended Guest Installation[1], though it can use other providers, too.
But I've noticed that it seems like you can skip straight to using docker containers for development environments as well. I'm wondering what other people's experience here is with both or either software.
In my opinion, both still seem like relevant software. When I need an Ubuntu development environment on my Mac, I don't turn to Docker. I think of Vagrant right now, because that's what it was built for. Vagrant is a tool for managing virtual machines and development environments.[2]
I've used both extensively. At least in my case, my uses are slightly different:
* Docker is nice for consistent development and deployment environments, particularly when I'm on a macOS host and I'm doing userspace programming.
* Vagrant is nice for creating a completely hermetic development environment, both on macOS and Linux hosts. My particular use case has been kernelspace programming: it's nice to not have to worry about accidentally taking my host down. Docker can't provide that kind of isolation, since it's still running on the host (or host VM) kernel at the end of the day.
The way I see it, Vagrant does for VMs what Docker does for containers: provides a way to build, deploy, manage, and teardown VMs consistently and without much manual effort. For example, I used Vagrant when learning to use Kubernetes, because I could quickly spin up a 3 node cluster on my laptop and destroy it if I messed up.
If you can use containers for local dev, great, but not everyone can or wants to migrate to containers. Vagrant still solves a very important use case.
A VM can still use systemd as init rather than an entrypoint script, so if you're developing systemd units, you need a VM. There is also the issue of running your application in a container in the first place. Docker in Docker is possible, but cumbersome. Redhat has certainly made it easier with Podman, but Docker still dominates developer mindshare. It's also quite a bit easier to persist changes in a VM, so rather than rearchitecting the OS or toolchain secrets management, user-specific configuration in a VM can just be done by the user after first launch and it sticks around. Simpler to do that than providing instructions and capability to mount git credentials store and gpg trust dbs into a container, then worrying about making sure you have the same UID in the container and out of it, so you don't mess up permissions.
Containers are largely just not a great medium for interactive work, i.e. development. They're great for resident applications that use a single process run via a service account.
I used to run Vagrant for python development back in 2015
it "worked" but was extremally slow and would break at least once a month. Not to mention Ubuntu suddenly removing the Vagrant user breaking more things. How much was VirtualBox or Vagrant fault, I'm not sure but it wasn't a happy experience.
It took like an hour minimum to even setup an environment and everyone dreaded the day when they need to recreate their machine.
Once WSL came into being we moved everything to docker containers and never looked back.
As a counterpoint, vagrant has never broken on me, nor been slow. It’s mostly a configuration system for VM providers, so I don’t even see how that makes sense.
I agree, this seems more like a provisioning issue, as Vagrant and VirtualBox aren't going to do anything too different than what you would normally do setting up your own native development environment.
I used Vagrant extensively when I managed infrastructure at a company that was almost entirely private cloud based. I would create Linux VMs in Vagrant to test our infrastructure automation. The automation would then be deployed into VMware to create the platform used by the developers.
I've found Vagrant very useful to test (or develop) GUI-based applications, especially on platforms which Docker cannot run as container, such as macOS. I've also collected further thoughts on when to choose which (and how they can be combined) here: https://www.augmentedmind.de/2020/08/02/vagrant-vs-docker/
We use vagrant with ansible to build a local dev environment and almost the same production environment from the same ansible project.
We experimented with docker, but it is still extremly slow on windows and our production machine is provisioned with ansible, so vagrant is a natural fit...
Very happy most of the time, only some strange ssh errors happen from time to time...
This is exactly what we do. It is like having a locally run staging environment, and is really great. Can exactly test things like http server configuration, cron jobs, multiple nodes (vagrant vms) coordinating with each other, etc.
Hello HN! I'm one of the founders of HashiCorp and original creators of Vagrant.
Vagrant isn't "HN hip" these days ;) and I feel like folks tend to think if they don't see it here it must be dead. Vagrant has been the opposite: downloads have still only gone up, and we're seeing it used in more and more places on the regular. Most people are quiet about it because it does what it was built to do well and quite honestly, its moved more into the stereotypically "boring" organizations. For example, I recently ran into an engineer that works at a local city utility who was raving about how they just adopted Vagrant last year and it has been completely life-changing for their team. This is 2021, Vagrant came out 2012. Products have super long lifecycles and the market is massive.
Regardless, we've remained committed to Vagrant. We've always had a team dedicated to Vagrant, and the Vagrant team is awesome and has continued to push out new features and new releases for the past decade.
We're super excited for Vagrant 3.0! There's a lot we have planned for it and the transition to Go is going to bring a lot of those improvements.
I think one topic that is interesting for the HN crowd: why are we rewriting Vagrant in Go? Is that too expensive or time consuming? Does that cause too much user pain?
HashiCorp has built dozens of products now in Go and we've built a large foundation of libraries and utilities for Go-based projects. All our products share the same CLI library (that we wrote), the same plugin framework (that we wrote), the same distributed systems libraries (that we wrote), etc. Besides code, we've built a security team and release engineering team that has pipelines ready to go for Go-based projects. And more.
This means that we move fast with Go, and we can do it confidently, and we can do it without a lot of overhead. From the moment we decided to port Vagrant to Go, the Vagrant team had a working demo in just a few weeks, and an "alpha" release they're ready to ship to users in a few months. We got this speed cause we just wrapped Vagrant around the same architecture and libraries that our other products are built on.
For our users, there are a bunch of improvements (mentioned in the blog post). We're being very careful about backwards compatibility and have a multi-year plan to ensure that existing Vagrantfiles and plugins continue to work. We're starting with continuing to embed a Ruby with our installers while the core is running Go. Over time, we'll ask users to bring their own Ruby (and we'll validate it) to use "older" Ruby-based plugins and Vagrantfiles.
Feel free to ask any questions, I'll try to be present in this thread.
Conversely, that's only when technology starts to become interesting to me. When it's lived long enough to be useful, and hasn't fallen out of favor for something else. Thank you for building such an excellent tool.
I want other people to prove those things for me. I want to spend as little time and effort possible acquiring scars and would rather learn from others.
Thanks for all the tools. You guys have well-deserved success.
Are there plans to somehow harmonize the concepts here with packer (and whatever the new packer cloud will be?). Vagrant for the most part feels like a vm-only version of packer (even though Vagrant came first).
I can easily see a self-hosted cloud-based "one-click dev machine" paired with VS Code Remote, provisioned by packer (or vagrant), with the configs stored in git. We'd love to roll out something like that for our devs so sensitive data then never leaves our sandbox or even gets into the dev laptops.
Not really any questions (other than my one in the comment thread about the best way forward for providers on Apple Silicon), but definitely a hat tip in your direction. I’ve had a good 8 years of plain sailing out of Vagrant and I appreciate the work you’ve done.
Glad to hear there is still development on this project. I've been using Vagrant a lot in the past, with great pleasure, but since most of the projects I manage nowadays are containers, I sadly have less and less use for this tool.
For the few projects that don't use containers however is still shines. One feature I'm missing though, which would make Vagrant a complete solution for me, is testing as a first class citizen. I've been using Test Kitchen[0] for this in this past which works well as a solution, but it would be really nice to have something like this built into and supported conceptually by Vagrant itself.
I like Vagrant, but I don't know how I'm going to keep using it when I switch to Apple Silicon (M1 Macs). Vagrant wraps around Oracle VirtualBox; VirtualBox doesn't support M1 Macs, and Oracle has given no indication that they plan to start supporting Apple Silicon at all.
Is there any working version of Vagrant that "just works" on M1 out of the box without paying for a VM provider?
If not, would I have to pay for VMWare and then use Vagrant to wrap that? Would I use Vagrant to wrap Docker? (Why would I bother to use Vagrant if I'm using Docker?)
We just open sourced our VMware plugin so yes, we will support M1 there. Note that VMware Fusion for personal use is now free so you can get that whole setup completely free when its ready.
Beyond that, we're looking into also writing a native Virtualization.framework implementation so you wouldn't need any 3rd party virtualization software at all. For headless Linux VMs we're confident this will work great.
We're also looking at qemu as an option because qemu also offers emulation for x86. Being written in Go, we can bind to qemu natively much more easily.
Thanks for Vagrant. I've used it quite a bit. I think it's underestimated how many large businesses there are that still manage their servers like "pets", rather than "cattle".
In my career, Vagrant has provided a good solution to move towards more automated deployment, where Docker would have been too big a paradigm shift, and/or not really greatly justify its use.
I love Vagrant, it was my secret weapon for years! However, I question the decision to release a complete new implementation under the 2.x version. The Go-based variant should be 3.0 from the start.
From [1]
I dont think Ruby would ever be able to compete in that space. Even Hashicorp, the company that was born out of vagrant left Ruby and Chose Go.
I am thinking someday Crystal post 1.0 would be able to put up a fight in that space.
I still remember I wrote somewhere, that one day Vagrant will be rewritten from Ruby to Go and got some very negative feedback from someone. ( It was in the Parker era.... )
Vagrant, and the whole HashiCorp products are all infrastructure product it makes sense to be running on Go rather than Ruby.
Nothing to say or ask really. But Thank You and congratulation. At some point I thought you have abandoned Vagrant. Its nice to see Hashicorp continue to update and take care of their first product. And honestly all the HashiCorp product are so much better or simpler than their competitors ( cough K8s cough ). Really wish you and your company continue to move forward with great products and services.
Mitch, thanks for vagrant. I use it daily. I like long running processes. And nothing yet, gives me the simple configuration that I can get with Vagrant. And with Vagrant switching to Go, I can only bet performance will get better.
For some of us, docker is pure overhead.
I really like Vagrant. However, it is extremely(!) slow, and every command takes 3-30 seconds (including commands that should take 50 ms). You also know about this, and chose not to act on it :( [0]. Would the transition to Go remove this impediment?
47 comments
[ 0.19 ms ] story [ 103 ms ] threadBut I've noticed that it seems like you can skip straight to using docker containers for development environments as well. I'm wondering what other people's experience here is with both or either software.
In my opinion, both still seem like relevant software. When I need an Ubuntu development environment on my Mac, I don't turn to Docker. I think of Vagrant right now, because that's what it was built for. Vagrant is a tool for managing virtual machines and development environments.[2]
[1]: https://www.virtualbox.org/manual/UserManual.html#basic-unat...
[2]: https://www.vagrantup.com/intro/vs/docker
Edit: Thank you for the excellent insights!
* Docker is nice for consistent development and deployment environments, particularly when I'm on a macOS host and I'm doing userspace programming.
* Vagrant is nice for creating a completely hermetic development environment, both on macOS and Linux hosts. My particular use case has been kernelspace programming: it's nice to not have to worry about accidentally taking my host down. Docker can't provide that kind of isolation, since it's still running on the host (or host VM) kernel at the end of the day.
If you can use containers for local dev, great, but not everyone can or wants to migrate to containers. Vagrant still solves a very important use case.
Containers are largely just not a great medium for interactive work, i.e. development. They're great for resident applications that use a single process run via a service account.
Once WSL came into being we moved everything to docker containers and never looked back.
I've been using VS Code + its Remote-Containers extension + docker-compose (when needed) and it's absolutely brilliant compared to other options.
I will use Vagrant when I want to avoid Docker in Docker, but I try to stay away from it as much as I can (for local app development).
That said Vagrant is really useful for testing infra, stuff like clusters etc.
On some host operating systems. On others it's a similar speed.
We experimented with docker, but it is still extremly slow on windows and our production machine is provisioned with ansible, so vagrant is a natural fit...
Very happy most of the time, only some strange ssh errors happen from time to time...
Vagrant isn't "HN hip" these days ;) and I feel like folks tend to think if they don't see it here it must be dead. Vagrant has been the opposite: downloads have still only gone up, and we're seeing it used in more and more places on the regular. Most people are quiet about it because it does what it was built to do well and quite honestly, its moved more into the stereotypically "boring" organizations. For example, I recently ran into an engineer that works at a local city utility who was raving about how they just adopted Vagrant last year and it has been completely life-changing for their team. This is 2021, Vagrant came out 2012. Products have super long lifecycles and the market is massive.
Regardless, we've remained committed to Vagrant. We've always had a team dedicated to Vagrant, and the Vagrant team is awesome and has continued to push out new features and new releases for the past decade.
We're super excited for Vagrant 3.0! There's a lot we have planned for it and the transition to Go is going to bring a lot of those improvements.
I think one topic that is interesting for the HN crowd: why are we rewriting Vagrant in Go? Is that too expensive or time consuming? Does that cause too much user pain?
HashiCorp has built dozens of products now in Go and we've built a large foundation of libraries and utilities for Go-based projects. All our products share the same CLI library (that we wrote), the same plugin framework (that we wrote), the same distributed systems libraries (that we wrote), etc. Besides code, we've built a security team and release engineering team that has pipelines ready to go for Go-based projects. And more.
This means that we move fast with Go, and we can do it confidently, and we can do it without a lot of overhead. From the moment we decided to port Vagrant to Go, the Vagrant team had a working demo in just a few weeks, and an "alpha" release they're ready to ship to users in a few months. We got this speed cause we just wrapped Vagrant around the same architecture and libraries that our other products are built on.
For our users, there are a bunch of improvements (mentioned in the blog post). We're being very careful about backwards compatibility and have a multi-year plan to ensure that existing Vagrantfiles and plugins continue to work. We're starting with continuing to embed a Ruby with our installers while the core is running Go. Over time, we'll ask users to bring their own Ruby (and we'll validate it) to use "older" Ruby-based plugins and Vagrantfiles.
Feel free to ask any questions, I'll try to be present in this thread.
-----------------------------------------------------
Also, if you're a Vagrant user, we also recently made our VMware plugin free and open source. Check it out: https://twitter.com/mitchellh/status/1402729560823664641
I want other people to prove those things for me. I want to spend as little time and effort possible acquiring scars and would rather learn from others.
Good technology lasts.
Are there plans to somehow harmonize the concepts here with packer (and whatever the new packer cloud will be?). Vagrant for the most part feels like a vm-only version of packer (even though Vagrant came first).
I can easily see a self-hosted cloud-based "one-click dev machine" paired with VS Code Remote, provisioned by packer (or vagrant), with the configs stored in git. We'd love to roll out something like that for our devs so sensitive data then never leaves our sandbox or even gets into the dev laptops.
For the few projects that don't use containers however is still shines. One feature I'm missing though, which would make Vagrant a complete solution for me, is testing as a first class citizen. I've been using Test Kitchen[0] for this in this past which works well as a solution, but it would be really nice to have something like this built into and supported conceptually by Vagrant itself.
[0] https://docs.chef.io/workstation/kitchen/
Is there any working version of Vagrant that "just works" on M1 out of the box without paying for a VM provider?
If not, would I have to pay for VMWare and then use Vagrant to wrap that? Would I use Vagrant to wrap Docker? (Why would I bother to use Vagrant if I'm using Docker?)
The current ruby Vagrant supports plugins, but I'm not aware of bhyve or hyperkit support.
I would assume rewriting in Go means the same VM drivers as packer will be supported.
Beyond that, we're looking into also writing a native Virtualization.framework implementation so you wouldn't need any 3rd party virtualization software at all. For headless Linux VMs we're confident this will work great.
We're also looking at qemu as an option because qemu also offers emulation for x86. Being written in Go, we can bind to qemu natively much more easily.
The point being: there will be multiple options!
Might be time for getting that Vagrant-Libvirt link happening after all? ;)
https://github.com/vagrant-libvirt/vagrant-libvirt
It's been happening for a while now...
In my career, Vagrant has provided a good solution to move towards more automated deployment, where Docker would have been too big a paradigm shift, and/or not really greatly justify its use.
From [1] I dont think Ruby would ever be able to compete in that space. Even Hashicorp, the company that was born out of vagrant left Ruby and Chose Go. I am thinking someday Crystal post 1.0 would be able to put up a fight in that space.
I still remember I wrote somewhere, that one day Vagrant will be rewritten from Ruby to Go and got some very negative feedback from someone. ( It was in the Parker era.... )
Vagrant, and the whole HashiCorp products are all infrastructure product it makes sense to be running on Go rather than Ruby.
Nothing to say or ask really. But Thank You and congratulation. At some point I thought you have abandoned Vagrant. Its nice to see Hashicorp continue to update and take care of their first product. And honestly all the HashiCorp product are so much better or simpler than their competitors ( cough K8s cough ). Really wish you and your company continue to move forward with great products and services.
[1] https://news.ycombinator.com/item?id=23457496
It'd potentially open up having remote stuff be hosted in HashiCorp Cloud (HashiCloud?) directly. ;)
That could be pretty lucrative over time...
[0] https://github.com/hashicorp/vagrant/issues/11853
We’re using UTM, which does the job as a frontend to qemu, but it’s be nice to switch back to a vagrant based declarative file.