Show HN: Layerform – Open-source development environments using Terraform files (github.com)
Whenever engineers run layerform spawn, we use plain .tf files to give them their own "staging" environment that looks just like production.
Many teams have a single (or too few) staging environments, which developers have to queue to use. This is particularly a problem when a system is large, because then engineers can't run it on their machines and cannot easily test their changes in a production-like environment. Often they end up with a cluttered Slack channel in which engineers wait for their turn to use staging. Sometimes, they don't even have that clunky channel and end up merging broken code or shipping bugs to production. Lucas and I decided to solve this because we previously suffered with shared staging environments.
Layerform gives each developer their own production-like environment.This eliminates the bottleneck, increasing the number of deploys engineers make. Additionally, it reduces the amount of bugs and rework because developers have a production-like environment to develop and test against. They can just run "layerform spawn" and get their own staging.
We wrap the MPL-licensed Terraform and allow engineers to encapsulate each part of their infrastructure into layers. They can then create multiple instances of a particular layer to create a development environment.The benefit of using layers instead of raw Terraform modules is that they're much easier to write and reuse, meaning multiple development environments can run on top of the same infrastructure.
Layerform's environments are quick and cheap to spin up because they share core pieces of infrastructure. Additionally, Layerform can automatically tag components in each layer, making it easier for FinOps teams to manage costs and do chargebacks.
For example: with Layerform, a product developer can spin up their own lambdas and pods for staging while still using a shared Kubernetes cluster and Kafka instance. That way, development environments are quicker to spin up and cheaper to maintain. Each developer's layer also gets a tag, meaning FinOps teams know how much each team's environments cost.
For the sake of transparency, the way we intend to make money is by providing a managed service with governance, management, and cost-control features, including turning off environments automatically on inactivity or after business hours. The Layerform CLI itself will remain free and open (GPL).
You can download the Layerform CLI right now and use it for free. Currently, all the state, permissions, and layer definitions stay in your cloud, under your control.
After the whole license change thing, I think it's also worth mentioning we'll be building on top of the community's fork and will consider adding support for Pulumi too.
We'd love your feedback on our solution to eliminate “the staging bottleneck". What do you think?
25 comments
[ 3.0 ms ] story [ 55.6 ms ] threadLooking forward to seeing how you'll use it.
Please don't hesitate to reach out to us if we can help in any way.
I use the light mode myself so I hadn’t noticed it.
I’ll go ahead and update the text color for the dark theme versions.
Agree that "preview apps are hard" is a massive bottleneck - especially when specs are evolving and a feature may spend some time being QA'd by technical and non-technical team members before being ready to land. Not to mention that processes for securely populating seed data from recent production data are highly domain-specific!
We've built our own system for preview apps on k8s, with a complex Github action that tries to encapsulate everything we can into a set of helm charts that are installed into a dedicated namespace for all the databases and services for that specific PR.
Of course, this can only go so far, and doesn't easily allow for tracking assets outside of k8s. But given that we can keep many of those shared, we've in part stayed away from Terraform due to adding yet another learning curve - but also for lack of tooling for this shared-some-things-but-not-all per-environment customizability - the exact problem you're solving!
I'm excited to see where this system goes - and am excited to see what value-add layers you build on the UI and governance/cost-control side!
Indeed, we've seem many people building similar alternatives using in-house solutions on top of Kubernetes. In fact, that's the main barrier to adoption we've found so far.
The Terraform point is really interesting. In our earlier solution we tried to use compose files because we thought they'd be easier to implement. In reality what happened was that most people couldn't use it because they had more complex infra (or at least _wanted_ to have more complex infra) in their previews.
Especially after the whole TF situation we'll probably be looking into other ways to configure it anyway.
Please let me know if you need any help setting it up. We'd love to help in any way we can.
not clear what layers are layerform are for?
from other side used nix codespaces/shells/rebuilds terranix.
so how you in comparison to these?
They enable a bunch of other use cases we'll implement on top. We believe layers (in the way we envisioned them) to be the ideal abstraction.
In the case of Kubernetes, for example, you can use the Kubernetes provider or even just Helm to deploy your pods.
Great idea to add ECS/Serverless specific examples. Thanks!