Almost exactly a year ago we have launched Digger. Since then thousands of developers started using it, and our mission remains to build the best developer platform ever.
But we are often asked - why so many products that look so similar? After Digger we have also launched Lemon, Alicorn and most recently AWS Bootstrap to name a few. The answer is simple: we want every feature of Digger to be best-in-class. So we are making a product out of it and launch it and perfect it if people need it, or kill the feature if there is not enough demand.
Terraform generation is one of the most powerful features of the platform. Not only your AWS account is taken care of - you also get the underlying code that is fully customisable. So you are not limited by the UI and can literally build whatever you want as long as AWS supports it. It's easy with Terragen:
1. Connect your AWS account
2. Connect a GitHub repository for your infrastructure code
3. Terragen will push generated Terraform there
4. Then you can add more apps and services - and Terraform will get automatically updated!
5. And you can use the digger_overrides folder to add your own custom terraform
We believe that the only way to make a truly great platform is to listen to the users and ship things they need. Tell us what you think!
Do you generate Terraform code based on the AWS account, to help with a migration to Terraform?
Or is it that your platform works in that it "compiles" whatever is created in it to Terraform, to be run by the user on their AWS account? In other words, it's kind of a GUI DSL for Terraform?
The latter, yes. The former is a feature on the roadmap.
Our idea is to make it as easy as possible for the user to take advantage of IaC without having to learn it. Remove the entry barrier, give a nice UI that would make it super easy to do the basic stuff, which is like 90% of what's needed on a day-to-day basis. For the remaining 10% the user has full control over underlying Terraform. So you can customise only the parts that need to be custom; that's much easier than starting from scratch.
I work on cloud infra in a healthcare setting. Basically a combination of on and off prem kubernetes (so multiple instances of the k8s provider) and other resources from the gcp and aws providers.
Really what a codegen would want to do in this case would be to make a terragrunt.hcl file that would make the accompanying terraform files callable.
It really wouldn't nessesitiate anything else unless you want to make it easy to parameterize values per CI environment. Then you want multiple terragrunt.hcls
At first I was going to complain that the whole "what we actually do" spiel was tiny and hidden away up the top there, but after clicking it think you have really well written docs that explained it well.
That's great stuff! We have exporter functionality in the Databricks Terraform provider [1] and a lot of customers started to use Terraform for existing Databricks resources because they don't need to re-implement everything manually - just point to workspace, and wait few minutes.
What do you call "resource"? I went directly to the pricing page (btw on my small 11" laptop screen the logo left overlaps the menu) and for the hobbyist it says one service and one resource - what is that "one resource"? One S3 bucket? Then it's useless to me - as hobbyist as I was - and I'd have to pay 200$/month (next plan is 50$/mo and minimum 4 users). I won't pay that much for my hobby projects. Or maybe you call "resource" something else, like one repo? I have my hobby projects in more than one repo, so again I wouldn't qualify for hobbyist pricing...
A database or a queue or things like Redis. S3 buckets we don't count, as well as any other stuff that you add to Terraform yourself. So you can have as many S3 buckets as you want.
We haven't yet put much attention into pricing. As a matter of fact the paywall is broken atm (don't tell anyone about it!) so technically nothing stops you from having more than one service or resource or whatever :)
Then how about you skip the pricing completely, just saying "we are in beta blah blah" as others did, so to not scare me away completely? Because really, right now I wouldn't even bother trying it, when looking at 200$/mo.
Presumably a Terraform resource. I don't think that's a great way to price though - Terragen will often have the choice of saving me money or creating a more flexible/generic structure (e.g. charging me for an IAM role, policy, and role policy attachment when an inline role would've sufficed).
Maybe flat per user + CI/CD (tf application) time? The latter grows with resources too.
I don't know much about trademarks, but from a quick search I found there are two terragen's that exist, one for agricultural stuff and one for something like rubber? (according to the classes associated to the trademarks).
Are names automatically trademarked? Or did I search incorrectly? (I did only search on the UK and US pages though)
Because it actually runs the Terraform on Terragen servers to solve the state conflict problem. You can think of it like "CI for infrastructure". Or a "compiler+runtime" that outputs Terraform instead of binary / bytecode. Higher-level concepts like dependencies between services are translated into Terraform and applied. So there is robust state management + extensibility, all of it with an easy to use UI.
Increasingly it seems like people think there is a need for tools that bridge the gap between simple and complicated IaC. I think this is a better idea than a bespoke cli tool that does something similar[1], but I don't think it escapes the core problems that tool has, which are the same problems that people and organizations encounter with other complicated infrastructure/deployment tooling: getting good at these things is a time sink, and shortcuts that magic away some of the pain also magic away the ability to learn.
This is an amazing project you have there! We have a very similar philosophy - but taking it one step further. We believe that developers do not and should not care about things like VPCs, security groups, etc. We believe that the current state of infrastructure-as-code is akin to that of assembly language. Low-level and not truly reusable - because using Terraform modules still requires understanding of low-level concepts. What we are trying to build is essentially a compiler of "human" concepts into Terraform. But because even simply running Terraform is a daunting task to un-initiated developers, we also need to make this part easy. Hence we run it on our servers. This way the user is completely shielded from complexity without losing the ability to go low-level and customise whatever they like.
As a Devops engineer who spends all day provisioning infra with IaC, I always wonder why developers intentionally avoid concepts like "security groups and VPCs". They concepts are actually really simple and sure as shit are simpler than learning nearly any dev fundamentals. Sometimes I get concerned that one day developers will figure out how simple our job is compared to actual programming, but I enjoy the perpetuation of the mystique of basic infra concepts in the meantime for the sake of my employment.
As an infrastructure engineer it is my job IMO to provide tooling, information, and services to reduce cognitive overhead for my peers and customers. For my job wouldn’t exist unless we had software to deliver in the first place. Utilizing existing cognates developers have is important for communicating the trade-offs we make in infrastructure. Developers are already asked a lot of them these days and only increasing (part of why I’m not a straightforward feature developer whatsoever anymore).
With some level of empathy for our stakeholders we can be more effective as a team and customer-vendor and so forth.
I am not really concerned by developers figuring out that this is that easy partly because it frankly isn’t yet to help make systems scale. And the day infrastructure is legitimately brain dead simple I’ll breathe a huge sigh of relief that we managed to tame this mess of short sighted decisions and bits and bytes plaguing humanity. So sad to see how something so important to our society is this stupidly unnecessarily complex, but once we look into many other disciplines we can see how they have their issues as well and that we as engineers and laymen tend to just be blind to the inherent complexities of human systems.
> We believe that developers do not and should not care about things like VPCs, security groups, etc.
I very strongly and fundamentally disagree with this sentiment. I know they don't, but they should!
Developers should understand how and where and why/why not their code is deployed. They should understand what their security model is AND HOW IT WORKS, because if they don't they will 100% without fail open their SG to 0.0.0.0/0 and embed hard coded passwords (usually "password", or "p@ssw0rd" at best). They should know what a VPC is, but maybe they don't need to know anything beyond "private network" and probably "NAT is involved".
Hiding these concepts is a double-edged sword. You get the simplicity of "make me a database" but don't understand why or how it works. You don't have to fiddle with security groups, until you do and then because you don't understand them you go with 0/0. You don't have to think about VPCs until you rack up a $10k bill because you never considered how you could route between instances vs round-tripping out of the VPC and back in through a LB - twice.
I see your point clearly, and it's a good one. However, consider the following counterargument: the exact same thing could be said (and was said) extensively about low-level programming concepts such as registers, later pointers, memory allocation, etc. It was essential for every software engineer to master... until it wasn't. Same with lower-level OS fundamentals. Same thing earlier with pre-PC hardware.
We've come full circle with the present-day cloud providers like AWS. 15 years ago they started as simplification, but today they are boxes of 200+ specialised building blocks that are anything but trivial to put together. A human specialist has to be involved. But why? This complexity is completely artificial, its sole purpose is to keep their customers locked in. This must and will be solved with better tooling. That's what we are trying to make.
I apologize if I made it appear as though that project were mine, because it is not, but I do consider yours a better version of the same concept.
At a basic level, I agree with you on the current state of IaC, but I feel like things such as CDK and Pulumi are attempting to address that issue more directly, so I'm curious why you decided not to use those instead. I believe it is much more difficult to get out of "cloud assembly" than these tools make it seem, and I think you have an interesting approach, but I agree with the sibling commenter who said that attempting to escape that completely is a bad idea.
One of the major problems solutions like this one tend to have is that they have sharp edges about what they can and cannot handle, and they also have places where there's a steep drop back into cloud assembly, for which the user attempting to avoid it in the first place is now woefully unprepared. This is true of AWS's myriad of simplification solutions, and this is also true of tools like Serverless. If I give a naïve user access to a service catalog that they have the ability to override at a low level, I might as well teach them how to handle the low level in the first place. It's not clear to me how you plan to address that.
Abstracting away the need to run Terraform directly is useful, but there are plenty of products that have made that their focus, including HashiCorp's, so I'm not sure how you differentiate yourself from those as well.
Without having the ability to try right now, so excuse the ignorance, are you setting up and creating terraform configs and then running them against aws or do we get the configs after generation?
Both - it first generates Terraform and run it on our servers to configure your AWS account (a bit like CI but for infra). And you can connect a dedicated GitHub repo for infrastructure that will get Terraform synced into. So after you make changes to your stack (for example, add a service or a database, or update an env variable) Terraform will get updated and pushed to that repository.
This project seems to wholesale ignore "terraform modules" in it's claims about reusability and modularity. Most of the problems it solves are already solved.
The description and marketing are very mixed. It generates bespoke Terraform based on... vague descriptions? "Give me a postgres DB" seems to be the level of input from what I can tell. It then goes off to create one and give you the TF it used? I was really hoping this was "automatically generate Terraform FROM AWS" to import existing resources.
This sounds like the SaaS-ified version of Gruntwork.io, where they give you best practices templates and leave it up to you to fill in the particulars.
Oh man, was excited to try this but then was prompted to login. Why do I need to sign in with my GitHub? Why can’t I just generate terraform like you claim? Why do I have to give you personal details to do so?
Fair point indeed. They way we built it is that it actually generates and applies Terraform in your AWS account and synchronises it with your Github. The idea is to eliminate all devops-y steps from the flow so that the user who knows nothing about AWS or Terraform can use it.
I’m not the target audience then because I would never allow a service to just automatically update my AWS account infrastructure without allowing me to scrutinize the terraform first.
I think that too much simplification without empowering people to understand how the pieces work is ultimately a detriment.
It’s important that there is a human understands how the stuff works that’s getting shipped to production. You can’t skip that step, or who knows what kinds of vulnerabilities and miss configurations will end up going live.
In my opinion, rather than building a tool for people who don’t understand the stuff, your target audience should change to instead be those who DO understand this stuff. This is not something that can be solved with automation-only if nobody understands how the pieces work.
Im gonna hide that from my manager or he will fire me.
They would love to sack some devops to replace them with juniors clicking stuff in UI.
Sooner or later real experts will be pushed out through SaaS solutions but overall the quality of solutions prepared will be crap.
I can already compare for example Datadog to our own dedicated Monitoring stack. Managers would never again allow us to build it, despite it being 10x better than generic solution.
56 comments
[ 2.7 ms ] story [ 133 ms ] threadBut we are often asked - why so many products that look so similar? After Digger we have also launched Lemon, Alicorn and most recently AWS Bootstrap to name a few. The answer is simple: we want every feature of Digger to be best-in-class. So we are making a product out of it and launch it and perfect it if people need it, or kill the feature if there is not enough demand.
Terraform generation is one of the most powerful features of the platform. Not only your AWS account is taken care of - you also get the underlying code that is fully customisable. So you are not limited by the UI and can literally build whatever you want as long as AWS supports it. It's easy with Terragen:
1. Connect your AWS account
2. Connect a GitHub repository for your infrastructure code
3. Terragen will push generated Terraform there
4. Then you can add more apps and services - and Terraform will get automatically updated!
5. And you can use the digger_overrides folder to add your own custom terraform
We believe that the only way to make a truly great platform is to listen to the users and ship things they need. Tell us what you think!
PS - we have also launched on ProductHunt today: https://www.producthunt.com/posts/terraform-generator-by-dig...
Do you generate Terraform code based on the AWS account, to help with a migration to Terraform?
Or is it that your platform works in that it "compiles" whatever is created in it to Terraform, to be run by the user on their AWS account? In other words, it's kind of a GUI DSL for Terraform?
my stuff uses terragrunt extensively. it would be really cool if this fit together nicely.
Really what a codegen would want to do in this case would be to make a terragrunt.hcl file that would make the accompanying terraform files callable.
It really wouldn't nessesitiate anything else unless you want to make it easy to parameterize values per CI environment. Then you want multiple terragrunt.hcls
Looks decent. Good luck.
[1] https://planetside.co.uk/
- free for non comm
- subscription by month / year
- perpetual fixed version if you want
- maintenance subscription for perpetual if you want
- upgrade path from each to other
The users here have quite the options. I wonder what made Planetside choose this kind of licensing model.
[1] https://registry.terraform.io/providers/databrickslabs/datab...
We haven't yet put much attention into pricing. As a matter of fact the paywall is broken atm (don't tell anyone about it!) so technically nothing stops you from having more than one service or resource or whatever :)
Maybe flat per user + CI/CD (tf application) time? The latter grows with resources too.
I’m pretty sure your company can be sued for trademark infringement.
Are names automatically trademarked? Or did I search incorrectly? (I did only search on the UK and US pages though)
[1] https://github.com/nathants/libaws
A DevOps Engineer should be a programmer focused on infrastructure, but we share toolkits.
With some level of empathy for our stakeholders we can be more effective as a team and customer-vendor and so forth.
I am not really concerned by developers figuring out that this is that easy partly because it frankly isn’t yet to help make systems scale. And the day infrastructure is legitimately brain dead simple I’ll breathe a huge sigh of relief that we managed to tame this mess of short sighted decisions and bits and bytes plaguing humanity. So sad to see how something so important to our society is this stupidly unnecessarily complex, but once we look into many other disciplines we can see how they have their issues as well and that we as engineers and laymen tend to just be blind to the inherent complexities of human systems.
I very strongly and fundamentally disagree with this sentiment. I know they don't, but they should!
Developers should understand how and where and why/why not their code is deployed. They should understand what their security model is AND HOW IT WORKS, because if they don't they will 100% without fail open their SG to 0.0.0.0/0 and embed hard coded passwords (usually "password", or "p@ssw0rd" at best). They should know what a VPC is, but maybe they don't need to know anything beyond "private network" and probably "NAT is involved".
Hiding these concepts is a double-edged sword. You get the simplicity of "make me a database" but don't understand why or how it works. You don't have to fiddle with security groups, until you do and then because you don't understand them you go with 0/0. You don't have to think about VPCs until you rack up a $10k bill because you never considered how you could route between instances vs round-tripping out of the VPC and back in through a LB - twice.
We've come full circle with the present-day cloud providers like AWS. 15 years ago they started as simplification, but today they are boxes of 200+ specialised building blocks that are anything but trivial to put together. A human specialist has to be involved. But why? This complexity is completely artificial, its sole purpose is to keep their customers locked in. This must and will be solved with better tooling. That's what we are trying to make.
1. Some of the stuff is genuinely complex.
2. The software engineers and development managers regularly ignore the user experience teams and their feedback.
At a basic level, I agree with you on the current state of IaC, but I feel like things such as CDK and Pulumi are attempting to address that issue more directly, so I'm curious why you decided not to use those instead. I believe it is much more difficult to get out of "cloud assembly" than these tools make it seem, and I think you have an interesting approach, but I agree with the sibling commenter who said that attempting to escape that completely is a bad idea.
One of the major problems solutions like this one tend to have is that they have sharp edges about what they can and cannot handle, and they also have places where there's a steep drop back into cloud assembly, for which the user attempting to avoid it in the first place is now woefully unprepared. This is true of AWS's myriad of simplification solutions, and this is also true of tools like Serverless. If I give a naïve user access to a service catalog that they have the ability to override at a low level, I might as well teach them how to handle the low level in the first place. It's not clear to me how you plan to address that.
Abstracting away the need to run Terraform directly is useful, but there are plenty of products that have made that their focus, including HashiCorp's, so I'm not sure how you differentiate yourself from those as well.
The description and marketing are very mixed. It generates bespoke Terraform based on... vague descriptions? "Give me a postgres DB" seems to be the level of input from what I can tell. It then goes off to create one and give you the TF it used? I was really hoping this was "automatically generate Terraform FROM AWS" to import existing resources.
This sounds like the SaaS-ified version of Gruntwork.io, where they give you best practices templates and leave it up to you to fill in the particulars.
You might be interested in Terraformer. https://github.com/GoogleCloudPlatform/terraformer
Maybe "Terragen" is going to try and "automagically" deploy all my GitHub projects into AWS or something nuts!
It’s important that there is a human understands how the stuff works that’s getting shipped to production. You can’t skip that step, or who knows what kinds of vulnerabilities and miss configurations will end up going live.
In my opinion, rather than building a tool for people who don’t understand the stuff, your target audience should change to instead be those who DO understand this stuff. This is not something that can be solved with automation-only if nobody understands how the pieces work.
They would love to sack some devops to replace them with juniors clicking stuff in UI.
Sooner or later real experts will be pushed out through SaaS solutions but overall the quality of solutions prepared will be crap.
I can already compare for example Datadog to our own dedicated Monitoring stack. Managers would never again allow us to build it, despite it being 10x better than generic solution.
Im bad at sensing this stuff during interview.