You can write declarative Infrastructure as Code in a language you’re already familiar with, rather than having to learn (& work around) HCL.
I’ve written a lot of HCL and find myself wanting to extract things into components and put in place abstractions in ways that don’t always jive with how Terraform requires you to do things. Being able to build those abstractions in my favorite language, then have that compile down into declarative IaC… that’s the draw.
I personally use Terraform CDK for this, which is Hashicorp’s answer to Pulumi.
Sounds cool. I've come to loathe HCL, it is annoying at best. It's not really like any other common programming language, I have to relearn it every time I go back to make a change to my TF code, only a few times per year.
One (big) limitation though with Terraform CDK is that it's limited to AWS only. Pulumi gives you access to more than 60 providers. Also testing which a big step forward for Iaac.
Pulumi does basically the exact same thing as terraform but in the language of your choice.
We started using it in our company because we noticed that terraform as a separate language was enough of a barrier that it prevented a good deal of our developers from contributing to the infrastructure codebase, creating a separation between 'devs' and 'devops' guys. We switched to Pulumi for a new project, and now basically everybody is contributing to the infrastructure. Which completely changed the dynamic of development for the better. Frankly for us it's been a gamechanger.
As someone who started out not as a Devops/ Cloud Engineer person but got into TF: Would you mind sharing more specifically how Pulumi has changed the game for you with lowering the barrier to "do infra"?
I am curious because everytime I considered using it so far, the prospect of just any dev who might well not know all the intricacies of infra should maybe not build it. I guess some barrier to entry is something I would see as a benefit.
In terms of code syntax, reading infra declarations in app languages always rubbed me the wrong way. Yes, HCL is no beauty and a bit to get into, but after that, I personally love it. I know exactly what to look for. The thought of reading this in all kinds of Python syntax is rather a downer for me.
Last but not least, there is a ton of tooling which was built around TF which I am not aware of if equivalents do exist for Pulumi.
Fwiw, I have no skin in the game of either company and am not trying to paint a black or white picture of what's the better tech. Just trying to learn what appeals to others about Pulumi.
> I am curious because everytime I considered using it so far, the prospect of just any dev who might well not know all the intricacies of infra should maybe not build it. I guess some barrier to entry is something I would see as a benefit.
First, your dev process should go through a review phase robust enough that beginners should be able to develop on things they are unfamiliar with confidence.
Second, the infra being 'intricate', needing huge tooling etc, is a sign that it has become too complex; maybe because it is not well aligned with the application side, maybe because the 'devops' have been making an infra monolith, or maybe because it is making use of too low level features.
For example we're on GCP, and we're making use of serverless tech such as cloud run, functions, cloud sql, pubsub, IAM, etc. Those are all high level building blocks that are very easy to configure & deploy using pulumi. Since the devs can for example add a cloud function for a feature & go to prod entirely on their own with just one language, they make use of it, so we get better architecture as a result. In a sense the cloud platform itself becomes the full stack framework.
The cloud platform provided blocks also enforce good practice so it's harder for newcomers to shoot themselves in the foot. If we were using native k8s, self managed postgres etc, then it would probably not work as well.
But yes, pulumi is less mature & we're hitting some bugs here & there, and we're completely locked into a cloud provider. But the upsides are impressive & IMHO worth it.
I used to be a proponent of monoliths for dev speed, but now I've seen that you can match the prototyping speed with microservice/serverless & get a much better solution as a result.
We've been using Pulumi in the nitric framework https://github.com/nitrictech/nitric for a while now. Originally, we used native solutions like CloudFormation, then looked at options like Terraform. Pulumi ended up being our favorite by far. The docs are awesome and using the features of a language + editor you're already familiar with are pretty appealing. Autocomplete, typesafety, automated testing, etc. make it really productive.
They're similar in that they both have a way to build a desired state for your infrastructure and some code for executing the changes to make that happen.
In Terraform you describe the desired state using a fairly limited language called HCL which looks a lot like a config file.
In Pulumi you describe the state by running some code in a normal language and building objects to describe the state. This means you can use features of normal programming languages to abstract your state more thoroughly - stuff like functions, data structures, loops, etc.
AWS and Terraform actually have a similar thing to Pulumi called CDK which uses a normal language to build either CloudFormation or HCL. This gives a lot of the benefits of Pulumi, but lets use use a more widely used tool like Terraform underneath.
Pulumi is more dev-friendly and adapts to existing skillsets / environments.
This being said I much prefer Terraform. It's by nature far more consistent and more portable. Pulumi makes sense for individual devs having to do infrastructure tasks on the side, or « mono-lingual » (in a programming sense) teams. But your Pulumi skills in one company aren't necessarilly transferable to another.
Pulumi looks really promising. Finally we have some real IaC and not some IaData. Good job for what you guys have accomplished so far! Is there any plan to add libs for the Elastic stack?
14 comments
[ 3.1 ms ] story [ 37.8 ms ] threadI'm trying to understand why anyone would choose this over TF.
I’ve written a lot of HCL and find myself wanting to extract things into components and put in place abstractions in ways that don’t always jive with how Terraform requires you to do things. Being able to build those abstractions in my favorite language, then have that compile down into declarative IaC… that’s the draw.
I personally use Terraform CDK for this, which is Hashicorp’s answer to Pulumi.
Thanks!
We started using it in our company because we noticed that terraform as a separate language was enough of a barrier that it prevented a good deal of our developers from contributing to the infrastructure codebase, creating a separation between 'devs' and 'devops' guys. We switched to Pulumi for a new project, and now basically everybody is contributing to the infrastructure. Which completely changed the dynamic of development for the better. Frankly for us it's been a gamechanger.
I am curious because everytime I considered using it so far, the prospect of just any dev who might well not know all the intricacies of infra should maybe not build it. I guess some barrier to entry is something I would see as a benefit.
In terms of code syntax, reading infra declarations in app languages always rubbed me the wrong way. Yes, HCL is no beauty and a bit to get into, but after that, I personally love it. I know exactly what to look for. The thought of reading this in all kinds of Python syntax is rather a downer for me.
Last but not least, there is a ton of tooling which was built around TF which I am not aware of if equivalents do exist for Pulumi.
Fwiw, I have no skin in the game of either company and am not trying to paint a black or white picture of what's the better tech. Just trying to learn what appeals to others about Pulumi.
First, your dev process should go through a review phase robust enough that beginners should be able to develop on things they are unfamiliar with confidence.
Second, the infra being 'intricate', needing huge tooling etc, is a sign that it has become too complex; maybe because it is not well aligned with the application side, maybe because the 'devops' have been making an infra monolith, or maybe because it is making use of too low level features.
For example we're on GCP, and we're making use of serverless tech such as cloud run, functions, cloud sql, pubsub, IAM, etc. Those are all high level building blocks that are very easy to configure & deploy using pulumi. Since the devs can for example add a cloud function for a feature & go to prod entirely on their own with just one language, they make use of it, so we get better architecture as a result. In a sense the cloud platform itself becomes the full stack framework.
The cloud platform provided blocks also enforce good practice so it's harder for newcomers to shoot themselves in the foot. If we were using native k8s, self managed postgres etc, then it would probably not work as well.
But yes, pulumi is less mature & we're hitting some bugs here & there, and we're completely locked into a cloud provider. But the upsides are impressive & IMHO worth it.
I used to be a proponent of monoliths for dev speed, but now I've seen that you can match the prototyping speed with microservice/serverless & get a much better solution as a result.
Infrastructure definitions are complex, general purpose languages are built for this kind of stuff
https://news.ycombinator.com/item?id=22866714
We added it successfully to an existing C# codebase that was hosted as an Azure web app.
In Terraform you describe the desired state using a fairly limited language called HCL which looks a lot like a config file.
In Pulumi you describe the state by running some code in a normal language and building objects to describe the state. This means you can use features of normal programming languages to abstract your state more thoroughly - stuff like functions, data structures, loops, etc.
AWS and Terraform actually have a similar thing to Pulumi called CDK which uses a normal language to build either CloudFormation or HCL. This gives a lot of the benefits of Pulumi, but lets use use a more widely used tool like Terraform underneath.
https://www.terraform.io/cdktf (still in development)
https://aws.amazon.com/cdk/
This being said I much prefer Terraform. It's by nature far more consistent and more portable. Pulumi makes sense for individual devs having to do infrastructure tasks on the side, or « mono-lingual » (in a programming sense) teams. But your Pulumi skills in one company aren't necessarilly transferable to another.