The infrastructure management space is a special case but let’s approach it with some common sense. If I know bash I can script anything. If I know puppet and bash I can script anything with much more work. Most fundamentally, I still need to know how to use bash! Puppet, chef, vagrant, dockerfiles and all the above subtract value and don’t add. Then there are systems like cloud formation which exist for one reason: lock you into a vendor. If that is how you define DSL then DSLs fail but it is not about DSLs it is “why do people keep making bad infrastructure management tools and why do people have the bad judgement to use bad infrastructure management tools?”
Matlab is a DSL that has made its creators a lot of money and which has been adopted widely in engineering and finance. It may eventually be pushed out by Python/Numpy or Julia, but it has been very successful.
The lisp LOOP macro is an excellent example of a DSL done right, in my opinion. It does exactly what you'd expect and allows you to write English sentences to solve problems.
I know that there are factions of lispers who are against it but I use it constantly.
In place of iterate, as tagbody still has a very specific place in the arsenal that loop doesn't fully solve. Or maybe it does and I'm missing something. I'm probably missing something.
Absolutely agreed that iterate would classify as a DSL, too. It's just that I prefer exploratory code to read like English as it forces me reason about what I'm trying to do. But honestly I haven't used iterate as much as I perhaps should have, I might need to change that.
A lot of DSLs are quite successful, but they need to be worth it and require a lot of effort into making them more usable than a simple library for general purpose language, or even just a simple JSON configuration file. A lot of DSLs just aren't worth it.
> It’s why I think TypeScript is the language of infrastructure-as-code.
What? Since when?
Edit: Did a google search for "infrastructure as code typescript" and this persons blog is one of the first results, where they talk about the new cloud development kits which are made to support different languages out of the box....
I don't think DSL's necessarily fail. Puppet's DSL was everywhere at one point in the late 00's and lots of people argue that Terraform's DSL is fairly ubiquitous in the IaC space.
The article brings up some fine points about why DSL's are restrictive, but the primary issue with them comes from the name: They're single use.
I meet with companies daily who have tried to adopt Terraform's DSL internally and found a few people feel comfortable with it, but the vast majority of the engineering org have zero interest in learning it because to them - what's the point? They can achieve the same goal with <insert language here> by using Pulumi or CDK. People will bring the flawed argument that Terraform's DSL stops you from footgunning yourself but I've seen plenty of awful, messy Terraform that's only maintainable by the person who wrote it.
Another consideration is just how much DSLs evolve over time. You eventually end up getting closer and closer to Turing completeness because people really want to solve problems that require it or solve them in a familiar way.
I've spoken frequently about my belief that in the long term, DSLs in the IaC space will largely be superseded by programming language based graph building and most of these thoughts come from frustrated users who are moving away from DSLs
It's amazing how easy it is to do this stuff with conventional programming languages.
I wrote a program that built AWS Marketplace images that involved installing a lot of software on a VM, setting up all sorts of resources, building a database for two hours, snapshotting it, etc.
This was just a java program that wrote a bash script that did all the work on the remove machine which notified the java program via SQS which then make the image.
Since I took the path less taken I got it done quick and if I had to migrate to Azure... easy.
If I did it the way most people do it I'd build some Rube Goldberg machine with IaaS that works sometimes and think it was normal.
Not really surprising that general purpose language that is is targeted at general computing are good at general computing.
Puppet was targeted at eventually consistent idempotent configuration changes, mostly as a restricted grammar for a particular use case when distributed clustered systems with shared state were rare.
It started to fail once people needed orchestrated stateful changes based on remote state. A task it was never written for.
TF's DSL was designed for configuration changes that were ordered and thus to avoid decidability problems enforces a tree like structure on changes exactly how JSON does to avoid cycles, race conditions etc...
Except for some who used CFEngine which was also focused around idempotent changes, most of us in the 90's used sed,awk,grep,bash and perl for configuration management.
But because we were working on fragile monoliths with limited separation of concerns and modularity....mixed in with freeform code writing that would allow for race conditions, cycles and other issues....the industry decided to limit operations to idempotent tasks.
I hand rolled my own OpenStack management code and it was pretty good.
But then we couldn't hire people to help us scale that and the company decided to go with Mirantis.
Mirantis obviously decided to use Puppet because they liked it or were familiar with it without considering the use case of orchestrating cluster components and Puppet (and chef) was also co-opting meaningful terms like orchestration and rolling out features that really weren't compatible with there original design decisions.
While the OpenStack had their own challenges with stability and growing pains, anyone who was a Mirantis customer had very very painful experiences due to them choosing an inappropriate tool, which was actually exceptional for it's original use case.
Mirantis's poor design decisions really played a large part in killing most of what was a potential hybrid cloud market, mixed in with companies buying expensive enterprise hardware and SANs.
The problem wasn't the DSL. The problem was not making design decisions based on the need of the problem at hand.
There is no problem with most DSLs if they are used within the limits of the design choices that were made.
If you can't move your systems deployment model to work with TF because you have too much coupling to describe a deployment in a tree...generalized computing languages will not solve your deployment problems because bad architectural decisions were made.
A general purpose Turing complete language won't change the fact that being recursively enumerable is required for problems to be semi-decidable.
To be clear, I don't view TF or any other existing solution to be perfect.
But after years of of helping people move to the cloud, almost universally the problems I see are people either trying to use the wrong product for their need or refusing to adapt there way of working to fit the limitations of the model they chose...and typically chose due to popularity vs the fitness for purpose.
As with most things in our industry, the products aren't the problem, the culture is the problem.
His DSLs fail because he doesn't know how to use DSLs. There must be one for each level of abstraction for each little problem, all built on top of a single host language so that they can be intermingled and COMPOSED.
17 comments
[ 3.1 ms ] story [ 44.4 ms ] threadThe infrastructure management space is a special case but let’s approach it with some common sense. If I know bash I can script anything. If I know puppet and bash I can script anything with much more work. Most fundamentally, I still need to know how to use bash! Puppet, chef, vagrant, dockerfiles and all the above subtract value and don’t add. Then there are systems like cloud formation which exist for one reason: lock you into a vendor. If that is how you define DSL then DSLs fail but it is not about DSLs it is “why do people keep making bad infrastructure management tools and why do people have the bad judgement to use bad infrastructure management tools?”
Who said that "DSLs Fail"?
There are tons of highly succesfully DSLs
I know that there are factions of lispers who are against it but I use it constantly.
IMO iterate is no less a DSL (it's a code-walker!), just one that has a more sexpry flavor
Absolutely agreed that iterate would classify as a DSL, too. It's just that I prefer exploratory code to read like English as it forces me reason about what I'm trying to do. But honestly I haven't used iterate as much as I perhaps should have, I might need to change that.
What? Since when?
Edit: Did a google search for "infrastructure as code typescript" and this persons blog is one of the first results, where they talk about the new cloud development kits which are made to support different languages out of the box....
https://i.imgur.com/jozV24f.png
The article brings up some fine points about why DSL's are restrictive, but the primary issue with them comes from the name: They're single use.
I meet with companies daily who have tried to adopt Terraform's DSL internally and found a few people feel comfortable with it, but the vast majority of the engineering org have zero interest in learning it because to them - what's the point? They can achieve the same goal with <insert language here> by using Pulumi or CDK. People will bring the flawed argument that Terraform's DSL stops you from footgunning yourself but I've seen plenty of awful, messy Terraform that's only maintainable by the person who wrote it.
Another consideration is just how much DSLs evolve over time. You eventually end up getting closer and closer to Turing completeness because people really want to solve problems that require it or solve them in a familiar way.
I've spoken frequently about my belief that in the long term, DSLs in the IaC space will largely be superseded by programming language based graph building and most of these thoughts come from frustrated users who are moving away from DSLs
I wrote a program that built AWS Marketplace images that involved installing a lot of software on a VM, setting up all sorts of resources, building a database for two hours, snapshotting it, etc.
This was just a java program that wrote a bash script that did all the work on the remove machine which notified the java program via SQS which then make the image.
Since I took the path less taken I got it done quick and if I had to migrate to Azure... easy.
If I did it the way most people do it I'd build some Rube Goldberg machine with IaaS that works sometimes and think it was normal.
Puppet was targeted at eventually consistent idempotent configuration changes, mostly as a restricted grammar for a particular use case when distributed clustered systems with shared state were rare.
It started to fail once people needed orchestrated stateful changes based on remote state. A task it was never written for.
TF's DSL was designed for configuration changes that were ordered and thus to avoid decidability problems enforces a tree like structure on changes exactly how JSON does to avoid cycles, race conditions etc...
Except for some who used CFEngine which was also focused around idempotent changes, most of us in the 90's used sed,awk,grep,bash and perl for configuration management.
But because we were working on fragile monoliths with limited separation of concerns and modularity....mixed in with freeform code writing that would allow for race conditions, cycles and other issues....the industry decided to limit operations to idempotent tasks.
I hand rolled my own OpenStack management code and it was pretty good.
But then we couldn't hire people to help us scale that and the company decided to go with Mirantis.
Mirantis obviously decided to use Puppet because they liked it or were familiar with it without considering the use case of orchestrating cluster components and Puppet (and chef) was also co-opting meaningful terms like orchestration and rolling out features that really weren't compatible with there original design decisions.
While the OpenStack had their own challenges with stability and growing pains, anyone who was a Mirantis customer had very very painful experiences due to them choosing an inappropriate tool, which was actually exceptional for it's original use case.
Mirantis's poor design decisions really played a large part in killing most of what was a potential hybrid cloud market, mixed in with companies buying expensive enterprise hardware and SANs.
The problem wasn't the DSL. The problem was not making design decisions based on the need of the problem at hand.
There is no problem with most DSLs if they are used within the limits of the design choices that were made.
If you can't move your systems deployment model to work with TF because you have too much coupling to describe a deployment in a tree...generalized computing languages will not solve your deployment problems because bad architectural decisions were made.
A general purpose Turing complete language won't change the fact that being recursively enumerable is required for problems to be semi-decidable.
To be clear, I don't view TF or any other existing solution to be perfect.
But after years of of helping people move to the cloud, almost universally the problems I see are people either trying to use the wrong product for their need or refusing to adapt there way of working to fit the limitations of the model they chose...and typically chose due to popularity vs the fitness for purpose.
As with most things in our industry, the products aren't the problem, the culture is the problem.