Really bummed that YAML managed to sink its teeth into the IaC community as badly as it has. I know we have a hard time agreeing on programming language preferences in general, but this has to be the worst case of fallacy of moderation that we're experiencing in modern times. Like -- this simple interaction is spread across 5 files with no explicit import system?
IMO, I understand the desire for a truly Turing complete language, however, myself, as a devops, after the horror stories I've seen with high level tooling such as Ansible, Helm, Kustomize or Terraform, I just don't want to imagine the hell that we could have if operations people was using a proper programing language with all the complexity and design decisions that it implies.
Honestly, I write pulumi in typescript for a lot of the things I used to use terraform for. I don't think it's perfect -- the goal of a "pulumi program" (what they call it) is to generate a dependency graph that is then executed by the engine. The output dependency graph is basically what I used to write in terraform, but not in HCL. I think that having access to a full programming language is less bad than people assume it will be if you just give access to a standard library that provides resources they might write poorly (pulumi used open source terraform providers to bootstrap their codebase). With typescript, I get access to modules, familiar testing libraries, normal module loaders, normal dependency upgrade notices, etc. The downside is I'm tied to the declarative configuration engine (in this case pulumi). If we so chose, we could write modules in multiple languages and coexist the codebases in an org (right now pulumi supports python, go, c# and type/javascript)
I think you accurately picked out a problem with using purely declarative data languages -- many of them adopt or bolt on rudimentary templating languages that are inferior, bespoke and often just as turing-complete as a normal language and they're terrible. HCL managed to go pretty far with just "count" but now in HCL2 they have looping structures.
So: what is ideal? I think there are several declarative functional languages which are starting to look really good for the space, albeit a bit early -- Cue, Dhall and nix. I think all of them come with their own baggage which is why it's probably good they weren't used, but I think the point is effort could have been made to adopt them rather than dump an untyped blob structure on users.
There are a few issues I take with YAML as the de-facto language used:
* It's dangerous for capturing types it supports accurately without devolving to JSON+, especially for a "human focused" data language. It's very easy to get into undefined or mal-defined areas that are trying to be syntactic sugar.
* It presents no concept of schema or type safety to its users without being hoisted into a tool, which means all validations are implementation specific, if they exist.
* It's incredibly verbose for duplicated items, which makes it really prone to typos and hard to spot mistakes.
I think strong gitops operators are already using tooling to generate yaml for them so it's basically an intermediate representation anyways. Languages like cue and dhall embed a concept of schema validation, but if we want to use our own external tools to generate it, why don't we just say "you know what, if you want to manually write it, it's JSON, and here's a schema you can validate against" but if you want to use tools to generate it - every major language has a *correct* JSON parser and there's already a well established schema specification for generating types, with libraries available.
I think the mistake was encouraging people to write the intermediate representation manually and then letting them find a way to bolt real language features on top.
You could convert the examples to JSON and it'd still work. Kubernetes supports multiple serialization formats. You could also use your favorite programming language with the k8s SDK to use structs/objects
Oh no, there's a Flux2 already? I think I am gonna give up on Kubernetes because every time I a wanna research a specific project/problem I encounter at least three new projects I wasn't aware of. Kind of like the JS framework world.
10 comments
[ 2.4 ms ] story [ 45.5 ms ] threadOr maybe I'm missing something, and this really is the simplest way to flush a cache on update.
IMO, I understand the desire for a truly Turing complete language, however, myself, as a devops, after the horror stories I've seen with high level tooling such as Ansible, Helm, Kustomize or Terraform, I just don't want to imagine the hell that we could have if operations people was using a proper programing language with all the complexity and design decisions that it implies.
I think you accurately picked out a problem with using purely declarative data languages -- many of them adopt or bolt on rudimentary templating languages that are inferior, bespoke and often just as turing-complete as a normal language and they're terrible. HCL managed to go pretty far with just "count" but now in HCL2 they have looping structures.
So: what is ideal? I think there are several declarative functional languages which are starting to look really good for the space, albeit a bit early -- Cue, Dhall and nix. I think all of them come with their own baggage which is why it's probably good they weren't used, but I think the point is effort could have been made to adopt them rather than dump an untyped blob structure on users.
There are a few issues I take with YAML as the de-facto language used:
* It's dangerous for capturing types it supports accurately without devolving to JSON+, especially for a "human focused" data language. It's very easy to get into undefined or mal-defined areas that are trying to be syntactic sugar.
* It presents no concept of schema or type safety to its users without being hoisted into a tool, which means all validations are implementation specific, if they exist.
* It's incredibly verbose for duplicated items, which makes it really prone to typos and hard to spot mistakes.
I think strong gitops operators are already using tooling to generate yaml for them so it's basically an intermediate representation anyways. Languages like cue and dhall embed a concept of schema validation, but if we want to use our own external tools to generate it, why don't we just say "you know what, if you want to manually write it, it's JSON, and here's a schema you can validate against" but if you want to use tools to generate it - every major language has a *correct* JSON parser and there's already a well established schema specification for generating types, with libraries available.
I think the mistake was encouraging people to write the intermediate representation manually and then letting them find a way to bolt real language features on top.
Flux and its successor flux2 are really great tools everyone working with kubernetes should check out!
https://github.com/fluxcd/flux2