As someone who's spent most of their career in cloud IAC, and likes to think they are pretty read up on the latest going on in that world, if you didn't know better you'd think YAML is one of the greatest threats facing mankind. There are plenty of things I certainly hate about it, but every configuration syntax I've ever used I have similar gripes about. It's like once a month this kind of "The world is growing tired of yaml" claim is just thrown out there like everyone just agrees with it. Choose something that works for you. This author repeatedly mentions TOML but there are plenty of issues with that one I could point out too. Syntax is one very small part of what makes an ecosystem great or not so great. Most of my exposure to yaml is helm chart templates, which admittedly is not pure YAML, but it works fine enough for me, at least to where I don't feel like writing lengthy blog posts about how much I hate it. I even wrote a library that converts yaml templates to HCL for internal use because I got so sick of people having this exact same argument like it deeply mattered. And guess what? They hate the HCL too.
I don't think the title and the article really communicates it's case well. Did not understand the goal until 90% through the article when they showed the source code of RCL with the loops.
This isn't syntax vs abstraction. This is how much programming language power do you want to enable in your configuration language. This is a big difference and I think we miss the interesting part of that discussion because we dip into this 'abstraction angle.
I think if people want a more powerful, programmable config language, maybe they should use something like Lua or Scheme instead of reinventing the wheel with those new niche languages.
Code as configuration loses one primary benefit which is being able to read the actual config and know exactly what will apply. In the example in the article you would get the same benefit by disallowing users to edit the config directly and instead require the config to be generated via a cmdline app/service that encodes the same policy?
The problem with configuration formats, is not syntax, or abstraction, it's the lack of consistent language server integration, it's problem when I can't lookup the definition for a key, the expected type, or quickly jump to definitions that clearly show which keys are available to configure.
To me the solution seems like it's adding complexity that could cause more issues further down the line.
The specific problems in the example could be solved by changing how the data is represented. Consider the following alternative representation, written in edn:
This prevents issues where the region is mistyped for a single bucket, makes the interval more readable by using a custom tag, and as a bonus prevents duplicate bucket names via the use of a map.
Obviously this doesn't prevent all errors, but it does prevent the specific errors that the RCL example solves, all without introducing a Turing-complete language.
the answer seems to be both of both worlds - if you're going to do for loops why not just use python?
the answer is both a faux programming language, and really bad ux / really hard to read / scan
maybe what they need is a program that generates better readable text; and somehow you can flip between the determinism of code and ux of readable text?!
The intention behind configuration languages is sound but there’s just too many of them. Not having a universally accepted one makes picking one harder. Also, migrating from one to another isn’t as straightforward. Plus,
for database in ["alpha", "bravo"]:
for period, days in period_retention_days:
At this point, you’re better off writing a Python script that spits out some JSON. I‘m aware that even the blog mentions it. The benefit is - having a more expressive language at your fingertips and not having to fight your peers while trying to add yet another idiosyncratic dependency.
Somewhere along the way we got lost into thinking config files were remote procedure calls, and that YAML was the only RPC interface available to us. The caller generates YAML, the receiver parses it, but more often than not the two are on the same host and use the same language*:
def say(message):
o = dict(f=say, v=message))
call(yaml.dumps(o))
…meanwhile elsewhere…
def eval(request):
o = yaml.loads(request.body)
match o[“f”]:
case “say”:
print(o[“v”])
Yes, referring to this as “madness” skips over the fact that you can now scale up your hello world printer across the internet, have auth, rate limit, etc etc. but for so many things the RPC just isn’t needed at all.
As this article gets at, config files are one of those things, and they benefit hugely from the rampant abstraction violation of skipping an intermediary text format between the program doing the configuration and the program doing the work.
*or have internal versions of their APIs available in matching languages.
I got confused reading this because I wasn't sure how I, as a reader with no knowledge of the system under discussion, was supposed to know that all the buckets should be in the same region.
Nor is it clear to me how the "for loop" version would handle the case where exceptionally one bucket is different. Which is a more interesting discussion imho, that's the whole point of having it as a configuration field, after all.
Before going all in with turing-complete configuration languages, why not simply augmenting existing declarative formats with parameter & arithmetic expansion à la bash? It's a syntax already familiar to many, that would mitigate the issues highlighted in TFA.
Yeah no. The reason we don't write code in config files is that, if we did, in order to know what you are deploying you have to run the code in your head. Not just this code, the code that Steve wrote six months ago before people noticed he was grossly incompetent and fired. Also worth mentioning, I can run this code in my head. The author can run this code in their head, but not everyone can. And finally, if the author thinks copy pasting yaml files cause bugs, wait till the llms start copy pasting the for loops.
Big fan of HCL as the configuration language to rule them all, being able to abstract stuff into reusable modules at the configurations language level is great. And there are implementation for HCL in multiple programming languages.
A "universal language" is a language with (a) sequences of instructions, (b) conditionals, ifs, branches, and (c) loops, repetition, iteration, y-combinator (the math one), recursion.
They are Turing complete. Leaving off any of the three features (difficult for the y-combinator) yields a non-Turing complete language, for example Sieve Script for filtering email lacks (c).
What the definition doesn’t cover is parameterization. Some people call this abstraction. Technically, lisp macros also fall under parameterization.
With parameterization it really seems like any and all functions, mappings, and operators that fail to be injective and composable are counter-productive in practice. The math term is "generative effects".
There seems to be this continuous contention between one side that wants its configuration files to basically be CSV and another side that, effectively, wants a full-blown programming language as their "configuration language".
Both sides have a point. Just happen to land in the "essentially CSV" camp myself. Even a lisp macro can generate CSV.
(Yes the phrase "universal language" is very difficult to search for if the specific academic term above is under consideration.)
When Kubernetes exposed its APIs as declarative configuration objects, I get the impression that they didn't originally mean for people to write the configuration by hand. The YAML/JSON/… is a conveniently universal interchange format for interfacing with Kubernetes from bindings, and representing target state as documents is just a good way to encode idempotence in the API.
I'd be interested to hear from someone involved with early Borg/K8s development what the original intention was.
25 comments
[ 5.5 ms ] story [ 47.6 ms ] threadThis isn't syntax vs abstraction. This is how much programming language power do you want to enable in your configuration language. This is a big difference and I think we miss the interesting part of that discussion because we dip into this 'abstraction angle.
the article talks about the trade off between plain data structure versus abstract ones and that’s the main issue
If you want configuration-as-code use Python. Please. Or Tcl if you must. Do not invent N+1 DSL for your engineers to waste time learning.
The specific problems in the example could be solved by changing how the data is represented. Consider the following alternative representation, written in edn:
This prevents issues where the region is mistyped for a single bucket, makes the interval more readable by using a custom tag, and as a bonus prevents duplicate bucket names via the use of a map.Obviously this doesn't prevent all errors, but it does prevent the specific errors that the RCL example solves, all without introducing a Turing-complete language.
the answer is both a faux programming language, and really bad ux / really hard to read / scan
maybe what they need is a program that generates better readable text; and somehow you can flip between the determinism of code and ux of readable text?!
(is that possible)
YAML has a merge key <<:, which might be helpful.
The merge key is a clever little trick, but it depends of the special hash key, so lists can’t be merged.
Syntax does matter, which is why YAML matters — even if imperfect.
Doesn't help every yaml parser has their own opinion on what a merge or an anchor should do, exactly.
[1]: https://news.ycombinator.com/item?id=45291858
As this article gets at, config files are one of those things, and they benefit hugely from the rampant abstraction violation of skipping an intermediary text format between the program doing the configuration and the program doing the work.
*or have internal versions of their APIs available in matching languages.
Nor is it clear to me how the "for loop" version would handle the case where exceptionally one bucket is different. Which is a more interesting discussion imho, that's the whole point of having it as a configuration field, after all.
They are Turing complete. Leaving off any of the three features (difficult for the y-combinator) yields a non-Turing complete language, for example Sieve Script for filtering email lacks (c).
What the definition doesn’t cover is parameterization. Some people call this abstraction. Technically, lisp macros also fall under parameterization.
With parameterization it really seems like any and all functions, mappings, and operators that fail to be injective and composable are counter-productive in practice. The math term is "generative effects".
There seems to be this continuous contention between one side that wants its configuration files to basically be CSV and another side that, effectively, wants a full-blown programming language as their "configuration language".
Both sides have a point. Just happen to land in the "essentially CSV" camp myself. Even a lisp macro can generate CSV.
(Yes the phrase "universal language" is very difficult to search for if the specific academic term above is under consideration.)
I'd be interested to hear from someone involved with early Borg/K8s development what the original intention was.