So long as you use only string keys you can completely replace YAML with JSON (i.e. you can write JSON to a YAML file). Just make a .d.ts for your config schema and you're done.
Avoiding YAML is this way has saved my sanity on more than one occasion.
People like YAML because it doesn’t require double quotes in the simple case of adding some text to a key. And people don’t like adding trailing commas to array items.
The problem is the insanity of parsing values like “true” or “off” or “123”.
IME, YAML appeals to people who like Ruby. I find both of them a nightmare, but it's clear that there is a kind of mine quite different to mine that sees something in them that i don't.
I am baffled that we as an industry landed on the concept of „code over configuration“, then went on to choose text based formats like yaml or json and consider it perfectly normal to write these by hand? Or even worse, we use bastardized in-file template strings to generate hopefully valid YAML in the end.
What‘s wrong with configuring our platforms with a real, perfectly working programming language? I think the author had a great point with the schema being disconnected from the configuration file, but he missed the big picture with JSX: JavaScript (or TypeScript) can be perfectly used to emit JSON. JS tooling works with .config.js files.
Why must I experience the living hell of helm charts and kubernetes configuration file templates, when I could just emit my config with my favorite programming language?
This pattern keeps repeating because the creators of an app think they need a configuration language (which some users do) and then when eventually other users end up needing an API or some feature the config language doesnt have. They both dont exist so they hack it with templating.
It's often quite hard to know what kind of problems need an API and what kind of problems need configuration. It's sometimes only obvious in retrospect - the same point when you also get held back from changing things by backwards compatibility and culture.
The reverse problem (no configuration, only turing complete code) carries its own set of problems and can be just as infuriating. Getting the balance just right is hard and often only clear in retrospect.
Kubernetes really should have done something about the templating horrors by now though.
> What‘s wrong with configuring our platforms with a real, perfectly working programming language?
Python's "you must execute arbitrary code to find out a packages dependencies", and the hell that makes working with their packaging ecosystem as a whole, is what's wrong with using a general purpose programming language for configuration (not that I'm a fan of YAML either)
To be fair, that’s because Python packaging was “invented”/formalised a long time ago, before it was generally considered a bad idea. And a lot of the rationale for this was because you wanted to select specific dependencies for specific platforms, and there wasn’t a proper way to do this.
But now, almost all packages are specified using plain text files. You still need to execute code to compile a sdist that has native code, but that’s a separate thing.
I've worked with frameworks that required users to use a programming language to specify configuration. It was a nightmare to debug, especially if the application would instantiate many different things in response to such completely free configurations.
Every user reporting a problem incurs in a lot of cognitive load for you because the user has the freedom to go crazy with a real programming language.. or not but it's just different enough from the previous bug report that you can't generalize your troubleshooting.
I think our best hope right now is to have jsonnet be more widely adopted.
This is not a problem if the configuration script just emits a valid JSON object according to your defined schema. On bug reports, your users send the generated config and you don‘t deal with the specifics.
Because that solution would be just your favourite language and the next person would create documentation using your least favourite language, next person would optimise the internals to be easier to work with their choice, next person would answer on SO using their choice, etc.
Nobody stops you from creating the configs right now using your own serialiser, but having a common understanding of what's being read by the app and mostly talking about that format helps the ecosystem.
Maybe a silly question, but if you have a programming language available why can’t you buld a native structure, validate it and serialise it to YAML/JSON/whatever completely skipping templating part?
This seems entirely reasonable to me. I built an application which gets used by the network team and consumes a JSON config file. The team edits a simple YAML file which is then heavily processed/checked and emitted as JSON.
I’ve been completely explicit that if there is some other language or process they want to use then I’m more than happy to assist.
The most useful construct I am aware of that I wish became more popular in backend languages is strongly typed tree literals. (Introduced by Meta in Hack as "XHP")
I read it if only to glean why anyone would do this. I learned nothing of the sort of person who would use Javascript to structure a YAML config.
10 years ago this article would have been "Used PHP for structured YAML config" and there would have been a minor riot of people poking fun at the article writer for attempting such sillyness. (And I say this someone who still made my money on PHP 10 years ago).
We got soft somewhere along the way and stopped calling out silliness.
30 comments
[ 0.29 ms ] story [ 45.1 ms ] threadnindent makes me yell especially.
Avoiding YAML is this way has saved my sanity on more than one occasion.
People like YAML because it doesn’t require double quotes in the simple case of adding some text to a key. And people don’t like adding trailing commas to array items.
The problem is the insanity of parsing values like “true” or “off” or “123”.
“template”
multiline string values back into valid yaml for proper indenting.
Works great, the issue is explaining to anybody why it was necessary, they understand it works but plead for a better way.
It’s one of those clever solutions that seems overcomplicated until you try to replace it and realize it’s actually quite elegant.
What‘s wrong with configuring our platforms with a real, perfectly working programming language? I think the author had a great point with the schema being disconnected from the configuration file, but he missed the big picture with JSX: JavaScript (or TypeScript) can be perfectly used to emit JSON. JS tooling works with .config.js files.
Why must I experience the living hell of helm charts and kubernetes configuration file templates, when I could just emit my config with my favorite programming language?
It's often quite hard to know what kind of problems need an API and what kind of problems need configuration. It's sometimes only obvious in retrospect - the same point when you also get held back from changing things by backwards compatibility and culture.
The reverse problem (no configuration, only turing complete code) carries its own set of problems and can be just as infuriating. Getting the balance just right is hard and often only clear in retrospect.
Kubernetes really should have done something about the templating horrors by now though.
Python's "you must execute arbitrary code to find out a packages dependencies", and the hell that makes working with their packaging ecosystem as a whole, is what's wrong with using a general purpose programming language for configuration (not that I'm a fan of YAML either)
But now, almost all packages are specified using plain text files. You still need to execute code to compile a sdist that has native code, but that’s a separate thing.
Candidly I think python is going to be forever stunted after the python 2 to python 3 multi year fiasco.
Every user reporting a problem incurs in a lot of cognitive load for you because the user has the freedom to go crazy with a real programming language.. or not but it's just different enough from the previous bug report that you can't generalize your troubleshooting.
I think our best hope right now is to have jsonnet be more widely adopted.
Good luck with that. You'll be teaching your users (especially the paying ones) how to use the language.
Nobody stops you from creating the configs right now using your own serialiser, but having a common understanding of what's being read by the app and mostly talking about that format helps the ecosystem.
I’ve been completely explicit that if there is some other language or process they want to use then I’m more than happy to assist.
Consider this pseudocode expression:
:Div{Hello :Span(style='bold'){world}} # expression type: HTMLDOMNode.Div
This is immensely powerful and saves a ton of time/bugs especially when compared to the alternative of string interpolation.
I know somebody once did XHP on Python, but regrettably this work was abandoned.
I read it if only to glean why anyone would do this. I learned nothing of the sort of person who would use Javascript to structure a YAML config.
10 years ago this article would have been "Used PHP for structured YAML config" and there would have been a minor riot of people poking fun at the article writer for attempting such sillyness. (And I say this someone who still made my money on PHP 10 years ago).
We got soft somewhere along the way and stopped calling out silliness.