Very interesting (and opinionated) overview of the existing configuration languages.
At the very bottom, author is presenting its own language (maml.dev). It is yet another JSON with comments, multiline strings, and optional commas. It's the most boring part of the page.
I don't like this article all that much but I do like the MAML format the author wrote this to promote. It seems like it’s everything that’s good about JSON, plus everything fixed for human writing, and nothing more.
Hats off, wish my package.json and build.yml were package.maml and build.maml.
It’s not a sufficient improvement over JSON for it to be worthwhile.
The configuration languages that are truly worthwhile are the ones that the author deride as being too similar to programming languages, like Jsonnet, Starlark, CUE, and Dhall (in increasing order of worthwhileness).
Shallow. Especially “ The era of XML is in the past… now it's dead.”
That’s not an argument against the merits of XML, it’s just a fashion declaration.
It’s also wrong. Podcasting boils down to XML files. It even heavily uses XML’s extension mechanism. XML is the basis for RSS and Atom. XML is the basis for LibreOffice and modern Microsoft Office files. XML is at the core of the epub digital book standard. And on and on.
XML may not be ideal for config (reasonable people can disagree on the topic) but it’s not dead.
It’s also interesting that he declares JSON “won” then adds a bunch of XML features to it like ordered entries and not having to quote element/attribute names.
JSON “won” for web apis - e.g. browser-server data interchange. It is not, and was never claimed to be, good for serial documents like config, or for when you need an extensible format (the “X” in XML). It’s fine for what it is, and so is XML, which, when it “won”, was similarly overused, like for web apis.
I kind of wish it had two and only two features: defined ordering, and comments
As to languages for configuration (specifically), I wish there was a good specification for a program config, where you would have one well-defined configuration file for defaults, and a user-specified configuration file for the values you have set/overriden.
say default.config:
{"player_name":"player", "mouse_speed":5}
and my.config:
{"player_name":"me"}
in some way that upgrading the program could do stuff with default.config without destroying your existing config.
Just some opinions with no serious arguments. Okay XML is in the past but so what? There are still use cases for it. And Pkl/CUE/Dhall/Starlark looking like full-blown programming languages is a feature, but being deliberately weaker than say TypeScript is also a feature (no arbitrary I/O, in some cases no Turing completeness). I don’t think the author understands or appreciates why.
> Well, this is some sort of programming language with dynamic types. But there are so many good programming languages, so I don't know why this one needs to be used.
RE: Jsonnet and others: because it has nice guarantees, like lack of arbitrary I/O and pure execution.
In the world of human-readable data formats (ie not programming languages), the best one I ever used was Jane Street's sexplib[1] s-expression format.
It was concise and expressive. There was a direct way to describe variants (types with multiple constructors), which is always awkward in JSON, but the format was still surprisingly low-noise for reading and editing by hand. I remember you could even use it as a lightweight markup format:
(here is some text with (em formatting) information)
(The format leaves the interpretation of things like (em ...) up to you; you could use it as a slightly more verbose Markdown, but you could also use it to structure readable text with other sorts of metadata instead.)
And, unlike certain other formats I won't name, it has comments!
It also helps that Emacs with Paredit makes editing s-expressions flow. The tool doesn't need to know anything about the sexplib format specifically; just relying on basic s-expression structure gives us fluid but simple structural editing.
I am constantly sad that nobody else uses this sort of format, and I have to deal with a mixture of JSON, YAML, TOML and other ad-hoc formats instead.
I agree that YAML has its issues. But saying it's bad because it has too many features seems misguided. Do those advanced features get in the way of using a simple YAML file? At least not in my experience.
No mention of The Configuration Complexity Clock [1] which I always think deserves a link, but credit to the author for actually keeping it slim and readable over (IMHO) most of the newer additions to the landscape which either declare themselves as 'obvious' and aren't or just add features such as Pkl's extends keyword, moving us further round the complexity clock.
Of course just adding multiline strings is the start of the rabbit hole, now you need to think about leading line breaks, trailing line breaks, intermediate line breaks, whitespace chomping, and- oh heavens I've reinvented YAML, I think I need to lie down.
Nice article. I've definitely moved the clock to 6.
And I remember toying with the idea of 9 but I think for a different purpose, I don't remember what. Never did get that far though, I'm acutely aware that tooling is also important, but it's still oh-so-tempting.
I wonder where the company is at now. I don't think they would have moved the clock ahead but they may have moved it backwards to 12.
I don't regret my little rules engine though. It held up very well. It was intended for non-devs to be able to use, but the article is quite right, a complex set up was not easy to understand! I often got questions when things looked wrong but after much thinking the rules were usually calculating things correctly.
The alternative would have been 100 hardcodings because each client needed something different.
There is also .ini, it's simpler than toml, though has more inconsistent implementations and formats. But the '[[table]]' is just '[table]' and I like the '[table.subtable]' syntax better.
But since we're criticizing formats ;-) maml https://maml.dev is fairly good good.
* I like multiline blocks and comment style
* I do not like optional commas, either require them or not. I'd lean toward requiring them.
* Not sure about ordered object keys. How should languages represent that, a map/dict kind of a data structure won't work it would have to be an ordered dict/list of kvs etc. Two objects {"a":"x", "b":"y"} and {"b":"y", "a":"x"} will be different then. I get the idea, but I am leaning toward not liking it by the time I finished typing all that.
* I like booleans and null. Good for not having ons and offs those are just annoying.
About the ordered keys, most good languages have an ordered map type, either by accident or by design. Those few languages that don’t would need a list of key-value pairs. Given that this is for config, the performance cost of linear-scanning such a list would be negligible in realistic use cases.
Personally I think it’s a very good decision. Simply by declaring that objects are ordered, it becomes the case and it unlocks a whole lot of useful mechanisms.
I wonder if "the right solution" is a programming language that is fast, concise, trivially easy to run, and outputs some efficient binary format like protobuf.
Programming languages have comments and control flow, multiple popular implementations, and can have nice literals. Lack of Turing completeness is actually not a terribly useful feature if you trust the input (and you should probably just use protobufs or similar for untrusted inputs in that case.)
The author seems to be using "markup language" as a concept basically synonymous with a configuration language, or something that is not a programming language. A markup language is a language used to "mark up" text with formatting and structure. This may sound like a terminology nitpick, but I would argue the reason why for example XML is not a great configuration language is that it was designed to be something else – a markup language.
XML is a fantastic configuration language. Its primary purpose is text markup, right, but it can really shine as a configuration language too. It is meant to be used as a language, that is to describe structures of unbounded complexity while keeping them syntactically constrained.
The best configuration language is the one that is custom made for a specific application. A domain-specific language. But DSLs requires a parser and adding one is usually too cumbersome to merely parse a configuration. This is where XML comes in.
Personally I have embraced using sqlite as my configuration infrastructure, as the more I've used configuration formats, the more I've felt they are glorified databases.
The perfect configuration system would be able to support any number of parameters, enforce arbitrary validation rules, be searchable, provide metadata, examples, descriptions for each parameter, provide an audit log of who/why something was changed, support rollbacks, and so on.
The problem is that folks want a flat human readable file format to solve all these problems. That's a pipe dream.
JSON and CSV are pretty close to "syntactically optimal" in terms of losslessly storing nested and tabular data, respectively, in a human readable format. We ought to just stick with those and think about how to design effective configuration systems on top of them.
CSV has too many features. I recently found out the hard way that line breaks are allowed inside values, so you can't actually read CSV line-by-line.
Or maybe not enough features, because there's no other way to define a linebreak. \n isn't a thing.
And also, some editors are picky about the number of commas on each line. Excel and Sheets don't care if you have uneven number of commas, but IntelliJ's CSV viewer I think just looks at the first line to determine how many columns there ought to be.
Need to configure 5 services with hundreds of replicas in 7 data centers? Some values depend on the service, some on the data center and some on the combination thereof? Maybe also overrides for a bunch of problemstic machines?
And you also want a manageable config language which doesn't turn into a full blown Turing tar pit?
Then jsonnet is for you.
So it's not entirely fair to compare it in the "pleasant syntax" contest. It's like putting a Unimog into a ranking of city cars.
I've often wondered if PostScript or PDF contained the roots of a very good config language. Perhaps it simply is (PDF docs at least) but nobody regards it as such.
My guess is the RPN nature would be a no-go for many people. Nevertheless: comments, dicts, arrays, good string syntax, numerics, binary data, etc. Maybe that makes it too complicated.
43 comments
[ 2.6 ms ] story [ 63.2 ms ] threadAt the very bottom, author is presenting its own language (maml.dev). It is yet another JSON with comments, multiline strings, and optional commas. It's the most boring part of the page.
Hats off, wish my package.json and build.yml were package.maml and build.maml.
The configuration languages that are truly worthwhile are the ones that the author deride as being too similar to programming languages, like Jsonnet, Starlark, CUE, and Dhall (in increasing order of worthwhileness).
All config formats are bad. You either don't need all the features at all, except key:value. Or you quickly run into weird limitations and quirks.
What I rather like instead, are custom build english-like DSL's, like:
- https://man.openbsd.org/pf.conf - https://man.openbsd.org/smtpd.conf - https://man.openbsd.org/httpd.conf - … and many more
That’s not an argument against the merits of XML, it’s just a fashion declaration.
It’s also wrong. Podcasting boils down to XML files. It even heavily uses XML’s extension mechanism. XML is the basis for RSS and Atom. XML is the basis for LibreOffice and modern Microsoft Office files. XML is at the core of the epub digital book standard. And on and on.
XML may not be ideal for config (reasonable people can disagree on the topic) but it’s not dead.
It’s also interesting that he declares JSON “won” then adds a bunch of XML features to it like ordered entries and not having to quote element/attribute names.
JSON “won” for web apis - e.g. browser-server data interchange. It is not, and was never claimed to be, good for serial documents like config, or for when you need an extensible format (the “X” in XML). It’s fine for what it is, and so is XML, which, when it “won”, was similarly overused, like for web apis.
I kind of wish it had two and only two features: defined ordering, and comments
As to languages for configuration (specifically), I wish there was a good specification for a program config, where you would have one well-defined configuration file for defaults, and a user-specified configuration file for the values you have set/overriden.
say default.config:
and my.config: in some way that upgrading the program could do stuff with default.config without destroying your existing config.I rely on it a tiny bit in PHP. In JS, if order matters, I just use an array, and it's really never been a big deal.
Maps/dicts should be unordered IMO.
meh.
RE: Jsonnet and others: because it has nice guarantees, like lack of arbitrary I/O and pure execution.
See: https://sre.google/workbook/configuration-specifics/#pitfall...
It was concise and expressive. There was a direct way to describe variants (types with multiple constructors), which is always awkward in JSON, but the format was still surprisingly low-noise for reading and editing by hand. I remember you could even use it as a lightweight markup format:
(The format leaves the interpretation of things like (em ...) up to you; you could use it as a slightly more verbose Markdown, but you could also use it to structure readable text with other sorts of metadata instead.)And, unlike certain other formats I won't name, it has comments!
It also helps that Emacs with Paredit makes editing s-expressions flow. The tool doesn't need to know anything about the sexplib format specifically; just relying on basic s-expression structure gives us fluid but simple structural editing.
I am constantly sad that nobody else uses this sort of format, and I have to deal with a mixture of JSON, YAML, TOML and other ad-hoc formats instead.
[1]: https://github.com/janestreet/sexplib
Well typed, simple syntax. Maps are annoying though.
That being said TOML would be my choice.
Of course just adding multiline strings is the start of the rabbit hole, now you need to think about leading line breaks, trailing line breaks, intermediate line breaks, whitespace chomping, and- oh heavens I've reinvented YAML, I think I need to lie down.
[1] https://mikehadlow.blogspot.com/2012/05/configuration-comple...
And I remember toying with the idea of 9 but I think for a different purpose, I don't remember what. Never did get that far though, I'm acutely aware that tooling is also important, but it's still oh-so-tempting.
I wonder where the company is at now. I don't think they would have moved the clock ahead but they may have moved it backwards to 12.
I don't regret my little rules engine though. It held up very well. It was intended for non-devs to be able to use, but the article is quite right, a complex set up was not easy to understand! I often got questions when things looked wrong but after much thinking the rules were usually calculating things correctly.
The alternative would have been 100 hardcodings because each client needed something different.
* https://www.zytrax.com/books/dns/ch6/#master
* https://bind9.readthedocs.io/en/latest/manpages.html#named-c...
ISC-style? (E/A)BNF-ish?
Personally I like (a) the open/close braces for better stanza navigation, (b) all statements/lines ending with semicolon.
(Something similar was used in (now-EOL) ISC DCHPd, and they've moved to "extended JSON" for their new DHCP server, KEA.)
It's a nice mix of syntax that looks like what it does and general support.
It's probably stockholm syndrome.
But since we're criticizing formats ;-) maml https://maml.dev is fairly good good.
* I like multiline blocks and comment style
* I do not like optional commas, either require them or not. I'd lean toward requiring them.
* Not sure about ordered object keys. How should languages represent that, a map/dict kind of a data structure won't work it would have to be an ordered dict/list of kvs etc. Two objects {"a":"x", "b":"y"} and {"b":"y", "a":"x"} will be different then. I get the idea, but I am leaning toward not liking it by the time I finished typing all that.
* I like booleans and null. Good for not having ons and offs those are just annoying.
Personally I think it’s a very good decision. Simply by declaring that objects are ordered, it becomes the case and it unlocks a whole lot of useful mechanisms.
Programming languages have comments and control flow, multiple popular implementations, and can have nice literals. Lack of Turing completeness is actually not a terribly useful feature if you trust the input (and you should probably just use protobufs or similar for untrusted inputs in that case.)
The best configuration language is the one that is custom made for a specific application. A domain-specific language. But DSLs requires a parser and adding one is usually too cumbersome to merely parse a configuration. This is where XML comes in.
The problem is that folks want a flat human readable file format to solve all these problems. That's a pipe dream.
JSON and CSV are pretty close to "syntactically optimal" in terms of losslessly storing nested and tabular data, respectively, in a human readable format. We ought to just stick with those and think about how to design effective configuration systems on top of them.
Or maybe not enough features, because there's no other way to define a linebreak. \n isn't a thing.
And also, some editors are picky about the number of commas on each line. Excel and Sheets don't care if you have uneven number of commas, but IntelliJ's CSV viewer I think just looks at the first line to determine how many columns there ought to be.
Need to configure 5 services with hundreds of replicas in 7 data centers? Some values depend on the service, some on the data center and some on the combination thereof? Maybe also overrides for a bunch of problemstic machines?
And you also want a manageable config language which doesn't turn into a full blown Turing tar pit?
Then jsonnet is for you.
So it's not entirely fair to compare it in the "pleasant syntax" contest. It's like putting a Unimog into a ranking of city cars.
My guess is the RPN nature would be a no-go for many people. Nevertheless: comments, dicts, arrays, good string syntax, numerics, binary data, etc. Maybe that makes it too complicated.