If you're looking for a human-friendly json superset (comments, non-quoted keys) that can also abstract away repetitive configuration with variables and list comprehensions, check out https://rcl-lang.org/.
It kinda becomes a question of "does this comment annotate the line it's on, the next one, or the arbitrary number of succeeding lines" since the order of the objects is not guaranteed by the standard and when writing comments by hand it's common to say "the next section shall do X".
When I work on some ad hoc configuration format I usually end up with quite a family of different comment types. Disabled values and prose about values are are the core set, but there might also be different prose types to separate the intention for a certain value (authored by the one setting the value) from documentation about the purpose of the field (authored by the one introducing the option). Also a type for key value pairs that have not been consumed (perhaps because of a typo in the key), and another for pairs that are applied as default, but should not be explicitly in the config if you want to go with the new default of they change in a software update.
Yes, this is for situations where the config is two way, e.g. when a GUI can be used to set some values. But I find some of those features so useful that I might sometimes be tempted to write out a processed version of the file parsed even when there isn't anything like a configuration UI.
I am using nickel[1] myself for writing what basically amounts to a pipeline that ultimately generates a json or toml. It has contracts that can validate a field or an object as well as set a default value if the field is not present.
I wish languages adopted structured comments (as in, semantically applying to syntax tree nodes rather than lines) more broadly. It used to be a thing in some early PLs but has mostly died out.
I’m a fan of JSON5. A common criticism is “we’ve already got YAML for human readable config with comments,” but IMO YAML’s readability sucks, it’s too hard to tell what’s an object and what’s an array at a glance (at least, with the way it’s often written).
When dealing with large YAML files, I find myself frequently popping them into online “YAML to JSON” tools to actually figure out WTF is going on. JSON5 is much easier to read, at least for me.
Fair, YAML has a lot of usability warts, and those suck too. Although personally I really do hate how tough it is to tell apart arrays and objects, at least with the most common YAML array/object style.
This. I hate how all these serialization/config formats come out of dynamically typed languages. Static typing is a must. Then so many classes of errors go away.
Just static type then. You can’t trust incoming data shapes anyway, e.g. if it specifies a schema and doesn’t even follow it. You always expect something in a typed language, not anything. So validate it and that’s it. Thinking that dynamic data can be typed is a mistake. It can only be structured ([], {}, "", …) into basic types and then matched to some template. Any above-data section about types is as good as none. It can help a human to make sense of its shape, but that’s it.
That one is likely more relevant than yq since folks in the json ecosystem are far more likely to be familiar with jq's syntax and thus using gojq is "one stop shopping," not to mention that its error handling is light-years beyond jqlang's copy
Yes, but I LOVE yq's ability to update YAML files without stripping existing comments. For example, I use it to programmatically update similar (but not identical) GitHub Actions files across projects.
> When dealing with large YAML files, I find myself frequently popping them into online “YAML to JSON” tools to actually figure out WTF is going on.
YAML is a strict superset of JSON, so defining the former in the syntax of the latter is fully supported by the spec. Perhaps not by every YAML library, to be sure, but those which do not are not conformant. From the YAML spec[0]:
The YAML 1.23 specification was published in 2009. Its
primary focus was making YAML a strict superset of JSON.
I'm confused about your point about YAML being "strict superset of JSON" leading to being able to convert YAML to JSON.
If YAML is a strict superset, wouldn't that mean that YAML must have at least one feature that is not part of JSON? Wouldn't that make it impossible to define all YAML files as valid JSON?
> If YAML is a strict superset, wouldn't that mean that YAML must have at least one feature that is not part of JSON?
Yes. One of the features YAML supports is the widely documented format we are all familiar with.
However, being a "strict superset of JSON" also means a conformant YAML implementation can load a "pure" JSON resource without issue. The converse is not generally possible as JSON cannot express what YAML can, such as octothorpe ('#') comments.
Where arrays and objects just look too similar (IMO), white space is significant, most strings are unquoted, etc. And personally I find it quite difficult to really understand what’s going on there, at a glance, compared to JSON (or JSON5).
I want to be able to easily read and understand configuration without having to pop it into a converter. The YAML I encounter in the wild is ~80% pure block style, ~20% mixed (within a single file, mostly block style with some flow style). And I just find the block style hard to read, I have to either spend significant mental effort trying to understand where the objects vs. arrays are, or I have to pop it into a converter (to either JSON or flow style) to understand. Whereas JSON/JSON5, it’s immediately clear without any mental overhead.
As much as i like this (yaml goes way too far, but trailing commas and comments would make json much nicer. I actually think this spec goes too far with single quotes) i hate that it is named json5. I think its unethical to imply you are the next version of something if you don't have the blessing of the original author.
The project description describes it as json plus syntax from emcascript 5.1 (commonly called ES5) [the official name of javascript is emcascript]. I imagine that is where the name comes from.
Although it doesn't really make sense since most of the stuff they add predates ES5.
The parser for tsconfig.json (typscript.parseConfigFileTextToJson(fileName, jsonText) or parseJsonText) seems to be what you want; I wonder if there is a name for that format.
It's not a binary choice. I don't think if Crockford chose a slightly larger subset of JavaScript for JSON (such as JSON5), it would hurt its adoption.
Could also be construed as paying homage, though. I think the number 5 here is a reference to congruence with ECMA Script 5 rather than to imply a version of JSON.
I feel like the comments are the only important part. I’d rather not have single quoted strings or unquoted identifiers to be honest. Trailing commas are nice to have though.
All I miss in JSON are comments and a native datetime type. Everything else, I’m fine with.
There's power to it being valid JSON, though; it can be stored in databases optimized for JSON payloads, and edited with JSON editors. There's a real agility to liberally sprinkling JSONFields and JSON code editors across an auto-generated admin interface like the Django admin system, knowing that you can leave breadcrumbs for colleagues, and that anything starting with a `__` is fair game for comments any time you see such a key name.
Luckily, we have decades of experience in e.g. SQL, and existing and well-polished ISO specs on dates that can just be used as is without reinventing the wheel.
I mean, floating point numbers are easy to get wrong. And why care about semantics? This is a data transfer format, it should be able to represent data that is being transferred gazillion times a day.
Make it raw ISO 8601 { "created_at": 2024-12-10T11:20:07Z } and do not accept any other format. I know there are way smarter people than me who can figure this out at this point.
DateTimes are the number 1 reason I can not use JSON data as is (if I simply parse it into a dynamic object like JSON.parse).
I've always been a big fan of KDL in principle, haven't used it in anger. After that HCL, then YAML, with JSON and others being my least favourite to use.
Of course the hard part is gaining enough critical mass to make a significant switch. JSON had AJAX. YAML had Rails. What could make JSON5 or KDL break out?
Would it be correct to say that this is basically any valid JS code that describes an object, excluding the use of references and function definitions?
If not, what is the difference, and why was it made to be different?
Most of these things seem like missing features that would fit right in. Except for maybe template strings which could contain references to template functions
I've always thought it would be nice to have a language whose spec describes a few different "power levels":
1. Literal values only (objects, arrays, strings, numerics, booleans, maybe datetimes), a la JSON, with standardized semantics
2. Literals, variable definitions and references, pure function definitions, and pure function calls, but prohibiting recursion, such that evaluation is not Turing-complete and any value is guaranteed to be able to be evaluated into its literal form in finite time
3. All of the above plus recursion and impure, side-effectful functions, custom type definitions, etc.
This way, implementing a literal parser in other languages would be comparatively straightforward (much like JSON), since it wouldn't have to support variables or functions, but it would also be possible to define values using variables and pure functions (much like HCL or Nix) and then use the language's own evaluator binary (or perhaps a wrapped FFI library) to safely convert these into literal values that another language can parse, while guaranteeing that evaluation will not have side-effects. It would also leave open the escape hatch of using a "full" Turing-complete language with side-effects and recursion, while ensuring that using that escape hatch is a deliberate choice and not the default.
I'm sure there are a few additional or hybrid levels that could be useful too (2 but with recursion? 1 but with variables?) but this seems like it would be a solid starting point.
Their dictionaries and arrays split into ini-like sections are not very readable though. The double [[ is just nasty and not possible to apply in all situations (array in map in array).
TOML explicitly disallows newlines in inline tables, so that's not a full solution (if you agree that there are some situations where multi-line inline tables are indeed required).
Like YAML, it's only better in the simple case where everything is top-level and there's only one level of nesting.
Once you want to have arrays of nested objects or objects with arrays, I immediately wish I was just reading JSON so I knew where I was in the tree.
And for that reason, I don't think it's a full contender. I want an answer for the hard cases like nested data, not just another way to write the simple cases which is what TOML is.
For example,
[[a.b]]
x = 1
[a]
y = 2
Versus:
{
"a": {
"b": [ { "x": 1 } ],
"y": 2
}
}
It's easy to complain that the latter is noisier. But that's nothing compared to being clear.
This is also the reason I prefer XML to JSON when things really got complex. XML is verbose but it is very readable on long form. I wish Rust actually used JSON or XML as Cargo file format.
I think it works quite nicely in Cargo as you don't generally need much nesting, but anything with depth should use JSON. It's the perfect format for clearly displaying hierarchy.
It works only for trivial projects that are compiled in a roughly up-to-date and popular environment. Custom distros, vendored forks, closed-source software, firmware projects etc. all end up with hairy Cargo files. A systems language should recognize handling complexity gradually is a part of one of its use cases.
It's not made explicit in the documentation, but TOML is very nearly a superset of JSON - just using `=` to separate key-value pairs instead of `:`, and requiring top-level names to be given explicitly, and requiring the "inline" bits to be on a single line. In TOML, your example can equivalently be:
a = { b = [ { x = 1 } ], y = 2 }
(And yes, that can be the entire document; you can have an inline table before the first table header.)
Of course, this doesn't help if you want the top level to be an "array" rather than an "object" (in JSON parlance), or if you want the entire document to represent a single primitive value. But these uses are discouraged at best anyway.
But really the goal of TOML is to highlight the location of important parts of the deserialized tree structure (hence the ability to use arbitrary long dotted paths in table headers) rather than the structure itself. It's IMO a beautiful implementation of the idea "flat is better than nested" from the Zen of Python, and it neatly sidesteps an issue I asked about many years ago on Stack Overflow (https://stackoverflow.com/questions/4372229 - the question was rightfully closed, as this sort of discussion doesn't fit the Q&A format; but it made sense to ask at the time).
I don't know if a direct comparison of TOML to YAML is fair. Among other differences, the standard way to parse YAML in Python involves a third-party library that brings in a ~2.5 MB compiled C library. Every TOML implementation I encountered - including the one that made it into the standard library - is pure Python.
To my knowledge, such uses were discouraged only because of a security issue from evaluating a JSON string as a JavaScript code and not via something like JSON.parse.
Given that TOML is intended primarily as a format for configs, a config where the root value is an array or a primitive value is hardly a relevant scenario.
Your TOML is rather convoluted, a more normal way to write it would be:
[a]
b = [{x = 1}]
y = 2
Or alternatively:
a.b = [{x = 1}]
a.y = 2
Some parsers allow newlines in TOML inline tables, so you can do:
a = {
b = [{x = 1}],
y = 2,
}
That's supposed to be in the next TOML standard, but that seems indefinitely stalled as the (only) maintainer has seemingly lost interest and there hasn't been any movement for a long time.
I think it allows for too much. I was glad that JSON only supports double-quoted strings. It is a feature that removes discussions about which quotes to use. Or even whether to use quotes at all (we still need them for keys with colons or minus in it, so what gives?).
The only thing that JSON is really missing are comments and trailing commas. I use JSONC for that. It's what VSC uses for the config format and it works.
> The only thing that JSON is really missing are comments and trailing commas. I use JSONC for that. It's what VSC uses for the config format and it works.
I disagree. Human-friendy multiline strings aren't really optional for a serialization format that will inevitably also be used as a config format sometimes because those are the same problem.
> The only thing that JSON is really missing are comments and trailing commas. I use JSONC for that.
YAML[0] supports JSON formatted resources and octothorpe ('#') comments as well. I didn't see anything in the spec specifically allowing for trailing commas however.
Here is an exemplar using the Ruby YAML module:
#!/usr/bin/ruby
require 'yaml'
puts YAML.load(
%/
# YAML is a strict superset of JSON, which
# means supporting octothorpe end-of-line
# comments is supported in JSON formatted
# YAML if and only if the content is multi-line
# formatted as well (like this example).
{
# This is valid YAML!
"foo" : "bar"
}
/
)
Whilst YAML is an option, if the choice is between having the unnecessary extra features of JSON5 or YAML, JSON5 seems like the clear winner.
Allowing multiple types of quotes may be an unnecessary feature but it is a clear lesser evil compared to the mountain of footguns that YAML brings with it.
How does defining a YAML resource strictly in terms of well-formed JSON + octothorpe comments introduce "the mountain of footguns that YAML brings with it"?
It doesn’t, quoting strings does solve almost all issues, but it does leave potential footguns for the future.
If you don’t enforce it, in the future the “subset of YAML” property might get weaker, especially if someone else is modifying the config.
If you treat config files the same as code, then using a safe subset of YAML is the same as using a safe subset of C. It is theoretically doable, but without extensive safeguards, someone will eventually slip up.
The problem of yaml is that it allows too much. It allows unquoted strings, and those can be interpreted by the parser as numbers, timestamps, booleans, etc. This is a source of many fooguns.
Use of indentation to denote nesting can sometimes be an anti-feature, too, because while using that the format does not provide a way to make certain that the entire stream has been read (parens balanced). This may lead to problems or even exploits.
Pure JSON is so painful for human consumption though, I willingly choose yaml if it's the only alternative.
JSON5 may indeed be a sweet spot between human-friendliness and lack of nasty surprises.
As I mentioned in a reply to a peer comment, the problems you describe regarding YAML appear to be about the commonly used format most of us think of and the totality of the YAML feature set.
What is illustrated above is the definition of a specification-compliant YAML resource strictly using JSON constructs + octothorpe end-of-line comments.
Does this usage mitigate the concerns you have identified?
The problem is that self-restraint only takes you so far. Typos exist. Human mistakes exist. Machine errors exist. Malicious inputs exist.
A good parser does not just accept the inputs you find valid, but also rejects inputs you deem invalid. Running a linter that would report or amend all the footgun-wielding features of yaml before parsing is tanamount to running another parser. Then why bother :)
Using a YAML parser to parse JSON+comments is like bringing a tank to a knife fight... If you only parse "trusted" input, i.e. you can guarantee that no one is ever going to pass anything but JSON+comments, and you don't do it in any high-TPS scenarios it's probably fine to use a YAML parser
Trailing to signal a floating point seems like to niche of a use case to me. Generally it's best to treat every JSON number literal as a 64-bit float anyway for the sake of interoperability.
It became quite normal for me to write 0.3 and read it as “point three”. I do agree that the English language makes it less awkward to just skip the zero. It leaves very little room for confusion.
Well what's more readable, .8675309 that is understood to have an implicit zero, or the parser giving up and unexpectedly making it a string? Maybe it's not your preference but I can't see any problem with making this more robust. The trailing one is strange to me but leaving off a leading zero isn't unusual at all for written numbers in my experience.
IMO the implicit zero is just as much an issue in regular written form. The period could be overlooked quite easily, but seeing a leading 0, one will know what’s really going on.
How could the parser see it as a string? This is not YAML and JSON5 still requires quotation marks.
> Well what's more readable, .8675309 that is understood to have an implicit zero
Is it universally understood? I think it's a US / English thing. In my country I've never seen numbers written in this way and many people would not "parse" it mentally as 0.8675309
What an extreme reaction. Many people would not be able to parse it since they've never seen a number written in this way, but you immediately write them off as assholes. Wow.
It will look funny to many people but they will be able to interpret it. Remember that this thread is in the context of whether to be strict or relaxed in a specific file syntax for files intended to be authored by humans.
> It will look funny to many people but they will be able to interpret it.
You're still approaching this with the background knowledge of what this is. If you've never seen this, you can only guess, and there are a couple of options.
I've been terminally online for the better part of the last 2 decades, yet I've seen this way of writing for the first time only ~5 year ago or so, and I still remember simply not knowing what it is. The first reaction is that it's simply a typo - the author mistyped - the dot should have been a digit or perhaps not be there at all.
- "a gallon". Not purely US thing but almost. It can be anything from 3.8 to 4.4 liters, depending mostly on what are you measuring.
- writing digits. Is it "1", is it "7", is it "i", is it "l"? Why do it to yourself, while their printed fonts are pretty much the same as everywhere else...
For people who grew up in countries where comma is the decimal separator (and dot the thousands grouping separator), this is highly unintuitive, because it would seem much more likely to be a misplaced punctuation mark.
It might be moderately intuitive to English native speakers because of oral usage like “point one three eight”, but that’s also not a thing in many other languages.
It’s not about readability, it’s being realistic about other humans and making software robust in the presence of common, trivial, unambiguous “errors”.
Arguably, if there's ambivalence about numbers or other core structures or doubt about the intent, the correct thing to do is to return an error and not try to guess. The point of an error bubbling up is asking for clarity from the end-user, while guessing will lead to random results in ways that can not be detected or corrected anymore later on in the process.
I think Postel's law was intended to apply to alternative implementations of machine-level protocols.
That's not to say that I don't agree that it might be better if JSON implementations would allow trailing commas, which is unlikely to lead to semantic ambiguities. That's too late now though, unless a new JSON to rule them all would appear and we would all agree on that new spec.
I don't think Postel's Law was necessarily a bad idea, in its time and context. However it seems to have fallen into usage as an argument-ender. This RFC is a useful antidote to that.
If we consider sigfigs, then isn't 100 and 100. two different numbers given one has a single significant digit and the other has 3? For 101 and 101. it doesn't matter because both have 3 significant digits. Then again, one may argue that it is better to write 1e2 and 1.00e2 instead of 100 and 100.. It also avoids the weirdness of the double period if the number with a dot ends a sentence.
On a personal level, I also don't like ending a number with a . because my brain immediately parses it as a sentence ender and it interrupts my reading flow.
It's just not wanting to keep track of more rules. If you've only ever used languages where a leading decimal point is allowed, it's a pain point to suddenly have to remember that it isn't here, and for no obviously intuitive reason.
It's about wanting to avoid unnecessary conceptual friction. Not lazy keyboard usage.
(Heck, your second example uses an extra keystroke. And it's perfectly readable to me, based on the languages I use.)
There's something to be said for being flexible in your inputs when they are non-ambiguous. Particularly when dealing with files written by hand.
There's no virtue in imposing overly strict syntax when it serves no human purpose. That's trying to alter people to fit machines, rather than altering machines to fit people.
Yeah, and we could easily say that any property value without a comma, closing bracket, or closing brace; that doesn't follow the format for numeric literals; and isn't the literal value 'true' or 'false' doesn't need to be quoted when part of a value and will implicitly be a string. This adds no ambiguity and some people are used to writing their string literals without any extra quotes, why add the extra cognitive load of having to remember to use quotes? Wouldn't it be nice to write `user: { type: admin, name: Bob }`?
This seems like exactly how we ended up with YAML-and-the-kitchen-sink.
For a spec like this, things should need a better justification than that. Everything starts at -100 points. How does this feature justify the additional complexity in the spec, to the users, and to the people trying to implement this? How does it justify increasing the opportunity for creating subtly incompatible parsers?
Or, if a goal of JSON is not to be simple and interoperable, then I fail to see how it's not just YAML but 5 or 10 years behind the curve, so we may as well skip a bunch of hassle and move to YAML and start fixing the bugs we've got there rather than creating new ones here.
I don't know what to tell you, you seem to be taking a black-or-white approach here of extremes. Like anything that isn't limited to exactly one way of expressing it is a slippery slope to YAML. It's not.
JSON5 is literally exactly JavaScript notation, which tons of people are familiar with. On the other hand, I don't know any programming languages that use strings without quotes or delimiters. (And of course, their existing in YAML is widely recognized as a major source of confusion.)
And in your example, there isn't parsing ambiguity but there would be change ambiguity when you alter abc (without quotes) to 123, because it changes its type.
I'm not aware of any change-ambiguity situations in JavaScript/JSON5 like that? In Python 5 and 5. are different types, but not in JavaScript or JSON.
> The only thing that JSON is really missing are comments and trailing commas.
The reason JSON doesn't have comments [1]:
I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't.
Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.
> the larger issue is an inventor kneecapping something by stating how you should use it
This isn't kneecapping something any more than an inventor requiring programmers in a new language use types, or not use types, whichever the inventor deems preferable.
He invented a thing. He declared how the thing is constructed. That's not kneecapping. That's just defining a thing.
He invented a thing and left out a well known and understood core function required in the problem space, deliberately, not through oversight.
That's what makes it kneecapping.
The whole useful thing, ie the idea of a data format, including both knees (ie all the basic features any such thing needs) was there. The concept and necessity of annotation was a well known thing by then, and indeed he knew of it himself too, said so himself, and actively removed one functioning knee from that whole, to produce only the kneecapped thing.
He defined a kneecapped thing. Or he kneecapped the design. Whichever way you want to say it.
The difference would be if it was 40 years earlier and you are taking the fist stab at designing any kind of data format and comments just never occurred to you yet.
This is more like making a new programming language and deciding that it shall not have one of the basic math operators. If you need to multiply, tough shit, do it some other way. Just pipe it through JSMulti.
But he designed it with intention, as stated. It wasn’t like “ha ha suckers, no comments for you!” He just didn’t think it was a good idea and had a reason for it.
Are inventors required to only innovate a little bit? He released it, and there was every opportunity for someone else to offer something better and achieve the same level of uptake.
FWIW, I'm well aware of Crockford's rationale, I think it's some of the dumbest rationalization I've heard, and time has shown that it was a giant mistake.
Trying to prevent programmers from doing something "because they may 'misuse' comments" is asinine to the extreme. This is not like removing a footgun, it's trying to dictate how you think programming should be done. This is particularly hilarious because I've instead seen tons of horribly hacky workarounds for JSON's lack of comments, e.g. "fake" object entries like "__previousKey_comment": "this is a comment for previous key", or even worse, putting two entries with the same key since the second key should "win", and thus making the first entry the comment.
As for his second point, "Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser." - just look at the everlasting shit show that is package.json since it can't support comments, because lots of NPM tools actually write to package.json, so there is no guarantee that other tools will be able to read package.json if it includes comments.
I think the thing that I hate most about Crockford's rationalization of not having comments is that you have to have your head stuck 10 feet in the sand to pretend that somehow the lack of comments is a good thing with the benefit of hindsight. I guess I could understand if Crockford's position was "Yeah, originally I thought it was better to keep them out because I was concerned about misuse, but now in retrospect to see the much larger problems it causes, and I realize it was a mistake." But instead he just keeps pushing his original crappy opinion on this topic.
I don't know who @fat is, but if he thinks Crockford's rationalization for the lack of comments in JSON is total and complete BS, I like the way he thinks.
Exactly, and that's a crappier workaround. E.g. in lots of places you can't just have willy-nilly, anything-goes key names.
An example I think every Node developer can commiserate with is that there isn't really a great way to add comments in the dependencies or devDependencies section in package.json, because those keys need to refer to module names, and that's where I most often want to use comments in package.json. I won't rehash all the details but just take a look at https://stackoverflow.com/questions/14221579/how-do-i-add-co... . Unfortunately none of the answers there are very good. In the past I've resorted to doing what https://stackoverflow.com/a/45815391/1075909 does, but that ends up not working very well once you've got a long list of dependencies, because the comment for any particular dependency ends up being far from the entry that specifies the dependency (instead of the line above), so people end up not reading it or forget to update it if that becomes necessary.
Yeah, I really, really hate that JSON doesn't have comments :)
I have encountered many systems that explicitly disallow “unexpected” JSON elements and such an approach would fail. This is particularly common when using JSON around API-like use cases.
I often want to comment individual array values to give more context or describe provenance. I don't think I've ever wanted to add a comment in the middle of a keyword.
These two are not remotely the same thing. You can have an array declared on multiple lines and add a comment to specific values. Or even inline with the value /* */.
When I think of how often and where JSON is used, as a data interchange format for APIs, in the browser, databases, and everywhere else - all the billions of bytes transferred in JSON every second - in all those use cases comments would be pointless and counterproductive and just take up storage space or bandwidth. That's JSON's primary use case in the world. It's only in the very few use cases specifically for programmers where comments would sort of be helpful in JSON, and most of those cases are not really that important to support when there are workarounds - structure your JSON well and you can include the comments as part of the JSON, and then you can even use the comments programmatically should that be useful (which I think is slightly more useful than storing knowledge in a JSON file as a comment).
>Trying to prevent programmers from doing something "because they may 'misuse' comments" is asinine to the extreme.
Programmers are often their own worst enemies. Some prefer rigid rulesets over less rigid freeform programming. See Typescript vs Javascript. No comments in JSON is just another example of over-rigidification, and some programmers love that.
>package.json since it can't support comments,
If you're needing to write comments in package.json, maybe you're not approaching package.json correctly? It's really for node things, not your things that you need to write comments about. I'm not even sure why someone would want to write comments in package.json. I get it with comments in other JSON files, but package.json probably should be left for nodejs things.
It's hard for me to find a comment where I disagree with literally every sentence, but congrats!
1. So what if JSON is primarily used for data interchange? It's not like allowing comments would somehow make them magically show up on serialized objects. This objection makes 0 sense to me. And heck, tons of other serialization formats (e.g. XML) support comments. Besides, there is a big reason that human-readable serialization formats are so popular - because they're human readable! If you're really worried about size you should be using a binary format anyway.
2. "Rigid rulesets" has nothing to do with Crockford's arguments. It's one thing to prefer a particular type system, or limit functionality if you think it has high potential for misuse. By JSON not having comments all you end up with is worse workarounds, like putting comments in object keys.
3. "I'm not even sure why someone would want to write comments in package.json" To be blunt, then, I can't believe you've ever written any code in a business (i.e. with multiple developers) in the Node/NPM ecosystem. Is it really that hard to wonder why someone would want to comment why they added a particular dependency to their project? The lack of comments in package.json is one of the biggest complaints of this format, and it's certainly not just me, e.g. https://github.com/npm/npm/issues/4482 and https://stackoverflow.com/questions/14221579/how-do-i-add-co...
3. So when that dependency gets removed by npm uninstall how should that comment be handled? You know that in business we just would end up with a bunch of dead comments in the package.json - is that really a better alternative than just using Ctrl+f to find where the dependency is used?
>The lack of comments in package.json is one of the biggest complaints of this format, and it's certainly not just me
Well then you and plenty of other people have some wrong ideas about package.json. That isn't surprising.
package.json gets rewritten for all kinds of things, which is not really compatible with adding comments wherever you want. Adding "why this dependency is here" comments may seem like a good idea to add to package.json, but you're kind of missing the point. If you need that level of documentation, trying to shoehorn it into package.json is just the wrong place for it. Soon enough your package.json looks like a graffiti wall.
>To be blunt, then, I can't believe you've ever written any code in a business (i.e. with multiple developers) in the Node/NPM ecosystem.
Then you'll be astonished that I have been working with nodejs for about 14 years professionally. Sure I have wanted to put comments into package.json, but I was naive and now I'm fine not doing that. I haven't wanted to in many years. I document things in other ways and it has served us all very well. YMMV.
> if you need that level of documentation, trying to shoehorn it into package.json is just the wrong place for it. Soon enough your package.json looks like a graffiti wall.
So the right place is to make a graffiti out of another place, instead of in the place where people actually declare the dependencies?
I find it bizarre when people believe in one true way of doing things. I mean, you can declare your dependencies how you like, but if others do it differently, then they're clueless?
You're clueless if you think adding comments to package.json - a file that regularly gets rewritten - is anything but an exercise in futility. Any time you run "npm install [whatever]" you are rewriting package.json. How exactly do you expect to maintain your random comments in this case? You expect nodejs to understand how comments are being written in package.json and not mess that up? You don't seem to understand how npm or package.json works.
This is the silliest of circular logic. Of course you can't add comments to package.json, so tools can do whatever they want to the file. The fact that tools can rewrite parts of it doesn't mean they should just be able to do whatever they want. Literally every other single other config file format supports comments, and I have never seen a problem with tooling on those files due to the presence of comments.
> comment why they added a particular dependency to their project
Surely that's what the commit message is for? I mean, I get that it's more convenient to have the comment right there in the file, but that should be balanced against the downsides: having to maintain the comment, making the file larger and more awkward to read, etc.
Your rationale could apply to all comments in general. Hey, just have folks scour through the commit messages instead!
Yes, I still think a commit message is important, but it absolutely does not take the place of a comment. Suppose you'd like to do something like this:
// DO NOT change to latest version. MUST pin to version 1.2.3 because
// 1.2.4 includes a subtle bug that doesn't work with our version of
// the zed library. We have an open issue on zed, see
// https://github.com/zed/zed/issues/1234.
// Once that is fixed, we can upgrade.
"foobar": "1.2.3"
There is zero chance that comment is going to be seen if its just in a commit message, furthermore you should never depend on git blame showing a particular comment because it's easy for later commits to overwrite it due to simple things like formatting changes, etc. Yes, in that example case there should be tests too, but IMO the primary purpose for comments is to highlight issues and concerns that aren't immediately apparent just by reading the code.
I simply cannot think of another file format that is used for configs that doesn't support comments.
>There is zero chance that comment is going to be seen if its just in a commit message,
I never suggested using a commit message, there are plenty of other ways to document these things and I'll leave that up to the user to figure out.
So now you've written this tome of a comment in package.json. What happens the next time someone installs a package using npm install? package.json gets rewritten, that's what. Your comment will be gone. I suppose you expect npm to somehow use AI to guess at how to rewrite package.json so it can put your comments in the correct places??
And expecting someone to read package.json before updating or installing a library is just as useful as putting it in a commit message. If you really need to be careful about dependency versioning, you better have more safeguards in place than just a comment in package.json.
>I simply cannot think of another file format that is used for configs that doesn't support comments.
I mostly see configs created as .js or .ts files, where comments are allowed. Not package.json. Never package.json, because package.json is guaranteed to be rewritten regularly. But maybe you missed that part of your bootcamp class? Yes, .js files can be used as "config". It's been done in plenty of projects. It isn't the end of the world. JSON also isn't solely used for configs or data that needs comments, in fact the majority of use cases in the world for JSON won't need any comments at all so writing comments in package.json is kind of an edge-case.
>I simply cannot think of another file format that is used for configs that doesn't support comments.
JSON isn't only a config file format. It's primary use is data transfer.
> I never suggested using a commit message, there are plenty of other ways to document these things and I'll leave that up to the user to figure out.
Dude, I think you're lost, in more ways than one. I was directly responding to a comment that stated "Surely that's what the commit message is for?"
For the rest of your comment, at this point I'd rather have an argument with a dining room table. No shit you can't have comments in package.json now, that's the entire reason that issue https://github.com/npm/npm/issues/4482 is unfixable. If JSON supported comments from the beginning, then tooling would have to respect that, just like the bajillion other config file formats that support tooling that updates the config file programmatically.
I one time by sort of accident coined a config format by parsing todo.txt It was so useful it stuck around for much longer than I had intended to. Rather than comment out stuff it just looks for "=" and ignores everything else.
I just took a look at an example package.json file and it seemed fine — no 'comments' shoehorned in anywhere, meaningful key names that reduced the need for comments. Do you have an example of a package.json file that would be better if json supported comments?
I don't have an example file on hand, but they would be useful for documenting non-explicit dependencies between packages. e.g. Kendo UI and the corresponding theme packages[1] - neither NPM package depends on the other, but if there's a comment then when seeing a change to one, the developer or a reviewer should know to double-check the other.
Well not offhand, comments aren't supported so workarounds are used instead. But I would find it very convenient if I could leave comments on dependencies and scripts. Or even various engine requirements.
Sure you can write everything in another file or put the comments in the commit message. But out-of-band comments are more easily missed or lost. If the package.json got rewritten by `npm install` you'd lose the comments. Inconvenient, but that's trivial to fix at commit time.
The reason never made much sense, anything can be abused, and the comment use case is easily way more important, and your suggestion doesn't help with eg syntax highlighting tools for your config that will treat comments as syntax errors, and also lose the ability to roundtrip, so your app can't edit user configs
You are saying as if there was an apology or it should be needed for some reason.
For any spec there are people who want something spec doesn't do and people writing the spec need to say no to requests that they consider not in scope as much or more often than they say yes
There was literally an apology, a douchebag backhanded false one, but still acknowledged explicitly that there was a thing he knew everyone would want, and knew it would "make some people sad".
Comments are not some weird thing one person wants for their weird reason that no one else needs to care about. It's like leaving out a letter from the alphabet.
It's not out of scope, because this is not a binary data format.
Saying "I intended to make a defctive thing doesn't" doesn't change the fact that it's defective.
A car without windshield wipers is defective, or at best inexcusably limited. Saying "I only designed it to use in the sun" doesn't make it suddenly perfectly useful.
If you want to try to say that json was actually intended to only be used in special conditions like an exotic car with no roof, then it's fair for everyone else to say "Ok, well that intentional design is a crap design of limited use. People actually need a car WITH windshield wipers."
The simple fact that it's a text format invalidates all the attemped arguments that it's just a machine to machine data format never intended for humans to mess with. People aren't "holding it wrong".
Then why did he invent json and everyone else invent yaml and toml and xml etc if csv already existed and was so complete?
If small parser was the most important, that's what binary formats are.
Or if the data needs to pass through a text handler that can't handle binary, we already had csv and a few different standard and cheap encodings.
csv's excuse for lacking some of the bare minimum features is that csv was first. Csv is from the time of typewriters and handwritten data. What would eventually become known, the most common needs, just hadn't been encountered yet.
A single person did not create csv after 40 years of seeing what is needed, and decide that csv shall not have an expected functionality in it's spec. There essentially isn't even any spec for csv. No one explicitly authored it. It's a typewriter format.
csv continues to get used after that point because of inertia. Once it's used many places, it continues to get used in new places because most new things have to interoperate with the existing ecosystem if you want to sell to the most possible customers.
Also csv, unlike json & similar, IS a pure data format, not used for things like configuration, because it's really only barely human usable, because the columns don't line up and there's no form of annotation other than a header record. Not to mention every part of the format, quotes, commas, even newlines, are valid data that may appear in a field, and so the only way to read the file is to manually reproduce the streaming state-machine in your head.
If something as limited as csv is so good and doesn't need anything else, then why did he invent json when csv already existed for jobs that narrow in scope?
csv is not remotely proof that text formats don't need annotation.
People keep bringing it up that "anything can be abused" but the point is that if you want to abuse something, abuse the simple parseable data rather than comments in the syntax tree.
Your two examples are just two examples of why we don't need comments for data interchange: yes, you put the data in a trivial, stable position in the parsed data that all parsers can parse rather than write some sort of DSL that has to be extracted from comment nodes that may appear in various places in the tree.
Is the whole point. The more "abusive" you imagine the contents of "directive" to be, the more reason it should exist as the latter data, not more reason we should accept the former.
It is more that people made comments semantically important, e.g. a tool which did not interpret comments would not correctly interpret the data in the document.
This actually would put JSON in a worse place than XML - while XML has an overly complex infoset, that infoset is actually defined and standardized. Representing "a property with a comment on the property name and one before and after the property value" so that information is not lost in parsing would explode the complexity needed for an "interoperable" JSON library.
if someone wants to create some sort of scheme where they do "createdAt$date" as a property name to indicate the value is to be interpreted in some agreed-upon date format, that at least doesn't lose data if the JSON data doesn't understand that scheme, or require a new custom parser in order to properly interpret that data, compared to something like /* $type: date */ "createdAt" :...
why would you need that complex representation in the interoperable library instead of a much simpler one: a property, a comment, a comment, a value, a comment, ...?
This doesn't explode anything and you don't need to lose any data, so the monstrosity of XML still has no benefit, and neither does "createdAt$date", which would need a custom library anyway, so it doesn't matter where you insert your types
XML may have no benefit to you but it absolutely has benefit.
A bit off topic when the only consideration here is comments, but XML allows for type definitions and structures data that simply isn't possible in JSON.
People have found ways to attempt to add types to JSON but they aren't part of the spec at all and are just convention-based solutions.
>and also lose the ability to roundtrip, so your app can't edit user configs
I think roundtrip with comments is not feasible in general. Most code just expects a hashmap, which it edits and then serializes back. You would need some really clever handling for comments to do it.
If you close your eyes, the data doesn't disappear! Of course you have the data, it's in the comments, and the original comment explains how it got there
I don't understand the problem. But I do understand the desire to keep the json definition simple and concise.
If you need additional fields in the json to hold comments, why not add the fields however you want? And if you need meta-data for the parser, you could add it the same way. In a project, i am working on right now, we simply use a attribute called "comment", for comments.
e.g. use "_" as a prefix to mark comments, and then tell you applications to ignore these attributes.
{
"mystring": "string123",
"_mystring": "i am a comment for mystring",
"mynum": 123,
"_mynum": "i am a comment for mynum",
"comment": "i am a comment for the entire object"
}
The challenge there is that the solution is convention rather than spec. Comments would only be understood by parsers that follow, or at least support, that convention and all other parsers would treat the data differently.
That may not be a huge problem for you, you see the "comment" key and know its just a comment and can ignore when a parser treats it like a normal string field. It could be an issue though, for example I could see any code that runs logic against all keys in the object becoming harder to maintain.
Adding comments doesn't contradict "simple and concise" as it doesn't add much, but on the contrary allows avoiding verbose solutions (repeating names) (but also now you need to do string escaping inside a comment???) such as the one you suggest, which will also not have custom syntax highlighting since it's not part of the spec and have a bunch of other issues (like, now you don't know how many keys you have without parsing every key name to exclude comments)
> If you need additional fields in the json to hold comments
I don't need additonal fields, I need comments
> why not add the fields however you want
Because I can't do that either, there are noticeable limitations
That solution is clearly weirder and more complex to understand than the comments it replaces. You impose semantic burden while demonstrating that the "abuse" of comments cannot really be prevented.
> The reason never made much sense, anything can be abused (...)
I don't think your take makes sense. Comments were purposely left out of JSON to mitigate the problem described by Hyrum's law.
Smaller interface means a smaller surface to allow abuse. You're just arguing to facilitate abuse because you feel the abuse that was indeed avoided by leaving comments out is unavoidable, which is a self-contradiction.
On top of that,think about the problem for a second. Why do you feel it's reasonable to support comments in data interchange formats? This use case literally means wasting bandwidth with data that clients should ignore. The only scenario where clients are not ignoring comments if exactly the scenario that it's being avoided: abusing it for control purposes.
If you want a data interchange format to support that, you should really adopt something that is better suited for your whole use case.
> and your suggestion doesn't help with eg syntax highlighting tools for your config that will treat comments as syntax errors (...)
That's the whole point. This is by design, not a fault. Why are you pretending a language supports a feature it never supported and explicitly was designed to not support?
> Smaller interface means a smaller surface to allow abuse.
If you take your law seriously, this is irrelevant because the surface of abuse is on the same scale of practical infinity, so it doesn't matter that one infinity is technically smaller.
For example, based of the example in the quote: you could stick those directives from comments info #hashtags in stringy values, with the same effect that there is no interoperable way to parse values (or if you add "_key_comment" - there is no interoperable way to even parse keys as some of them need to be ignored)
So the designer has achieved no benefit by removing a valuable feature
> abuse that was indeed avoided
Nothing was avoided, you can have the exact same abuse tucked into other elements
> Why do you feel it's reasonable to support comments in data interchange formats?
Why does the author of the quote sees the obvious which you don't see even after reading the quote? Go convince him his comment makes no sense because of "data interchange"
Obviously it's not only used for data interchange in cases where every byte matters (reminder: this is a TEXT-based format) and also comments matter to humans working with this data
> adopt something that is better suited for your whole use case.
And this discussion is literally about a format supports that? But also, how does this in any way mitigate the flaws in the designer's arguments?
> Why are you pretending a language supports a feature it never supported and explicitly was designed to not support?
Same thing, why is the author of the quote makes this senseless suggestions then? Go convince him first
Hyrum's law is not mine. It's a statement of fact that goes by that name already for a few decades and is noted by anyone who ever worked on production services that exposes an interface and is consumed by third parties.
> (...) this is irrelevant because the surface of abuse is on the same scale of practical infinity, so it doesn't matter that one infinity is technically smaller.
It really isn't. JSON does not support comments, thus they aren't abused in ways that sabotage interoperability. The only option you have is to support everything at the document schema level. You can't go around it.
> For example, based of the example in the quote: you could stick those directives from comments info #hashtags in stringy values, (...)
Irrelevant. You're specifying your own schema, not relying in out-of-band information.
That's exactly how a data interchange format is designed to be used.
There is no way around it. If you understand the problem domain, the fact that leaving out comments is an elegant solution to a whole class of problems is something that's immediately obvious to you.
The arguments presented here would also lead someone to not use JSON for data interchange.
If specifying the schema is important then why not an interchange format with strict schema enforcement like XML?
If minimal feature sets to avoid abuse are so important, then why not a binary format like protobufs which also have strict schema dictates?
And if interoperability is important then why not ditch REST/JSON all together and instead favor SOAP which strictly defines how interoperability works?
That's why I don't buy the "comments might be abused" argument. JSON doesn't have a single problem domain and it's not the only solution in town.
I think the opinionated approach is what made json “win” the format battle more than anything else. If you’ve been around enterprise in the early 00’s you’ll know the horror that XML became when people weren’t using it in any sane manner. Which a lot of people simply weren’t, for whatever reason. Now this is anecdotal but over the decades I’ve never had issues parsing json, and I largely attribute its strictness to that. Yes I suppose you could abuse it, but I wouldn’t have to parse your nonsense, which couldn’t be said for XML.
I don’t have anything against XML by the way, it was purely horrible to work with because people were using it in so many weird ways. Personally I prefer toml, but I guess we all have our preferences.
JSON wasn't some magical, made-on-the-fly format that makes Crockford some kind of genius. It was simply the standard Javascript object literal notation with some added constraints. I think some of those constraints make sense (i.e. are there any other languages that support both single and double quotes for string literals?), but funnily enough, some of the biggest issues with JSON interoperability is it is very underspecified in the areas that matter, such as the type and width of numeric literals, what to do in some edge cases like duplicate keys, etc. Just did a quick search, and here is a post that outlines some of the real security risks this underspecification leads to: https://bishopfox.com/blog/json-interoperability-vulnerabili...
Back in the early 00’s json wasn’t winning because it was derived from JavaScript, it was winning because it was the “easiest to use” standard on web services.
Sure it’s derived from JavaScript and it plays a major part in frontend development today. It was really the other way around though, when Ajax picked up people started realising that they could use json to make frontends work with “just” html and JavaScript.
That doesn't make it a good reason. People are placing those directives into json docs anyway, but instead they're relying on nobody causing a namespace collision with their special key whos associated value has the directives of interest.
> I removed comments from JSON because I saw people were using them to hold parsing directives
there's always someone putting actual logic in comments, and when I rule the world, those people are all going to be put on an island by themselves and they're going to have to deal with each other.
> So instead of comments with parsing directives, people use underscore prefixed keys to keep metadata and comments, that's not a win at all
Your comment doesn't make any sense. You're just pointing out that developers designed data interchange formats as subsets of JSON which explicitly support metadata and comments. This means they are specifying their schemas to support these usecases in a way clients can and do support it. That, if anything, proves right the decision to not support comments because it resulted in better-defined and well-behaving clients.
The thing is that JSON was intended to be a data exchange format, not a configuration file format or anything else. IMHO Crockford's reasoning makes a lot more sense with that in mind.
Data interchange formats are often used for configuration. It makes sense to have a single source of truth in json if your configuration is consumed by an app.
Even if we stick to the data exchange format, it would be practical to have comments in examples of data. This would be good for training and learning, for documentation.
This is still possible. There's nothing stopping you from putting comments in the JSON in your file or a webpage. It's just not going to be accepted as valid by JSON parser, which for learning material should be fine.
XML was intended as a data exchange format and has comments. Image file formats like JPEG and PNG support comments (and so does SVG by virtue of being XML). Database systems support comments on database objects. It’s really not a convincing argument.
JSON is a human-readable data exchange format. There is a good reason human-readable data exchange formats are so popular - they can be read and understood by humans! So it seems pretty absurd that a format designed to be read by humans doesn't support comments. If you're really concerned about performance or size, you should use a more efficient binary format for data exchange like protobufs, or heck, BSON (binary json)!
One way to bifurcate the world is that there are people who tend to restrict options to prevent misuse vs. people who tend to enable options to the positive uses easier. I fall in the latter camp. Preventing errors by restricting options also prevents positive uses that make life better.
We get to choose what approach we take. I prefer the "give them the rope they might need" philosophy. If they hang themselves, oh well.
I agree, engineering is all a series of tradeoffs, and tradeoffs can go one way or the other.
That said, I have yet to see anyone ever say "Oh man, this programming/configuration/data transfer language would be so great, but I wish they hadn't supported comments!" I see the opposite about JSON all the time.
And I don't see this about other features, e.g. Go specifically decided not to support the ternary operator, and while some folks may like that feature, I think more folks really appreciate that with Go there is generally a "standard" way to do things and it doesn't give you a million different ways to essentially do the same thing.
I'm not a fan of forcing single or double quotes because escape codes are such a pain to deal with and to me make things significantly harder to read than an inconsistent quoting style ever could.
Agreed. The workaround (arrays of strings) isn't great as it means an extra transformation has to be done between the reader and the usage. I would go so far as to say this is more important than comments.
JSON only allowing double quotes is something I have grown to not care about, but as someone that was using JavaScript object literals before JSON became a thing, I confess I do not understand why it is an advantage? If you were at a place where it was a heavy discussion on what quote to use, I'm forced to think there were deeper cultural issues at play?
Don't get me wrong, the ubiquity of JSON speaks for itself and is the reason to use it. But, to say it has tangible benefits feels very dishonest.
The choice of single vs double quotes means you can use single quotes if the contents contain a double quote and vice-versa. With JSON containing shell scripts (looking at package.json scripts) that's a valuable addon imo.
To the reader: if you haven't before, take a stroll through the YAML spec to see what people are talking about when they bemoan its complexity: https://yaml.org/spec/1.2.2/
I like JSON5 and have used it some. When GPT was younger and I was parsing its JSON output directly, JSON5 was forgiving in useful ways.
The one thing I really wish it had was some form of multi-line string, mostly so I could use it with line diffs. Also sometimes for really simple property editors it's nice to put the serialization in a textarea, and that works fine for everything but multiline strings.
(I suppose you can escape newline characters to put a string on multiple lines, but I find that rather inhumane and fragile)
I use it for personal scripts and it's been wonderful. I get to write more beautiful and concise configuration than any other format I've used.
If I were doing a professional project, I'd be hesitant to use it over something with more popularity and support. The syntax has so many variations two files can look like two totally different config languages, which is both cool and alarming.
I always thought that “JSON5” is a deceptive name. It is not the fifth version of JSON; it is an alternative/extension of JSON, of which there are many alternatives, and this one is no more official than any other.
JSON5 is from ECMAScript 5.1, which is called ES5.
As an (unfortunate) JavaScript developer, it was clear to me the intent was to "update" JSON with ES5 features, and not say ES4 or ES6.
Why ES5? ES5 is when trailing commas were introduced. Commas are one of defining features of JSON5. Other languages, like Go, also made this a priority.
The name may have a logical reason for being what it is. But it is still misleading. I have seen people implicicly assume that JSON5 is what they should be using instead of JSON, just because of the name.
The whole reason JSON rules the world is because it's brutally simple.
We already have 5+ replacements that are far more robust(XML, YML) and IMO they are not great replacements for JSON.
Why? Because you can't trust most people with anything more complicated than JSON.
I shutter at some of the SOAP / XML I have seen and whenever you enable something more complicated inevitably someone comes up with a "clever" idea that ruins your day.
Okay, I originally didn't respond, but your comment got upvoted out of gray so I'll put in the effort.
Your claim of "No, it wouldn't" is simply wrong. They said "since 1.2" and you can't just ignore that when you're citing a 1.1 problem. The disclaimer that you edited in gets at a relevant point but it's not strong enough to make your claim actually be true.
And the winky face and the "hint" just make things worse, since they knew exactly what you meant.
> The whole reason JSON rules the world is because it's brutally simple.
I don't think that's the primary reason. JSON is pervasive because it started out being trivially parseable by JavaScript going back to when people just evaluated it, even before browsers had ridiculously high-perfomance safe JSON parsers. All the other formats are still harder to work with from JavaScript.
If not for that, personally I'd advocate TOML, which is incredibly simple.
We don't use any other JSON5 features, which are primarily just that numbers may be encoded in hexadecimal and field names may have quotes elided.
We typically encode values with RFC 4648 base 64 URI canonical with padding truncated (b64ut) with values too large to be a JSON number, so hex isn't useful anyway. We haven't found that omitting field name quotes is a big deal.
To be precise, JSON was a replacement for XML, not the other way around. And the problem with XML was that it's way to verbose and difficult to write by hand, so it's exactly the opposite of the direction YAML/JSON5/... are taking.
Have you looked at CUE? I haven't looked back since discovering it. CUE is a proper language for configuration that two-way integrates with many other config and schema formats. All JSON is valid CUE too
Take a look at https://typeconf.dev
We wanted to define configs with types and avoid custom DSLs as much as possible. So we ended up with using Typespec.io for schema and plain Typescript for authoring configs. This should be as normal as possible!
353 comments
[ 3.6 ms ] story [ 333 ms ] threadYes, this is for situations where the config is two way, e.g. when a GUI can be used to set some values. But I find some of those features so useful that I might sometimes be tempted to write out a processed version of the file parsed even when there isn't anything like a configuration UI.
[1]: https://nickel-lang.org/
When dealing with large YAML files, I find myself frequently popping them into online “YAML to JSON” tools to actually figure out WTF is going on. JSON5 is much easier to read, at least for me.
Additionally plugins let you link your prototxt file with the corresponding proto so you can spot errors right away.
That's a big advantage compared to converting to YAML.
That one is likely more relevant than yq since folks in the json ecosystem are far more likely to be familiar with jq's syntax and thus using gojq is "one stop shopping," not to mention that its error handling is light-years beyond jqlang's copy
YAML is a strict superset of JSON, so defining the former in the syntax of the latter is fully supported by the spec. Perhaps not by every YAML library, to be sure, but those which do not are not conformant. From the YAML spec[0]:
0 - https://yaml.org/spec/1.2.2/If YAML is a strict superset, wouldn't that mean that YAML must have at least one feature that is not part of JSON? Wouldn't that make it impossible to define all YAML files as valid JSON?
For a feature like references, you'd have to do the annoying thing and duplicate that section of the file.
For a feature like unquoted strings or extra commas, you just quote the strings or remove the commas.
The various YAML features are in between and mostly close to the latter.
Yes. One of the features YAML supports is the widely documented format we are all familiar with.
However, being a "strict superset of JSON" also means a conformant YAML implementation can load a "pure" JSON resource without issue. The converse is not generally possible as JSON cannot express what YAML can, such as octothorpe ('#') comments.
HTH
EDIT: see also https://news.ycombinator.com/context?id=42361994
Where arrays and objects just look too similar (IMO), white space is significant, most strings are unquoted, etc. And personally I find it quite difficult to really understand what’s going on there, at a glance, compared to JSON (or JSON5).
So what? YAML can be trivially mechanically translated between flow and block syntax.
If it didn't originate from that, what else?
Although it doesn't really make sense since most of the stuff they add predates ES5.
JSON5 is closer to "javascript object notation" than JSON itself. It's partly an update and partly a removal of arbitrary restrictions.
edit: well as this comment thread indicates, the name is pretty confusing for everyone :)
All I miss in JSON are comments and a native datetime type. Everything else, I’m fine with.
(Reference: https://youtu.be/y8OnoxKotPQ)
JavaScript certainly doesn't.
Make it raw ISO 8601 { "created_at": 2024-12-10T11:20:07Z } and do not accept any other format. I know there are way smarter people than me who can figure this out at this point.
DateTimes are the number 1 reason I can not use JSON data as is (if I simply parse it into a dynamic object like JSON.parse).
Of course the hard part is gaining enough critical mass to make a significant switch. JSON had AJAX. YAML had Rails. What could make JSON5 or KDL break out?
If not, what is the difference, and why was it made to be different?
No template literals (backticks)
No regex literals
No octal numbers (0123, 0o123)
No boolean numbers (0b1001)
No big ints (the "n" suffix)
1. Literal values only (objects, arrays, strings, numerics, booleans, maybe datetimes), a la JSON, with standardized semantics
2. Literals, variable definitions and references, pure function definitions, and pure function calls, but prohibiting recursion, such that evaluation is not Turing-complete and any value is guaranteed to be able to be evaluated into its literal form in finite time
3. All of the above plus recursion and impure, side-effectful functions, custom type definitions, etc.
This way, implementing a literal parser in other languages would be comparatively straightforward (much like JSON), since it wouldn't have to support variables or functions, but it would also be possible to define values using variables and pure functions (much like HCL or Nix) and then use the language's own evaluator binary (or perhaps a wrapped FFI library) to safely convert these into literal values that another language can parse, while guaranteeing that evaluation will not have side-effects. It would also leave open the escape hatch of using a "full" Turing-complete language with side-effects and recursion, while ensuring that using that escape hatch is a deliberate choice and not the default.
I'm sure there are a few additional or hybrid levels that could be useful too (2 but with recursion? 1 but with variables?) but this seems like it would be a solid starting point.
https://toml.io/en/
Yes, that's largely why inline tables and arrays exist:
Once you want to have arrays of nested objects or objects with arrays, I immediately wish I was just reading JSON so I knew where I was in the tree.
And for that reason, I don't think it's a full contender. I want an answer for the hard cases like nested data, not just another way to write the simple cases which is what TOML is.
For example,
Versus: It's easy to complain that the latter is noisier. But that's nothing compared to being clear.Of course, this doesn't help if you want the top level to be an "array" rather than an "object" (in JSON parlance), or if you want the entire document to represent a single primitive value. But these uses are discouraged at best anyway.
But really the goal of TOML is to highlight the location of important parts of the deserialized tree structure (hence the ability to use arbitrary long dotted paths in table headers) rather than the structure itself. It's IMO a beautiful implementation of the idea "flat is better than nested" from the Zen of Python, and it neatly sidesteps an issue I asked about many years ago on Stack Overflow (https://stackoverflow.com/questions/4372229 - the question was rightfully closed, as this sort of discussion doesn't fit the Q&A format; but it made sense to ask at the time).
I don't know if a direct comparison of TOML to YAML is fair. Among other differences, the standard way to parse YAML in Python involves a third-party library that brings in a ~2.5 MB compiled C library. Every TOML implementation I encountered - including the one that made it into the standard library - is pure Python.
To my knowledge, such uses were discouraged only because of a security issue from evaluating a JSON string as a JavaScript code and not via something like JSON.parse.
Anecdata, granted, but I've seen a lot of TOML written exactly like the "convoluted" example, and none like your more spread out version.
The only thing that JSON is really missing are comments and trailing commas. I use JSONC for that. It's what VSC uses for the config format and it works.
JWCC is also what Tailscale call HuJSON, as in "JSON for Humans", which as amusingly also what json5 claims to be.
https://github.com/tailscale/hujson
https://hjson.github.io/
It has implementations in JavaScript, C#, C++, Go, Java, Lua, PHP, Python, Rust.
I disagree. Human-friendy multiline strings aren't really optional for a serialization format that will inevitably also be used as a config format sometimes because those are the same problem.
[^1] https://github.com/hemashushu/ason
YAML[0] supports JSON formatted resources and octothorpe ('#') comments as well. I didn't see anything in the spec specifically allowing for trailing commas however.
Here is an exemplar using the Ruby YAML module:
0 - https://yaml.org/spec/1.2.2/Allowing multiple types of quotes may be an unnecessary feature but it is a clear lesser evil compared to the mountain of footguns that YAML brings with it.
If you don’t enforce it, in the future the “subset of YAML” property might get weaker, especially if someone else is modifying the config.
If you treat config files the same as code, then using a safe subset of YAML is the same as using a safe subset of C. It is theoretically doable, but without extensive safeguards, someone will eventually slip up.
Use of indentation to denote nesting can sometimes be an anti-feature, too, because while using that the format does not provide a way to make certain that the entire stream has been read (parens balanced). This may lead to problems or even exploits.
Pure JSON is so painful for human consumption though, I willingly choose yaml if it's the only alternative.
JSON5 may indeed be a sweet spot between human-friendliness and lack of nasty surprises.
What is illustrated above is the definition of a specification-compliant YAML resource strictly using JSON constructs + octothorpe end-of-line comments.
Does this usage mitigate the concerns you have identified?
A good parser does not just accept the inputs you find valid, but also rejects inputs you deem invalid. Running a linter that would report or amend all the footgun-wielding features of yaml before parsing is tanamount to running another parser. Then why bother :)
Still, it would have been nice if YAML had reduced the surface area to verify in order to establish confidence in its content.
Depending what you use JSON for, "Numbers may be IEEE 754 positive infinity, negative infinity, and NaN." could be a huge plus.
On the other hand:
> leadingDecimalPoint: .8675309
This is just lazy. Can we discuss in depth how much time you saved by skipping the “0” in favor of lesser readability?
> andTrailing: 8675309.,
This doesn’t mean anything to me.
Trailing makes it a floating point type instead of an integer
Still… I can be required to put a zero in, read 0.3, and still think “that’s point three”.
How could the parser see it as a string? This is not YAML and JSON5 still requires quotation marks.
Is it universally understood? I think it's a US / English thing. In my country I've never seen numbers written in this way and many people would not "parse" it mentally as 0.8675309
You're still approaching this with the background knowledge of what this is. If you've never seen this, you can only guess, and there are a couple of options.
I've been terminally online for the better part of the last 2 decades, yet I've seen this way of writing for the first time only ~5 year ago or so, and I still remember simply not knowing what it is. The first reaction is that it's simply a typo - the author mistyped - the dot should have been a digit or perhaps not be there at all.
For most of the world that is even more ridiculous than using dot as decimal space separator or writing dates with month not in the middle place.
Even Americans I work with don't write it like that when doing quick draft discussion, as they know it's confusing to others.
I only know it's a thing because I watched some math-related edu shows. Didn't even see that when I briefly worked in US.
Come on, let's be fair—nothing else will ever be as ridiculous as that!
- "a gallon". Not purely US thing but almost. It can be anything from 3.8 to 4.4 liters, depending mostly on what are you measuring.
- writing digits. Is it "1", is it "7", is it "i", is it "l"? Why do it to yourself, while their printed fonts are pretty much the same as everywhere else...
It might be moderately intuitive to English native speakers because of oral usage like “point one three eight”, but that’s also not a thing in many other languages.
Reference: Postel’s Law
https://www.laws-of-software.com/laws/postel/
I think Postel's law was intended to apply to alternative implementations of machine-level protocols.
That's not to say that I don't agree that it might be better if JSON implementations would allow trailing commas, which is unlikely to lead to semantic ambiguities. That's too late now though, unless a new JSON to rule them all would appear and we would all agree on that new spec.
https://datatracker.ietf.org/doc/html/draft-thomson-postel-w...
https://datatracker.ietf.org/doc/html/rfc9413
I don't think Postel's Law was necessarily a bad idea, in its time and context. However it seems to have fallen into usage as an argument-ender. This RFC is a useful antidote to that.
On a personal level, I also don't like ending a number with a . because my brain immediately parses it as a sentence ender and it interrupts my reading flow.
It's just not wanting to keep track of more rules. If you've only ever used languages where a leading decimal point is allowed, it's a pain point to suddenly have to remember that it isn't here, and for no obviously intuitive reason.
It's about wanting to avoid unnecessary conceptual friction. Not lazy keyboard usage.
(Heck, your second example uses an extra keystroke. And it's perfectly readable to me, based on the languages I use.)
JSON5 is not.
There's something to be said for being flexible in your inputs when they are non-ambiguous. Particularly when dealing with files written by hand.
There's no virtue in imposing overly strict syntax when it serves no human purpose. That's trying to alter people to fit machines, rather than altering machines to fit people.
This seems like exactly how we ended up with YAML-and-the-kitchen-sink.
For a spec like this, things should need a better justification than that. Everything starts at -100 points. How does this feature justify the additional complexity in the spec, to the users, and to the people trying to implement this? How does it justify increasing the opportunity for creating subtly incompatible parsers?
Or, if a goal of JSON is not to be simple and interoperable, then I fail to see how it's not just YAML but 5 or 10 years behind the curve, so we may as well skip a bunch of hassle and move to YAML and start fixing the bugs we've got there rather than creating new ones here.
JSON5 is literally exactly JavaScript notation, which tons of people are familiar with. On the other hand, I don't know any programming languages that use strings without quotes or delimiters. (And of course, their existing in YAML is widely recognized as a major source of confusion.)
And in your example, there isn't parsing ambiguity but there would be change ambiguity when you alter abc (without quotes) to 123, because it changes its type.
I'm not aware of any change-ambiguity situations in JavaScript/JSON5 like that? In Python 5 and 5. are different types, but not in JavaScript or JSON.
The reason JSON doesn't have comments [1]:
[1] http://archive.today/8FWsAIt’s not like there isn’t another side to this argument.
This isn't kneecapping something any more than an inventor requiring programmers in a new language use types, or not use types, whichever the inventor deems preferable.
He invented a thing. He declared how the thing is constructed. That's not kneecapping. That's just defining a thing.
He invented a thing and left out a well known and understood core function required in the problem space, deliberately, not through oversight.
That's what makes it kneecapping.
The whole useful thing, ie the idea of a data format, including both knees (ie all the basic features any such thing needs) was there. The concept and necessity of annotation was a well known thing by then, and indeed he knew of it himself too, said so himself, and actively removed one functioning knee from that whole, to produce only the kneecapped thing.
He defined a kneecapped thing. Or he kneecapped the design. Whichever way you want to say it.
The difference would be if it was 40 years earlier and you are taking the fist stab at designing any kind of data format and comments just never occurred to you yet.
This is more like making a new programming language and deciding that it shall not have one of the basic math operators. If you need to multiply, tough shit, do it some other way. Just pipe it through JSMulti.
Are inventors required to only innovate a little bit? He released it, and there was every opportunity for someone else to offer something better and achieve the same level of uptake.
Trying to prevent programmers from doing something "because they may 'misuse' comments" is asinine to the extreme. This is not like removing a footgun, it's trying to dictate how you think programming should be done. This is particularly hilarious because I've instead seen tons of horribly hacky workarounds for JSON's lack of comments, e.g. "fake" object entries like "__previousKey_comment": "this is a comment for previous key", or even worse, putting two entries with the same key since the second key should "win", and thus making the first entry the comment.
As for his second point, "Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser." - just look at the everlasting shit show that is package.json since it can't support comments, because lots of NPM tools actually write to package.json, so there is no guarantee that other tools will be able to read package.json if it includes comments.
I think the thing that I hate most about Crockford's rationalization of not having comments is that you have to have your head stuck 10 feet in the sand to pretend that somehow the lack of comments is a good thing with the benefit of hindsight. I guess I could understand if Crockford's position was "Yeah, originally I thought it was better to keep them out because I was concerned about misuse, but now in retrospect to see the much larger problems it causes, and I realize it was a mistake." But instead he just keeps pushing his original crappy opinion on this topic.
I don't know who @fat is, but if he thinks Crockford's rationalization for the lack of comments in JSON is total and complete BS, I like the way he thinks.
Not sure why all the downvotes? Thought this was a classic.
An example I think every Node developer can commiserate with is that there isn't really a great way to add comments in the dependencies or devDependencies section in package.json, because those keys need to refer to module names, and that's where I most often want to use comments in package.json. I won't rehash all the details but just take a look at https://stackoverflow.com/questions/14221579/how-do-i-add-co... . Unfortunately none of the answers there are very good. In the past I've resorted to doing what https://stackoverflow.com/a/45815391/1075909 does, but that ends up not working very well once you've got a long list of dependencies, because the comment for any particular dependency ends up being far from the entry that specifies the dependency (instead of the line above), so people end up not reading it or forget to update it if that becomes necessary.
Yeah, I really, really hate that JSON doesn't have comments :)
They just preserve the key starts with $ for special use
So a class A with a double and and an int field will be something like
And what about keys that actually starts with $?You just escape it with special $raw key
It's a bit way too verbose. But given it is just a serialization format inside some library or app, it won't cause too much problems.>Trying to prevent programmers from doing something "because they may 'misuse' comments" is asinine to the extreme.
Programmers are often their own worst enemies. Some prefer rigid rulesets over less rigid freeform programming. See Typescript vs Javascript. No comments in JSON is just another example of over-rigidification, and some programmers love that.
>package.json since it can't support comments,
If you're needing to write comments in package.json, maybe you're not approaching package.json correctly? It's really for node things, not your things that you need to write comments about. I'm not even sure why someone would want to write comments in package.json. I get it with comments in other JSON files, but package.json probably should be left for nodejs things.
1. So what if JSON is primarily used for data interchange? It's not like allowing comments would somehow make them magically show up on serialized objects. This objection makes 0 sense to me. And heck, tons of other serialization formats (e.g. XML) support comments. Besides, there is a big reason that human-readable serialization formats are so popular - because they're human readable! If you're really worried about size you should be using a binary format anyway.
2. "Rigid rulesets" has nothing to do with Crockford's arguments. It's one thing to prefer a particular type system, or limit functionality if you think it has high potential for misuse. By JSON not having comments all you end up with is worse workarounds, like putting comments in object keys.
3. "I'm not even sure why someone would want to write comments in package.json" To be blunt, then, I can't believe you've ever written any code in a business (i.e. with multiple developers) in the Node/NPM ecosystem. Is it really that hard to wonder why someone would want to comment why they added a particular dependency to their project? The lack of comments in package.json is one of the biggest complaints of this format, and it's certainly not just me, e.g. https://github.com/npm/npm/issues/4482 and https://stackoverflow.com/questions/14221579/how-do-i-add-co...
Well then you and plenty of other people have some wrong ideas about package.json. That isn't surprising.
package.json gets rewritten for all kinds of things, which is not really compatible with adding comments wherever you want. Adding "why this dependency is here" comments may seem like a good idea to add to package.json, but you're kind of missing the point. If you need that level of documentation, trying to shoehorn it into package.json is just the wrong place for it. Soon enough your package.json looks like a graffiti wall.
>To be blunt, then, I can't believe you've ever written any code in a business (i.e. with multiple developers) in the Node/NPM ecosystem.
Then you'll be astonished that I have been working with nodejs for about 14 years professionally. Sure I have wanted to put comments into package.json, but I was naive and now I'm fine not doing that. I haven't wanted to in many years. I document things in other ways and it has served us all very well. YMMV.
Either https://en.wikipedia.org/wiki/Learned_helplessness or https://en.wikipedia.org/wiki/Normalization_of_deviance
So the right place is to make a graffiti out of another place, instead of in the place where people actually declare the dependencies?
I find it bizarre when people believe in one true way of doing things. I mean, you can declare your dependencies how you like, but if others do it differently, then they're clueless?
Surely that's what the commit message is for? I mean, I get that it's more convenient to have the comment right there in the file, but that should be balanced against the downsides: having to maintain the comment, making the file larger and more awkward to read, etc.
Yes, I still think a commit message is important, but it absolutely does not take the place of a comment. Suppose you'd like to do something like this:
There is zero chance that comment is going to be seen if its just in a commit message, furthermore you should never depend on git blame showing a particular comment because it's easy for later commits to overwrite it due to simple things like formatting changes, etc. Yes, in that example case there should be tests too, but IMO the primary purpose for comments is to highlight issues and concerns that aren't immediately apparent just by reading the code.I simply cannot think of another file format that is used for configs that doesn't support comments.
I never suggested using a commit message, there are plenty of other ways to document these things and I'll leave that up to the user to figure out.
So now you've written this tome of a comment in package.json. What happens the next time someone installs a package using npm install? package.json gets rewritten, that's what. Your comment will be gone. I suppose you expect npm to somehow use AI to guess at how to rewrite package.json so it can put your comments in the correct places??
And expecting someone to read package.json before updating or installing a library is just as useful as putting it in a commit message. If you really need to be careful about dependency versioning, you better have more safeguards in place than just a comment in package.json.
>I simply cannot think of another file format that is used for configs that doesn't support comments.
I mostly see configs created as .js or .ts files, where comments are allowed. Not package.json. Never package.json, because package.json is guaranteed to be rewritten regularly. But maybe you missed that part of your bootcamp class? Yes, .js files can be used as "config". It's been done in plenty of projects. It isn't the end of the world. JSON also isn't solely used for configs or data that needs comments, in fact the majority of use cases in the world for JSON won't need any comments at all so writing comments in package.json is kind of an edge-case.
>I simply cannot think of another file format that is used for configs that doesn't support comments.
JSON isn't only a config file format. It's primary use is data transfer.
Dude, I think you're lost, in more ways than one. I was directly responding to a comment that stated "Surely that's what the commit message is for?"
For the rest of your comment, at this point I'd rather have an argument with a dining room table. No shit you can't have comments in package.json now, that's the entire reason that issue https://github.com/npm/npm/issues/4482 is unfixable. If JSON supported comments from the beginning, then tooling would have to respect that, just like the bajillion other config file formats that support tooling that updates the config file programmatically.
speed = 100 km/h
weight = 60 kg
maxBuffer = 200 chars (between 100 and 300 works best)
output: {"speed":100,"weight":60,"maxBuffer":200}
It had walls of text with headings and something configurable sprinkled in. Crockford would be screaming. lol
[1] https://en.wikipedia.org/wiki/Whitespace_(programming_langua...
[1] https://docs.telerik.com/kendo-ui/styles-and-layout/sass-ver...
Sure you can write everything in another file or put the comments in the commit message. But out-of-band comments are more easily missed or lost. If the package.json got rewritten by `npm install` you'd lose the comments. Inconvenient, but that's trivial to fix at commit time.
You can make special key names that are really directions for something.
You can make enrire k v pairs that are never used by anything that actually parses the json normally.
Argument was invalid as far as I can see and calling it "sorry it makes you sad" is, wow I don't even know where to begin with that.
Having annotation happen in a dedicated place designed for it is better than having it happen where it was not designed to be, end of math problem.
For any spec there are people who want something spec doesn't do and people writing the spec need to say no to requests that they consider not in scope as much or more often than they say yes
Comments are not some weird thing one person wants for their weird reason that no one else needs to care about. It's like leaving out a letter from the alphabet.
Just acknowledging someone wants something out of scope is not an apology
Saying "I intended to make a defctive thing doesn't" doesn't change the fact that it's defective.
A car without windshield wipers is defective, or at best inexcusably limited. Saying "I only designed it to use in the sun" doesn't make it suddenly perfectly useful.
If you want to try to say that json was actually intended to only be used in special conditions like an exotic car with no roof, then it's fair for everyone else to say "Ok, well that intentional design is a crap design of limited use. People actually need a car WITH windshield wipers."
The simple fact that it's a text format invalidates all the attemped arguments that it's just a machine to machine data format never intended for humans to mess with. People aren't "holding it wrong".
CSV has no comments and is probably more popular than json.
If small parser was the most important, that's what binary formats are.
Or if the data needs to pass through a text handler that can't handle binary, we already had csv and a few different standard and cheap encodings.
csv's excuse for lacking some of the bare minimum features is that csv was first. Csv is from the time of typewriters and handwritten data. What would eventually become known, the most common needs, just hadn't been encountered yet.
A single person did not create csv after 40 years of seeing what is needed, and decide that csv shall not have an expected functionality in it's spec. There essentially isn't even any spec for csv. No one explicitly authored it. It's a typewriter format.
csv continues to get used after that point because of inertia. Once it's used many places, it continues to get used in new places because most new things have to interoperate with the existing ecosystem if you want to sell to the most possible customers.
Also csv, unlike json & similar, IS a pure data format, not used for things like configuration, because it's really only barely human usable, because the columns don't line up and there's no form of annotation other than a header record. Not to mention every part of the format, quotes, commas, even newlines, are valid data that may appear in a field, and so the only way to read the file is to manually reproduce the streaming state-machine in your head.
If something as limited as csv is so good and doesn't need anything else, then why did he invent json when csv already existed for jobs that narrow in scope?
csv is not remotely proof that text formats don't need annotation.
Your two examples are just two examples of why we don't need comments for data interchange: yes, you put the data in a trivial, stable position in the parsed data that all parsers can parse rather than write some sort of DSL that has to be extracted from comment nodes that may appear in various places in the tree.
Turning this:
Into this: Is the whole point. The more "abusive" you imagine the contents of "directive" to be, the more reason it should exist as the latter data, not more reason we should accept the former.Most json users are not writing all of the software that both generates and consumes the json.
This actually would put JSON in a worse place than XML - while XML has an overly complex infoset, that infoset is actually defined and standardized. Representing "a property with a comment on the property name and one before and after the property value" so that information is not lost in parsing would explode the complexity needed for an "interoperable" JSON library.
if someone wants to create some sort of scheme where they do "createdAt$date" as a property name to indicate the value is to be interpreted in some agreed-upon date format, that at least doesn't lose data if the JSON data doesn't understand that scheme, or require a new custom parser in order to properly interpret that data, compared to something like /* $type: date */ "createdAt" :...
This doesn't explode anything and you don't need to lose any data, so the monstrosity of XML still has no benefit, and neither does "createdAt$date", which would need a custom library anyway, so it doesn't matter where you insert your types
A bit off topic when the only consideration here is comments, but XML allows for type definitions and structures data that simply isn't possible in JSON.
People have found ways to attempt to add types to JSON but they aren't part of the spec at all and are just convention-based solutions.
I think roundtrip with comments is not feasible in general. Most code just expects a hashmap, which it edits and then serializes back. You would need some really clever handling for comments to do it.
Yes the comment free spec forces to normalize what would be comments into data spec and it is frustrating to put more effort into things
If you need additional fields in the json to hold comments, why not add the fields however you want? And if you need meta-data for the parser, you could add it the same way. In a project, i am working on right now, we simply use a attribute called "comment", for comments.
e.g. use "_" as a prefix to mark comments, and then tell you applications to ignore these attributes.
That may not be a huge problem for you, you see the "comment" key and know its just a comment and can ignore when a parser treats it like a normal string field. It could be an issue though, for example I could see any code that runs logic against all keys in the object becoming harder to maintain.
> If you need additional fields in the json to hold comments
I don't need additonal fields, I need comments
> why not add the fields however you want
Because I can't do that either, there are noticeable limitations
Syntax errors and erroneous highlighting are not even item 10000 on my list of JSON concerns.
Dare I pull a tired cliche and say “you’re using it wrong”
I don't think your take makes sense. Comments were purposely left out of JSON to mitigate the problem described by Hyrum's law.
Smaller interface means a smaller surface to allow abuse. You're just arguing to facilitate abuse because you feel the abuse that was indeed avoided by leaving comments out is unavoidable, which is a self-contradiction.
On top of that,think about the problem for a second. Why do you feel it's reasonable to support comments in data interchange formats? This use case literally means wasting bandwidth with data that clients should ignore. The only scenario where clients are not ignoring comments if exactly the scenario that it's being avoided: abusing it for control purposes.
If you want a data interchange format to support that, you should really adopt something that is better suited for your whole use case.
> and your suggestion doesn't help with eg syntax highlighting tools for your config that will treat comments as syntax errors (...)
That's the whole point. This is by design, not a fault. Why are you pretending a language supports a feature it never supported and explicitly was designed to not support?
If you take your law seriously, this is irrelevant because the surface of abuse is on the same scale of practical infinity, so it doesn't matter that one infinity is technically smaller.
For example, based of the example in the quote: you could stick those directives from comments info #hashtags in stringy values, with the same effect that there is no interoperable way to parse values (or if you add "_key_comment" - there is no interoperable way to even parse keys as some of them need to be ignored)
So the designer has achieved no benefit by removing a valuable feature
> abuse that was indeed avoided
Nothing was avoided, you can have the exact same abuse tucked into other elements
> Why do you feel it's reasonable to support comments in data interchange formats?
Why does the author of the quote sees the obvious which you don't see even after reading the quote? Go convince him his comment makes no sense because of "data interchange"
Obviously it's not only used for data interchange in cases where every byte matters (reminder: this is a TEXT-based format) and also comments matter to humans working with this data
> adopt something that is better suited for your whole use case.
And this discussion is literally about a format supports that? But also, how does this in any way mitigate the flaws in the designer's arguments?
> Why are you pretending a language supports a feature it never supported and explicitly was designed to not support?
Same thing, why is the author of the quote makes this senseless suggestions then? Go convince him first
Hyrum's law is not mine. It's a statement of fact that goes by that name already for a few decades and is noted by anyone who ever worked on production services that exposes an interface and is consumed by third parties.
> (...) this is irrelevant because the surface of abuse is on the same scale of practical infinity, so it doesn't matter that one infinity is technically smaller.
It really isn't. JSON does not support comments, thus they aren't abused in ways that sabotage interoperability. The only option you have is to support everything at the document schema level. You can't go around it.
> For example, based of the example in the quote: you could stick those directives from comments info #hashtags in stringy values, (...)
Irrelevant. You're specifying your own schema, not relying in out-of-band information.
That's exactly how a data interchange format is designed to be used.
There is no way around it. If you understand the problem domain, the fact that leaving out comments is an elegant solution to a whole class of problems is something that's immediately obvious to you.
If specifying the schema is important then why not an interchange format with strict schema enforcement like XML?
If minimal feature sets to avoid abuse are so important, then why not a binary format like protobufs which also have strict schema dictates?
And if interoperability is important then why not ditch REST/JSON all together and instead favor SOAP which strictly defines how interoperability works?
That's why I don't buy the "comments might be abused" argument. JSON doesn't have a single problem domain and it's not the only solution in town.
I don’t have anything against XML by the way, it was purely horrible to work with because people were using it in so many weird ways. Personally I prefer toml, but I guess we all have our preferences.
JSON wasn't some magical, made-on-the-fly format that makes Crockford some kind of genius. It was simply the standard Javascript object literal notation with some added constraints. I think some of those constraints make sense (i.e. are there any other languages that support both single and double quotes for string literals?), but funnily enough, some of the biggest issues with JSON interoperability is it is very underspecified in the areas that matter, such as the type and width of numeric literals, what to do in some edge cases like duplicate keys, etc. Just did a quick search, and here is a post that outlines some of the real security risks this underspecification leads to: https://bishopfox.com/blog/json-interoperability-vulnerabili...
Yes, quite a few like Python, PHP, Fortran, COBOL, Lua, R, Ruby, Perl, Bourne Shell, Dart, Groovy, etc.
Though some only interpolate with one or the other.
Sure it’s derived from JavaScript and it plays a major part in frontend development today. It was really the other way around though, when Ajax picked up people started realising that they could use json to make frontends work with “just” html and JavaScript.
All that needed to be said was, "Using comment programmatically is bad practice. Don't do it."
there's always someone putting actual logic in comments, and when I rule the world, those people are all going to be put on an island by themselves and they're going to have to deal with each other.
Your comment doesn't make any sense. You're just pointing out that developers designed data interchange formats as subsets of JSON which explicitly support metadata and comments. This means they are specifying their schemas to support these usecases in a way clients can and do support it. That, if anything, proves right the decision to not support comments because it resulted in better-defined and well-behaving clients.
We get to choose what approach we take. I prefer the "give them the rope they might need" philosophy. If they hang themselves, oh well.
That said, I have yet to see anyone ever say "Oh man, this programming/configuration/data transfer language would be so great, but I wish they hadn't supported comments!" I see the opposite about JSON all the time.
And I don't see this about other features, e.g. Go specifically decided not to support the ternary operator, and while some folks may like that feature, I think more folks really appreciate that with Go there is generally a "standard" way to do things and it doesn't give you a million different ways to essentially do the same thing.
https://www2.jwz.org/doc/cddb.html
(Unfortunately, because jwz blocks HN referrals, you can't click on this link, but will need to copy it into the address bar)
And multi-line strings. You don't always need that, but when you do, it's absence is very painful.
"//key":"this is here so that foo bars", "key":"value",
valid JSON. Most software handles extra propertiesjust fine
Don't get me wrong, the ubiquity of JSON speaks for itself and is the reason to use it. But, to say it has tangible benefits feels very dishonest.
Then take a look at the JSON spec: https://datatracker.ietf.org/doc/html/rfc7159
The one thing I really wish it had was some form of multi-line string, mostly so I could use it with line diffs. Also sometimes for really simple property editors it's nice to put the serialization in a textarea, and that works fine for everything but multiline strings.
(I suppose you can escape newline characters to put a string on multiple lines, but I find that rather inhumane and fragile)
Sorry but what is the benefit of this? Lazy shorthand? This is too much. Is this a string in other languages? PHP the `.` is a string concat.
What would be the advantages/disadvantages?
https://hjson.github.io
If I were doing a professional project, I'd be hesitant to use it over something with more popularity and support. The syntax has so many variations two files can look like two totally different config languages, which is both cool and alarming.
There is a std json library as well but the aesthetics weren’t great imo.
The specs are quite pleasant to implement.
0 - https://github.com/lightbend/config/blob/main/HOCON.md
I agree it would be nice to have something with more data types. Binary b64/hex would be nice.
When you read 847548, is that a number or is that Saturday, 10 January 1970?
Having a type removes that ambiguity. It would be more JSONish for it to be human readable, maybe @1970-01-10T19:25:48.
Mongo types the field name, `{"$date":1623132000}`, if I'm not mistaken. Rust style would be `1623132000_unix`.
Or with anticipation of a more full typing system, where the time is explicitly named:
`{"now": unix:1623132000}`
For now, when I need typing, I use https://json-schema.org
If you need more, you enter territory that is much too complex to build into the "simplest data format" spec.
As an (unfortunate) JavaScript developer, it was clear to me the intent was to "update" JSON with ES5 features, and not say ES4 or ES6.
Why ES5? ES5 is when trailing commas were introduced. Commas are one of defining features of JSON5. Other languages, like Go, also made this a priority.
We already have 5+ replacements that are far more robust(XML, YML) and IMO they are not great replacements for JSON.
Why? Because you can't trust most people with anything more complicated than JSON.
I shutter at some of the SOAP / XML I have seen and whenever you enable something more complicated inevitably someone comes up with a "clever" idea that ruins your day.
Extra hint:
Edit: it's extremely unlikely a yaml parser implements that spec; the spec is irrelevant.Your claim of "No, it wouldn't" is simply wrong. They said "since 1.2" and you can't just ignore that when you're citing a 1.1 problem. The disclaimer that you edited in gets at a relevant point but it's not strong enough to make your claim actually be true.
And the winky face and the "hint" just make things worse, since they knew exactly what you meant.
I don't think that's the primary reason. JSON is pervasive because it started out being trivially parseable by JavaScript going back to when people just evaluated it, even before browsers had ridiculously high-perfomance safe JSON parsers. All the other formats are still harder to work with from JavaScript.
If not for that, personally I'd advocate TOML, which is incredibly simple.
Comments and all, ironically.
I mean sure, "and all" would frequently include script injections etc, but you can't argue it wasn't more feature rich!
1. Comments 2. Trailing commas
We don't use any other JSON5 features, which are primarily just that numbers may be encoded in hexadecimal and field names may have quotes elided.
We typically encode values with RFC 4648 base 64 URI canonical with padding truncated (b64ut) with values too large to be a JSON number, so hex isn't useful anyway. We haven't found that omitting field name quotes is a big deal.
JSON5, in contrast, has an actual spec that has been stable for 6 years now.
Every time I learn a new format I want to scream "why can't you be normal?"
https://cuelang.org | https://cuetorials.com