200 comments

[ 2.6 ms ] story [ 240 ms ] thread
For what it's worth, yaml is JSON that allows comments (and a bunch of other syntax).
And ambiguity. It's too far in the other direction IMO.
TOML is one-to-one with JSON, except allow comments.

I like JSON lines format where I can strip comment lines out.

Just say no to yaml
I like yaml, easily readable by computers as well as humans (without any coding experience even). I don’t see the problem with it.
> easily readable by computers

Have you ever written or tried to write a YAML parser?

It's already done, so no need.
I will forever be aware of the pain of the developers of pyaml whenever I `import yaml` from now on. And even more grateful too.
It's actually not easy for computers to read YAML. It's very difficult, especially when compared to JSON.
My computer has little problems with it!

Ok, we're being pedantic, perhaps it depends on how much information you put in a Yaml file, I usually see it used for small config files.

And is so pedantic about whitespace that you need to specifically configure your editor to be able to retain your sanity while editing it, no thanks.
Significant whitespace owes me many days of my life.
What? Just write it in JSON format.
That is _possible_, but in the real world if your team has chosen YAML, it’s not because they want to write more reasonable JSON. They want to write YAML.
yaml is a superset of JSON, which means that any parser that expect yaml will happily consume JSON!

Also JSON + comment that's JSON5.

I used to do this frequently. Take a JSON file, add comments, and parse it as YAML.

Then I discovered a handful of YAML parsers in various languages support code execution on parsing. Which basically killed YAML for me.

It's definitely no good for JSON that you don't write yourself. But that's not the JSON that you want to write comments into.
I appreciate that Python makes you choose between a “full” parser and a safe one.
At least in Python, YAML parsers are orders of magnitude slower than JSON parsers.
I switched out my app config to YAML which parses much slower and now my app starts exactly as fast as before.

I don’t think anyone proposes switching to YAML for your API endpoints

If you have 1000s of lines of yaml you're restarts will be delayed by multiple seconds. So YMMV.
I mean I can imagine such a slow yaml parser but have never experienced it
Using YAML will make you a more spiritually advanced being because every time you copy and paste content into a yaml document it requires a prayer.
I want to blow my brains out every time there is a mysterious tab indent in my file
What comment syntax does he propose?

    //

    /*  ... */

    #

    --
He didn't propose one. But given the "JS" part of the "JSON" acronym, probably only the first two are reasonable.

But the point is, if you are in control of your own parser, you can just use any one of those.

JSON is not JS, says Doug, too. Since he said "any new JSON should not new JS or ON in the name, since we claims it is not a javascript object notation, in the end. Perhaps since JS/ES is changing, and not everything is an object in every language that wants to deal with JSON?
I don't exactly understand what you're talking about but RFC 8259 explicitly says JSON is "javascript object notation".
(comment deleted)
> probably only the first two are reasonable.

Yet YAML, which is somewhat related to JSON, uses

    # This is a comment
I'm neither pro-comment or anti-comment, but I'm all for standardized comment syntax to avoid parsing hell.
That's basically a new standard. I like it, but getting all JSON parsers on board will be tricky if not impossible. Maybe it deserves its own name. How about JSONc (JAYSON-SEE)?

Edit: I read the fine print on the page after posting this... Turns out that I'm not the first to come up with jsonc. The author just isn't a huge fan and wants JSON parsers to accept comments. Good luck with that!

but my user can't write a browser
You don't _have_ to enable Mallett's Law.
JSON as a data-format should not have comments. JSON as a file-format should allow comments. The problem is this conflation between the two.
Can I confirm that the reason it's not preferred to have comments in data-formats is because it's to be machine read only and as such should be as efficient as possible and not contain information that wont be used?

Seeing as I can only see the use case as a file format to be read/written by humans in the loop, then maybe the conversation should be about compiling the file format to a data format for compatibility outside of the user tooling.

> is because it's to be machine read only

Why did they bother making it text-only ASCII then ?

> ASCII

JSON is notable for making UTF-8 encoding a hard requirement.

…which was pretty ballsy back in the mid-2000s. We were still fighting with Shift-JIS and Windows-1252. Excel didn’t add proper support for UTF-8 until depressingly recently.

Late 90’s I had to fix bugs in a shiftJIS implementation. And I couldn’t read a lick of Japanese. Still can’t.

I don’t remember when I started pushing for utf-8 everywhere but it was “early” by most people’s standards, so I know what you mean.

And one of the things that makes me dislike MySQL is that they have a field type called utf-8 that isn’t. And they didn’t fix it, they introduced a new type instead. So that footgun was still there for all to trigger. So mad.

JSON does not require UTF-8 encoding.
> JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8

https://datatracker.ietf.org/doc/html/rfc8259

Ah ok, fair enough. This is a more recent (2017) clarification of the standard which I hadn't seen. The original mid 2000s specification did not require UTF-8.

> Previous specifications of JSON have not required the use of UTF-8 when transmitting JSON text. However, the vast majority of JSON-based software implementations have chosen to use the UTF-8 encoding, to the extent that it is the only encoding that achieves interoperability.

The original spec did require that all JSON decoders support UTF-8, though.
Hmm, not as I read it. It says that UTF-8 is the 'default' encoding. In context that just means that it's the encoding you assume if the first four octets don't match a pattern characteristic of one of the other encodings (when restricted to ASCII characters). See section 3 of https://datatracker.ietf.org/doc/html/rfc4627. The original RFC is vague, but I think the idea is that a fully conformant implementation would support all the encodings mentioned.
Pretty sure they meant plaintext instead of ASCII.
JSON wins because it can be casually inspected by people testing bizarre theories. The importance of this is lost on people who don’t treat triage as a skill that can be honed.

I like to solve problems - or at least bringing them to me doesn’t result in a loss of status for either party. People notice this about me and bring me problems. Someone recently described to people what is essentially my process: the likelihood of the cause divided by the difficulty of verification. Partially sort and just start checking off assumptions.

A lot of cheap but low probability options get shuffled higher, and just sending the wrong data is a common enough problem, especially with caching. And if it’s nearly free to look at the payload, it’ll get checked. If it isn’t people will try everything else to avoid it.

(comment deleted)
The argument is that comments are often used as an escape hatch from specified formats to carry further instructions. So you got a properly specified format and then want to do vendor&extensions but not break other implementations ... just make your extensions a comment. Then other parsers ignore it and you can do your thing.

The idea is that this forces better formats.

How well this works? Well, then I got an "x-comment" property or non-standard comments. Nonetheless. If people see the need to hack some extension in, they'll find a way.

I think in the JSON case its because you can't have true comments, any comments are intrinsically part of the data structure, and you invite problems by including irrelevant information
The problem is using JSON as a file format in the first place. It’s not designed for humans to edit. (Then again, it’s better than the Norway-sceptic YAML.)
I disagree. At least in an ought vs is sense: it's entirely the kind of format that I would create as an editable format. As witnessed by the fact that my workmates and I did create very nearly JSON previously as a file format in the 90s (but for C code programs)
For a very narrow target audience (programmers), JSON is fine. If you want the file to be edited by anyone else, JSON is pain.
(comment deleted)
But it happens. 'npm install' will edit your json file, but so can I.

That said, I don't like it as a config file read/written by humans.

What example(s) of file format would you say are designed for humans to edit and still represent the kind of structured contents that json does?
I can’t think of anything that is not painful in some way.
TOML, extensions of json like json5 and hjson, a bunch of lesser known formats for nested structures like NestedText, UCL, kdl, Eno,sdlang, eldf, etc.

Also languages with some progrommatic capabilities like cue, dhall, jsonnet, nickel etc.

Non of them are perfect, and some are less suitable for certain use cases than others. But IMO pretty much all of them are better for human editing than json, and in many cases yaml.

Any format that:

- doesn't require to quote everything

- has lists/dictionaries

- uses indentation and new lines instead of commas and brackets

- doesn't have 1000 unnecessary features like YAML

Also, you don't need all types from JSON.

> you don't need all types from JSON

JSON has a very minimal set of types and I regularly use all of them. I guess you could argue that integers and numbers could be combined, but I think that's it.

But they're not two different formats—they're two different jobs being done by the same format.

JSON as currently spec'd is honestly quite bad at both jobs, but the most rational defense of its use as a data format is that it's (mostly) human readable. Given that that's its main value proposition, what exactly is the reason for saying that JSON-as-data-format should not have comments? What do we lose if we allow them?

I think json should allow comments.

But there are dangers there - look at how horribly comments get abused in code:

* doctests are nonsense, just write tests. (doctests like rusts that just validate example snippets are the closest thing to good I've seen so far, but still make me nervous).

* load bearing comments that code mangling/generation tools rely on (see a whole bunch of generated scripts in your linux systen - DO NOT EDIT BELOW THIS LINE)

* things like modelines in editors that affect how programs interact with the code

* things like html or xml comments that on parsing affect end user program logic.

Comments can be abused, and in something like JSON on the wire I can see systems which take additional info from the comments as part of the primary data input. Often a completely different format... and you end up with something like the front-matter on your markdown files as found in static site generators.

Point being, comments are not a purely benign addition.

How is abusing comments any different than abusing a top-level property with a key like “__comment”?
at least a top-level metadata property can be explicitly defined in a .json.schema[0] and formalized, rather than being some kind of ad-hoc pre-processor step you have to evaluate before actually using the JSON data. I didn't even know about that approach before I read your comment but it instantly makes more sense to me in terms of maintainability and interoperability.

[0] https://json-schema.org/

I feel like most of your examples of "abuse" are just getting things done.

I don't see anything intrinsically wrong with doctests. I also can't see a better way to do "load bearing comments," and I'm not eager to go back to "Step 2: Edit your .bashrc to include foo."

> see a whole bunch of generated scripts in your linux systen - DO NOT EDIT BELOW THIS LINE

these are mostly a warning sign for humans, to be read as "if you need to modify the script below this line, a) you gotta be knowing what you're doing, we are not held liable for support if you change stuff around there b) please contact us to make sure we didn't miss a legitimate need or c) you're trying to do something in a bad way and there's better ways to do so".

> doctests are nonsense, just write tests

Why are doctests not tests?

> doctests like rusts that just validate example snippets are the closest thing to good I've seen so far

Rust's doctests don't seem to be fundamentally different from Python doctests, which is the language I've seen most commonly make use of doctests.

> but the most rational defense of its use as a data format is that it's (mostly) human readable

I would call out portability instead, which is not dependent on the byte ordering or endianness issues of binary data formats.

sort of like: javascript is portable code, json is portable data.

> Given that that's its main value proposition, what exactly is the reason for saying that JSON-as-data-format should not have comments? What do we lose if we allow them?

Because JSON originally did have comments, and people were putting pragmas into them, and so different parsers would act different depending on whether they understood them or not. Comments ended up being an anti-feature in JSON because people were abusing them.

Source:

> I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't. […]

* https://web.archive.org/web/20190112173904/https://plus.goog...

What are some examples of people doing this?
I don't buy it, what's stopping people from putting pragmas in key:value pairs? There's a chance of collision, but you're already deciding to sacrifice interoperability, so just accept that the myJson spec says '___declare___' is a reserved key.
If I parse json, I dont want to lose data. Having the parser read the comments (however they are, as long as they are in spec and therefore read by the parser) is a good thing. Having to parse the file again, with a fuzzy out-of-spec system (looking for comments) is clearly worse. The whole point of json is to serialize stuff, breaking that to insert non-machine readable comments makes the spec less reliable.
You cut the rest of his comment

> Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.

Doesn't seems he is that against the idea

The only reason it became popular is that conflation!
Disallow comments and now you just have {"comment": "the quick brown fox.."}, the worst of both worlds.
That's a harmless example and a tiny price to pay.

What no-comments saved us from was stuff like this in our data interchange:

    {  
        "count": 123 // bigint
        "price": 10.99 // @precision=2
        "date": "2024-08-12" // @format=YY-MM-dd
        "data": /* !transform(rot13) */ "uryyb" 
        "storage": 5 // Unit(TB)
    }
And who knows what deeper layers of hell we avoided.

Frankly, VSCode shows that all this time people were complaining about no comments in JSON config and how hard it was to write config in JSON, they could have just written their apps to strip comments at read time.

So we do have the best of both worlds.

Your example is perfect, I'm stealing this for the next time JSON comments comes up.
> JSON as a file-format should allow comments.

JSON is awful for writing manually because it requires typing too many quotes, commas etc. I think JSON is meant to be machine-generated and machine-read and therefore doesn't need any comments.

You're a programmer and you're against writing quotes and commas? You must really hate coding. I've never found JSON to be too much typing.
If you're entirely machine writing and reading, but still want to be human-legible, then XML does a much better job while also allowing for schema.
Thinking from the first principles:

1. comments are metadata (specifically Human/LLM-readable metadata vs machine-readable metadata)

2. general-purpose data formats should support metadata

I think it’s more “it would be nice if JSON intended to be read or written by humans allowed comments”.
You can also put JPEG binary data in RSS feeds, send a string of emojis for an SSL handshake and stick HTML 1.0 directly in a docker image. You can do those things. You can even write tools that accept those things.
All you're doing is creating a new spec that's not JSON. JSON has no comments. Your spec does. Your parsers handle both specs.

https://xkcd.com/927/ applies

Another advantage of JSON5 over JSON is that extra comma at the end of a list is not an issue.
Isn't this the problem that JSON5 (and probably other similar projects) is supposed to solve?

Both JSON (as defined in the RFC) and JSON5 have a nice property of being well-defined, meaning that you can use different libraries in different languages on different platforms to parse them, and expect the same result. "JSON but parser behaves reasonably (as defined by the speaker)" does not have this property.

http://seriot.ch/projects/parsing_json.html

"Despite the clarifications they bring, RFC 7159 and 8259 contain several approximations and leaves many details loosely specified."

Nothing will probably ever top Markdown in my mind for bullshit specifications.

And Gruber wouldn’t give Jeff Atwood permission to call his variant <something> Markdown, or it seems anybody else, so we ended up with CommonMark, and GFM.

Json5 is good for JSON at rest, as others have mentioned already.

JSON5 would be ok if that's all it did. They added so much additional unnecessary complication that it undermines the simplicity of JSON that makes it good.
(comment deleted)
Lol “If Douglas Crockford actually is your dad then please do not read this web site.”
One man’s extremely horrible mistake and look how the whole industry has been suffering for decades - countless articles debating the existence or absence of comments, new formats, new parsers, new ways of adding comments, post-processing, etc. What a pity!
and also life of LLMs would be so much easier, all the JSON content, so much missing context!
It’s better than having to deal with the ad hoc out-of-band data formats that people would inevitably have stuffed into comments if given the opportunity.

Remember how JavaScript used to reside inside a HTML comment?

Yes, but you expect much better from a seasoned engineer.
Definitely. It's important that every ad-hoc out-of-band data format look

   "like this"
never

   // Like this
Bad Things will surely befall us if we break the taboo.
Why would a comment be required for this when they can just use a string property? YAML has comments, and they are not used by any DSL abomination that I'm aware of.
Conda package recipes have a preprocessor using "selectors" in comments to conditionally exclude certain lines in a YAML file. Not that this is particularly more ugly than other YAML DSLs, but it has been used.
> It’s better than having to deal with the ad hoc out-of-band data formats that people would inevitably have stuffed into comments if given the opportunity.

As has been pointed out multiple times in the thread so far, a random string that is ignored by the receiver is semantically equivalent to a comment, and people totally stuff random crap into strings to extend the data format in sometimes-horrifying ways. Banning comments doesn't stop us from abusing strings, it just means we don't have a good way to indicate that a part of the file should be ignored by the parser because it's not actually data.

But the problem isn't with abusing strings, but abusing something that isn't data.

Abusing the string is the correct place to put poorly planned data because it's just data and it has a stable position in the data (and syntax) tree.

But consider this example:

    {
      "a": /* directive */ 42,
      "b": 42, // directive
      // directive
      "d": 42,
      /* directive */ "d": 42
    }
What does the code look like that has to reconcile the directive with the data it modifies?

That's a completely different and worse problem than someone "abusing a string" with something like `{ "a": "42 directive" }`.

I think JSON made the right call. People just confuse that with it being somehow forbidden to use comments in JSON config, but they're wrong. Just look at VSCode's config. It was possible all along despite decades of people whining in blogs.

We didn't need comments in JSON over the wire. We didn't need a new format. We didn't need another blog post about it. Just write your tool to accept comments in its config.

I don't understand what the problem is.

You can just make a parser that ignores comments, while still allowing them in the syntax. You don't need to store the comments in the AST, or to actually parse them. You just need to skip them.

That only works if you control the data that gets parsed.

Imagine you’re a HTML parser library author in 1998. You’ve been happily skipping comments. Now you get complaints that your parser doesn’t see JavaScript. Turns out that everyone has agreed that these new script tags should be embedded inside comments.

Should you keep skipping comments and tell your users you’ll never support this HTML that everybody else now considers valid?

Should we not allow comments in HTML, then?

There are parsers which allow for comments if one wants, so if someone wants to engineer an insane system no one is stopping them, provided they can ensure the parser on the other end of it.

This is a data format meant to be readable by humans. As such, it's natural to want to support things like configuration via end-user editing values in the text as a use case. Data is occasionally going to need comments to explain valid options or add context for people to edit. This is a reasonable thing to have.

We should’ve kept XML.
Right? People forget history faster than ever.
I keep repeating this.

We already have XML. Stop trying to make JSON another XML.

I'm now knee-deep in a gig that uses all sorts of w3c standards, which enforce json-ld, json-schema, json-whatnots and so on. It's a terrible mess of unreadable complexity that has LONG been solved in XML. Decades ago!

I keep thinking that if you need one of those "fancy" features, why not just use XML? Why bolt them onto JSON, a format that was deliberately (?) set up to not have all these features?

I think that if you don't need these features: by all means, use JSON. It's simpler, cleaner, easier. But that it's only simpler, cleaner, easier because it's not XML. And that all these attempts at making it XML, end up with a JSON that's harder, more complex and often messier than the XML that we had for decades and that JSON tried to "solve" by being simpler. So it's one step forward, two steps back.

We should’ve kept S-expressions!
There's something to be said for the hard limitations of json that made it successful.

Because of that it didn't become XML or YAML or markdown or protobufs or *RPC or microsoft config file format or all the rest. All those formats have "reasons", but they are harder to understand, harder to parse or not as portable or on and on...

(that said, I wouldn't mind comments. JUST comments).

Most reasonable parsers has an allow-comments flag, and even in JS you could it in a simple regexp replace all with:

const obj = JSON.parse(jsonText.replaceAll(/("(?:\\"|\\.|[^"])"|[^\/])|\/\/.|(\/)/g,"$1$2"));

The above should be multiline JSON safe and give you single-line comments to end-of-line, it matches first strings (and checks for escaped double-quotes and escapes to get them correctly) or all other characters except // sequences (outside of strings) and passes that straight through via group 1(anything outside of string not starting with /) or 2(single / even if it's outside of JSON spec), if a // sequence is found it's not passed through and disappears.

Yes, regexps can be abused. This one should be fine though but only use it for config files you control :), use as CC0 and keep an keen eye if translating to another language since escapes will differ if the regexp goes into a string instead of a regexp literal like in JS.

Whats the comment syntax they want? I didn't see it on the Lua page. Is it not on the page?

{"comment" : "This works everywhere doesn't it?"}

Json5/jsonc one, so:

    {
      "A": 1 // Don't remove this important but undocumented value
    }
Only if you don't also validate JSON schemas.
Wayyyyy before comments, the one thing I'd want to see JSON support is effing trailing commas.
This one unambiguously seems like a mistake. Is there any steel man argument for why an extra comma should not be acceptable? Save a byte over the network?
Ecmascript is changing JS. JSON is old and unchanging. Which is an advantage and a disadvantage.
I think that’s literally the only argument for it. And if you use compression, now you have to waste an extra symbol on `”` in addition to `”,`, so even that’s debatable.
“ is already a symbol, but for formatted JSON it’s more likely to be “,\r and “\r
I bet `: “` is more common than a lonely double quote, except at the end of a list of quoted values.

Eh, who knows? But I’m skeptical that the omitted comma actually saves any space in common situations.

{"comment":"This valid JSON document totally has a comment."}

    char str[] = "That's like saying this is a C comment";
(comment deleted)
Possibly not the optimal way to comment code, but sure.
Why am I always the last to find out
// or /* */ comments?

Doug, please adopt me!

As a file format, just use jsonc or json5, if you want to use comments.

Do the people who think JSON should have comments also think the CSV format should have comments?
For my part they are for slightly different use cases. While I seldom see CSV files being used for things like configuration and markup, I often see JSON used. And a lot of times configuration choices require explanations.

It's not a binary choice. I am more in favour of JSON comments than I am for CSV comments, but if I were to need them, I would probably use the Emacs style:

https://github.com/emacsmirror/emacswiki.org/blob/34edac6f86...

If JSON had comments, people would start putting data in comments (mostly annotations probably). What will be next? Allow apostrophes for strings? Add schemas so that parsing JSON is getting blocked by firewall rules? No thanks.
People are already putting comments in JSON, it just goes in a "comment" field. So now resources are wasted processing it as a property instead of ignoring it.
By putting it in a "comment" field the comment survives round-tripping, which is a virtue rather than a vice.
Already explained in other comments. Devs can put whatever they want in the comments; (third party) parsers won't need to change anything, because they still treats those "special" comments as regular.
> Douglas Crockford Is Not Your Dad

No, he's your drunk uncle who sounds smart when you're young. When you learn to form opinions of your own, you realize that his opinions are worthless hot air.

Many years ago I went to a session where he promoted asynchronous Javascript as the "one true way." In the Q&A, I pointed out that callbacks made code much harder to read and maintain compared to threaded code; and then asked if there was a cleaner way to do it.

His answer was quite rude. A few years later, we got the "async" keyword, which solved the problem.

As soon as you lose stack traces things become so much harder to debug
Deeply nested inline anonymous function callbacks are the anti pattern. Decoupled explicit named functions passed as callbacks are a significantly better developer experience and provide clearer stack traces. I never understood why this conversation didn't take the lexicon and instead the imo inferior `async` pattern got pushed hard.
Extracting callbacks into separate functions is just more indirection.

And they don't solve the core issue of async control flow that async/await solves, so it's not an alternative to async/await much less a superior one.

A classic example is when you want to conditionally do something asynchronously like B() in this case.

    function process(id, callback) {
      A(id, (result) => {
        if (result === 3) {
          B(() => {
            C(result, callback);
          });
        } else {
          C(result, callback);
        }
      });
    }
Versus:

    async function process(id) {
      const result = await A(id);
      if (result === 3) {
        await B();
      }
      return C(result);
    }
Add a couple more layers of this and the async/await function stays simple and flat, and the callback version grows significantly more complex and nested.
Without the async keyword, it was far superior to anons. Also, named functions showed up on stack traces making it easy to know what is getting called

No serious person is going to claim that any form of callbacks is superior to async/wait

Don't forget having to use try... catch for exception handling. This is why promises are the one true way!
> he's your drunk uncle

The way you've bitterly dwelled [0][1] on something Crockford once said about async during a Q&A for 15 years or something is maybe a little bit weird.

[0] https://news.ycombinator.com/item?id=41142972

[1] https://news.ycombinator.com/item?id=22365466

I think it's weirder to dig though my comment history and link to it.
I remembered your recent, similar comment in a Crockford says something mildly controversial thread and looked to see how many times you'd commented on the same incident. (of course I expect I missed a few of them)

There, I've explained myself, and I think the explanation is relatively straightforward. Now you. Why do you obsess bitterly over something Crockford said to you fifteen years ago during a Q&A? Perhaps more importantly, have you discussed Crockford with a therapist?

>I went to a session where he promoted asynchronous Javascript as the "one true way."

Note that your question didn't challenge this point (Async javascript is the "one true way", given that blocking in browser context always degrades the user experience).

If we had not accepted async we would almost certainly got threading support in Browsers.
Threads in browsers sounds like an absolute nightmare of new browser bugs and dev issues
He was promoting Node for server-side programming. Every other framework was either multiprocess or multithreaded.
Everyone already know which xkcd applies to this, right?
Aren’t comments already part of JSON5?