All of these ideas are good (from a consumer perspective, it's too early in the morning to think about implementation issues)...but I have to ask out loud what I frequently just think...why don't we just start adopting YAML more? Or, at least, a subset of the "safe" kind of YAML, mostly restricted to what JSON can do?
Sure YAML is more complex than JSON, but that's because it's more than just a data structure. If you are using it only for things that JSON can do, it's really not that complex.
Agreed. I use a YAML file to track my daily agenda. It was easier to figure out the format, and to leverage existing parser libraries for my reports, than to write my own "agenda-optimized" file format. A good example of http://xkcd.com/1205/.
As a JS developer, I use JSON because I can just put it directly in my code as a variable.
Annoyingly, "x = some_json" is much less strict than "JSON.parse(some_json)".
Whilst YAML is less ambiguous, the significant whitespace and not being able to just cut and paste an argument into usefuleness (without wrapping it in a parser) means the effort is too high for what little pay-off it provides.
I don't know about your comma statement. I never have a problem writing symantically correct JSON by hand - though there are limited use cases where you should need to do so anyway. If your IDE isn't marking invalid JSON syntax you need a better IDE.
Yes, exactly. But it's not even meant to replace JSON. It's not for exchanging data between computers, and its not for webservices etc.
Its for when you need a quick config file, and you think "Gosh, I wish I could just put some sloppy JSON/Python-like code in here, with a few comments.", and you don't want to use XML, INI, or YAML, or roll your own format.
I guess most people here are in the web development business. Which is fine, but I am not currently. I don't need JSON for what JSON is made for, but I often need something almost like JSON for my use cases.
Just use JSON, and add a key (which you can ignore!) for comments. Even better, you get to use any of the dozens of already-tested JSON parsers out there for this! There is absolutely no reason you need to reinvent the wheel here.
You know what's annoying? Having to remember a dozen-odd JSON-but-not-really config file syntaxes instead of just JSON.
> You know what's annoying? Having to remember a dozen-odd JSON-but-not-really config file syntaxes instead of just JSON.
That's why YAML is out of the question for me. But JSON is too restricted to write it manually (no trailing commas, no comments, no round-trip-safety). What I'd ideally like is to take this Python/JS5 file (differ only in the comment indicator # vs //)
# this is an important comment
obj = {'hello': 'world',
'list': [1,2,3,4,5,]}
...
from configfile import obj
and to turn it into a safe
// this is an important comment
{'hello': 'world',
'list': [1,2,3,4,5,]}
...
obj = parser.loads("configfile.cfg")
It's not that I don't like JSON. Its just that I have a different requirements. When I need JSON, I use JSON. When I need easily editable config files that look like Python or JS (which I can easily remember), I use something like HJSON.
It's a human friendly (to some. It's ambiguous, so I don't think it's very friendly) , but it's not JSON and doesn't offer any of the advantages of JSON.
YAML has an enormous amount of odd features that go beyond simple data types, similar to XML (still seeing XXE here and there).
I like the idea of a slightly simpler JSON but still keeping just the primitive data types yet not being able to say, include arbitrary files into the document.
Think of this as an alternative to YAML for people who deal with a lot of JSON and don't want to always be switching back and forth between JSON and YAML (or toml or edn or whatever) with all the cognitive friction that can cause.
They just want to be able to write sloppy JSON and have code handle all the strict requirements of real JSON.
YAML was ruined by whatever committee controls the standard. Thanks to extra features, that were implemented by most parsers, it's not safe to parse an untrusted document.
Some people are pushing for TOML as an alternative but I'm not sure if that's going to happen.
It would be nice if JSON accepted comments and Python/ES5 have proven the developer ergonomics of trailing commas, but I'm not convinced the rest of these are improvements.
Many implementations I've seen will raise a syntax error if you include a // in JSON. Then, you have to do clever shit like make a key that you don't use anywhere:
Sorry, was there a problem with JSON that I wasn't aware of? Oh, you mean, the problem of following the spec or not paying attention. Ok, in that case I see high adoption rates for this. Or, we can just make sure we do follow the spec and pay attention to what we're doing and stick with what works and is practically an industry standard.
Yes, there is a problem. Many programs use JSON for config files. You can't really comment stuff in there, and if you change something programmatically, you loose all comments and layout. A major example is Sublime Text, but I also heard the package management in node.js uses JSON, and people complain about not being able to include comments.
Actually, all the JSON I produce at work is per hand. It would be a major productivity increase if I had a more lenient parser. You don't have to call it JSON if it offends you, call it "Object literals that look like JavaScript or Python" if you like.
I still think that either said config files should use another format, or people should adhere to the spec. For this to have massive adoption new parsers will be required for all major languages. The problem it solves is not that great to merit new parsers I don't think. Of course, I could be completely wrong and this will become the next standard, but I doubt it.
The point of JSON is to have a standard way for data-interchange. Adding the complexity of "sometimes you might have quotes, sometimes not" and "you can display multi-line strings in different ways" gets away from this.
JSON is great already. I don't really see this as necessary. I could understand the comments (but use // instead of # as per Javascript comments) to increase clarity to people who don't know the original design syntax. But the others do not seem really useful.
No, the point of XML is to have a standard way for data-interchange. The point of JSON is to have a lighter and simpler way of data interchange, where the data is arrays and maps of strings and numbers. It also works well with Javascript.
If there's one thing JSON has proven it's that having a standard way for data-interchange is not very important. Having a format that's easily parse- and usable is much more important, being accessible to human inspection is a good advantage as well.
(if having a standard was important we'd all still be using XML)
I'd say that JSON proved that having a standard for data-interchange was important, especially because as a standard it was both straightforward to implement and limited a lot of the crazy feature creep that made/makes XML annoying. It not only was a standard, it was a strictly better standard.
At the end of the day, in a data format, you need the following things:
+ Some way of associating records with keys
+ Some way of storing records in a particular order
+ Some way of nesting records within records
+ Some simple way of pathing through a record
+ Some way of delimiting the start and end of a record
+ Some way of storing numbers and strings
Anything else is gravy, and JSON managed to nail that core supporting set of features right on the head, without adding other bogus stuff.
XML picked an overly verbose way of nesting and delimiting records, as well as presenting three options for storing data: attributes, body, and CDATA sections when one would have sufficed.
The comments thing in HJSON really annoys me, because comments are not needed in an interchange format--and if you need them because you're using it as a config file format, you can just add a comment key and name your variables better. Moreover, the temptation to use comments for, say, type annotations, is huge, and an antipattern (which is why Crockford left them out).
EDIT:
For everyone talking about a "simpler" JSON...the parser for this is about twice as long as the json2 parser from Crockford, with arguably worse comments.
One could argue "But it parses JSON anyways and lets me write sloppy^H^H^H^H^H^Hfriendly JSON", but that's not the point. People complaining about having to put their keys in quotes and leave off trailing commas are also missing the point.
I'd say that was exactly backwards. The reason JSON succeeded is precisely that it is a very, very specific standard. You may still be operating under the common misunderstanding that JSON is just "Javascript object notation", but in fact it is a carefully chosen and very rigid subset. It is actually illegal to not use double-quotes to identify a key, even though in Javascript itself often no quotes are required, and apostrophes are accepted as quotes. JSON's human-readability is a bit deceptive... it is also a very rigid format.
And it is precisely this rigidity that led to its success. There's a tons of other things that existed before JSON, and JSON won, and it's hard not to read the fact that the very looseness of its competition, which the designers of those formats perceived as strengths, were also weaknesses. Any slight little corner in a spec where there can be a divergence of opinion, there will be divergence of opinion when being implemented dozens of times. So having the spec be rigidly precise was a key advantage for JSON.
Which, incidentally, means HJSON's competition isn't JSON. This is why everyone here is talking about YAML, and the other such languages. And that's a rich, established field with probably no room for a new language to emerge.
HJSON's only hope would be to have a mode where it could be trivially compiled to real JSON, where it could work like a dialect. Unfortunately it's promise to round-trip comments wrecks that ability.
> The point of JSON is to have a lighter and simpler way of data interchange, where the data is arrays and maps of strings and numbers. It also works well with Javascript.
I contend there's no "point" to JSON at all. JSON is simply JavaScript (in fact, you can eval it) that was discovered could be used to send data from servers to browsers. It became a standard out of sheer convenience.
By breaking Javascript's ability to execute the code, you're eliminating an important feature of JSON. (I'll admit that nobody should be eval'ing JSON, but the fact that you can makes its existence better understood.)
No you can't, JSON is not JavaScript and eval is not capable of working on all correct JSON documents. JSON requires a JSON specific parser. So this breaks nothing, eval doesn't work now and JSON has never been JavaScript.
When JSON was first used, it was evaluated with eval. Certain issues might cause JSON not to evaluate properly (the only article I could find arguing that it's not Javascript involve Unicode issues that are rarely encountered), but it doesn't invalidate my point.
Look, you're just wrong. The only exception to "eval" is some obscure unicode bug others are mentioning. JSON is literally a JS object literal (with the additional requirement that any key be a string). There's also no reason to use eval when JSON.parse will do it safely.
No I'm not wrong and there mere fact that you mention the exception means you've conceded my point; and those aren't bugs, they're in the spec which means it was never a subset of JS.
> JSON is literally a JS object literal (with the additional requirement that any key be a string).
Which makes it not a JS object literal but a near subset with slightly different rules with unicode strings that make it not even a subset.
What spec are you looking at? http://www.ietf.org/rfc/rfc4627.txt specifically says in section 2.5 that all unicode characters are meant to be supported.
Seems like a construed attempt to make JSON easier for "humans" (whatever those are) but in general JSON isn't something my mom worries about. I'm not sure why something like a simplified YAML couldn't be used for the intended use cases.
I've been doing a lot of ansible work recently and YAML feels fairly reasonable and I can show it to my wife and she "gets" what is happening.
Comments are nice when your JSON is at-rest (e.g. in a file) but more often than not it's in-flight and my code is probably not adding comments when building a message.
Come to think of it, what would an API look like that writes HJSON? Not pretty, probably.
I like the unquoted strings and trailing comma, but the rest I don't think is unnecessary. I'd be happy with the standard ECMAScript map literal format.
Sure, but messages in-fligt not what this is for. This is only for writing "JSON" manually. I don't think there are many use cases where you have to generate comments programmatically.
I haven't looked at HJSON in detail, but JSON5 which is very similar has exactly the same writing API as JSON. Namely just `stringify`, which produces plain compliant JSON.
HJSON additionally seems round-trip safe, which means if you start with comments, they are preserved, which is nice for config files.
I don't think anybody sane considers sending out non-standard HJSON over the wire where plain JSON is expected.
Why do our machine data formats (JSON, XML, etc) need to be more human friendly? I don't build business processes that involve a human reading and manipulating JSON or XML.
I don't choose a data format for readability. In fact I think that's one of the least important factors. I choose JSON when the producer or consumer is JavaScript code. That's pretty much the only reason, I think, to pick JSON over XML or anything else.
Also, both JSON and XML are terrible for config file intended to be manually edited.
> Why do our machine data formats (JSON, XML, etc) need to be more human friendly? I don't build business processes that involve a human reading and manipulating JSON or XML.
I do.
JSON is a great format for config files. For example, Sublime Text's config files are in JSON. You configure the editor by editing those files. I think they additionally support comments, but if you edit something in the GUI you loose them. The round-trip-safeness of this seems like a really good addition for this use case (e.g. you change a value programmatically, and only that place in the file changes - all comments and whitespace stay where they are).
I currently use a lot of Python and some XML as "glue" code, and to configure other programs. Definitions for plots, settings for batch jobs, and so on. XML is overkill, and hard to parse. Python scripts are limited to python, and you have a full interpreter, so people can put logic in your config files, which is usually not what you want. YAML is too non-standard and has too many varieties and edge cases. JSON is really great for this case - even better, leninent JSON+comments. And accidentially, JSON is not only Javascript, but almost valid Python, and a couple of other languages. I think the sweet spot is hit by JSON5, which is JSON extended with some ECMAScript5 features, and additional comments.
I agree of course that plain JSON is the best when you write webservices, or consume or produce with JavaScript. I wouldn't use any non-standard thing on the wire. But if you just want to set a few variables from outside in your program, its great to be able to put a JavaScript/Python/PHP object literal in a file with a few comments, and it just works.
The 'next JSON' should be EDN. It addresses the shortcomings suggested by hjson and more. It has comments, all commas are gone, it has symbols (aka light-weight strings), it has built-in support for time and uuid objects.
Handling of strings is way too complex for very little gain, but the rest - trailing commas, comments and no quotes for alphanumeric keys - are actually quite useful. They complicate syntax only a little, yet make writing JSON by hand much nicer. These small additions merely take us back to Javascript object initialization syntax, straitjacketed version of which Crockford called JSON.
Having an ultra-relaxed JSON for config files is an excellent idea; ingy and I called our take on it JSONY - see http://p3rl.org/JSONY for some examples.
The more you try and push JSON along these lines the more it ends up looking like Rebol.
Here are the examples in Rebol:
[
; comments
;; specify rate in requests/second
rate: 1000
; quoting
key: 'value
text: "Look ma, some quotes!"
; path & HTML
path: %\c\windows
inject: <div class="important"></div>
; no need for escapes
space: %\c\windows\foo%20bar.txt
space2: %"\c\windows\foo bar.txt"
; no commas
one: 1
two: 2
; multiline string
haiku: {
Line 1
Line 2
Line 3
}
; array/list/block
fav-numbers: [1 2 3 6 42]
; and finally...
name: 'hjson
desc: "Rebol data format"
main: %./lib/rebol.reb
version: 1.0.1
tags: [
json
hjson
rebol
]
; and really finally...
light-switch: on
useful: yes
boolean: true
co-ords: 10x100
ip: 192.168.100.1
date: 1-Jan-1970
]
Of course this shouldn't be a surprise because Douglas Crockford cites Rebol as an influence on JSON.
For those interested have a look at REN (REadable Notation), a subset/dialect of Rebol, which is an attempt to produce a standardised format - https://github.com/humanistic/REN
I will never understand the tendency, especially in the web community, to consider complicated, ambiguous, inconsistent, and difficult to parse syntaxes as "friendly".
It might seem like a good idea at first, but over time it just results in a culture that publishes horrifically broken markup, and increasingly baroque implementations that try to work around every possible screw-up instead of simply flagging errors. This is, in my opinion, the biggest screw-up of the web: if early browsers simply flagged errors and pointed out where and what they are clearly, the web would be a much nicer place. Instead, actually writing a parser for web markup is a nearly impossible task, and not the weekend hack it should be.
One of the, if not the, great thing(s) about JSON is its beautifully simple syntax. 5 images on json.org precisely explain what it is, and that's that. You can write a parser for it in basically any language without any fancy libraries or frameworks in a few hours. That is a good thing, regardless of whether you personally would ever have to do such a thing, because it keeps the barrier of entry low and trickles through the ecosystem in positive ways.
Missing the point of JSON? Yes, but missing much more, including lessons that should have been learned.
That said, a few of the ideas are good ones (comments and such), but the bad ones (making crucial delimiters optional) are so bad they more than outweigh the benefits.
83 comments
[ 3.8 ms ] story [ 141 ms ] threadAnnoyingly, "x = some_json" is much less strict than "JSON.parse(some_json)".
Whilst YAML is less ambiguous, the significant whitespace and not being able to just cut and paste an argument into usefuleness (without wrapping it in a parser) means the effort is too high for what little pay-off it provides.
If you want something that's easier to edit for config files, you can use YAML.
And.. yeah, for human processing/reading/writing, ignoring trailing commas and having comment does improve JSON IMO.
I don't know why JSON is so strict in fact. Virtually everyone forgets to NOT have the last comma of a list all the time.
Now then again HJSON is +1 standard and incompatible with JSON so im not sure how that'll take off.
JSON:
YAML:Its for when you need a quick config file, and you think "Gosh, I wish I could just put some sloppy JSON/Python-like code in here, with a few comments.", and you don't want to use XML, INI, or YAML, or roll your own format.
I guess most people here are in the web development business. Which is fine, but I am not currently. I don't need JSON for what JSON is made for, but I often need something almost like JSON for my use cases.
You know what's annoying? Having to remember a dozen-odd JSON-but-not-really config file syntaxes instead of just JSON.
That's why YAML is out of the question for me. But JSON is too restricted to write it manually (no trailing commas, no comments, no round-trip-safety). What I'd ideally like is to take this Python/JS5 file (differ only in the comment indicator # vs //)
and to turn it into a safe It's not that I don't like JSON. Its just that I have a different requirements. When I need JSON, I use JSON. When I need easily editable config files that look like Python or JS (which I can easily remember), I use something like HJSON.It's a human friendly (to some. It's ambiguous, so I don't think it's very friendly) , but it's not JSON and doesn't offer any of the advantages of JSON.
I like the idea of a slightly simpler JSON but still keeping just the primitive data types yet not being able to say, include arbitrary files into the document.
They just want to be able to write sloppy JSON and have code handle all the strict requirements of real JSON.
Some people are pushing for TOML as an alternative but I'm not sure if that's going to happen.
Actually, all the JSON I produce at work is per hand. It would be a major productivity increase if I had a more lenient parser. You don't have to call it JSON if it offends you, call it "Object literals that look like JavaScript or Python" if you like.
JSON is great already. I don't really see this as necessary. I could understand the comments (but use // instead of # as per Javascript comments) to increase clarity to people who don't know the original design syntax. But the others do not seem really useful.
If there's one thing JSON has proven it's that having a standard way for data-interchange is not very important. Having a format that's easily parse- and usable is much more important, being accessible to human inspection is a good advantage as well.
(if having a standard was important we'd all still be using XML)
At the end of the day, in a data format, you need the following things:
Anything else is gravy, and JSON managed to nail that core supporting set of features right on the head, without adding other bogus stuff.XML picked an overly verbose way of nesting and delimiting records, as well as presenting three options for storing data: attributes, body, and CDATA sections when one would have sufficed.
The comments thing in HJSON really annoys me, because comments are not needed in an interchange format--and if you need them because you're using it as a config file format, you can just add a comment key and name your variables better. Moreover, the temptation to use comments for, say, type annotations, is huge, and an antipattern (which is why Crockford left them out).
EDIT:
For everyone talking about a "simpler" JSON...the parser for this is about twice as long as the json2 parser from Crockford, with arguably worse comments.
One could argue "But it parses JSON anyways and lets me write sloppy^H^H^H^H^H^Hfriendly JSON", but that's not the point. People complaining about having to put their keys in quotes and leave off trailing commas are also missing the point.
And it is precisely this rigidity that led to its success. There's a tons of other things that existed before JSON, and JSON won, and it's hard not to read the fact that the very looseness of its competition, which the designers of those formats perceived as strengths, were also weaknesses. Any slight little corner in a spec where there can be a divergence of opinion, there will be divergence of opinion when being implemented dozens of times. So having the spec be rigidly precise was a key advantage for JSON.
Which, incidentally, means HJSON's competition isn't JSON. This is why everyone here is talking about YAML, and the other such languages. And that's a rich, established field with probably no room for a new language to emerge.
HJSON's only hope would be to have a mode where it could be trivially compiled to real JSON, where it could work like a dialect. Unfortunately it's promise to round-trip comments wrecks that ability.
I contend there's no "point" to JSON at all. JSON is simply JavaScript (in fact, you can eval it) that was discovered could be used to send data from servers to browsers. It became a standard out of sheer convenience.
By breaking Javascript's ability to execute the code, you're eliminating an important feature of JSON. (I'll admit that nobody should be eval'ing JSON, but the fact that you can makes its existence better understood.)
> JSON is literally a JS object literal (with the additional requirement that any key be a string).
Which makes it not a JS object literal but a near subset with slightly different rules with unicode strings that make it not even a subset.
So now you're wrong twice.
I've been doing a lot of ansible work recently and YAML feels fairly reasonable and I can show it to my wife and she "gets" what is happening.
Sorry just reminds me of: http://xkcd.com/927/
Maybe we need a HN bot that links that automatically?
Come to think of it, what would an API look like that writes HJSON? Not pretty, probably.
I like the unquoted strings and trailing comma, but the rest I don't think is unnecessary. I'd be happy with the standard ECMAScript map literal format.
I haven't looked at HJSON in detail, but JSON5 which is very similar has exactly the same writing API as JSON. Namely just `stringify`, which produces plain compliant JSON.
HJSON additionally seems round-trip safe, which means if you start with comments, they are preserved, which is nice for config files.
I don't think anybody sane considers sending out non-standard HJSON over the wire where plain JSON is expected.
Cut it out. JSON is fine.
I don't choose a data format for readability. In fact I think that's one of the least important factors. I choose JSON when the producer or consumer is JavaScript code. That's pretty much the only reason, I think, to pick JSON over XML or anything else.
Also, both JSON and XML are terrible for config file intended to be manually edited.
I do.
JSON is a great format for config files. For example, Sublime Text's config files are in JSON. You configure the editor by editing those files. I think they additionally support comments, but if you edit something in the GUI you loose them. The round-trip-safeness of this seems like a really good addition for this use case (e.g. you change a value programmatically, and only that place in the file changes - all comments and whitespace stay where they are).
I currently use a lot of Python and some XML as "glue" code, and to configure other programs. Definitions for plots, settings for batch jobs, and so on. XML is overkill, and hard to parse. Python scripts are limited to python, and you have a full interpreter, so people can put logic in your config files, which is usually not what you want. YAML is too non-standard and has too many varieties and edge cases. JSON is really great for this case - even better, leninent JSON+comments. And accidentially, JSON is not only Javascript, but almost valid Python, and a couple of other languages. I think the sweet spot is hit by JSON5, which is JSON extended with some ECMAScript5 features, and additional comments.
I agree of course that plain JSON is the best when you write webservices, or consume or produce with JavaScript. I wouldn't use any non-standard thing on the wire. But if you just want to set a few variables from outside in your program, its great to be able to put a JavaScript/Python/PHP object literal in a file with a few comments, and it just works.
[1]: https://github.com/bevry/cson
https://github.com/edn-format/edn
Ping me if you want to help out.
a) has a lot more clear use case, and
b) doesn't really compete with JSON for data transfer (its primary use case)
Here are the examples in Rebol:
Of course this shouldn't be a surprise because Douglas Crockford cites Rebol as an influence on JSON.For those interested have a look at REN (REadable Notation), a subset/dialect of Rebol, which is an attempt to produce a standardised format - https://github.com/humanistic/REN
It might seem like a good idea at first, but over time it just results in a culture that publishes horrifically broken markup, and increasingly baroque implementations that try to work around every possible screw-up instead of simply flagging errors. This is, in my opinion, the biggest screw-up of the web: if early browsers simply flagged errors and pointed out where and what they are clearly, the web would be a much nicer place. Instead, actually writing a parser for web markup is a nearly impossible task, and not the weekend hack it should be.
One of the, if not the, great thing(s) about JSON is its beautifully simple syntax. 5 images on json.org precisely explain what it is, and that's that. You can write a parser for it in basically any language without any fancy libraries or frameworks in a few hours. That is a good thing, regardless of whether you personally would ever have to do such a thing, because it keeps the barrier of entry low and trickles through the ecosystem in positive ways.
Missing the point of JSON? Yes, but missing much more, including lessons that should have been learned.
That said, a few of the ideas are good ones (comments and such), but the bad ones (making crucial delimiters optional) are so bad they more than outweigh the benefits.