S-expressions are the most logical representation of data structures. “CAR”/“CDR” are not a part of the expression syntax, so this argument against them won’t matter.
S-expressions are great once you've figured out functions, atoms, complex data types, etc. That's how Amazon's Ion[1] uses them: as a nice supplement to a serialization standard.
> “CAR”/“CDR” are not a part of the expression syntax
If you were implementing them for a serialization format, you'd leave out cons cells and car / cdr.
But in late-binding LISPs, which is where they're typically used, (a . b) is part of the syntax. It's worth calling out for why you don't want it: it's badly typed since it could be a 2-tuple, or it could be a list depending on what 'b' is.
S-expressions lack a standard representation of maps/dictionaries. Even LISP is inconsistent: it has both alists (list of pairs) and plists (list of alternating keys and values).
So S-expressions suffer a similar problem to XML: there is not a clear way to represent many data structures.
As for comments, I think JSON is only "human readable" by accident. It was never really meant to be read by humans, and was created in a way that implicitly discouraged people from trying to write or edit it by hand. Many parsers are picky about whitespace to discourage people from trying to create config files in JSON.
Ah, so now you're encoding out-of-band communication into your data? How is that not worse? This also completely breaks if your data is validated against a schema, or if your comment gets parsed as a data entry. For example, in Javascript projects you define your dependencies in package.json. I can't add a comment to explain why I had to add a strange dependency, because npm will interpret it as a dependency itself and then break.
Very interesting article and I like the style in which it was written.
Another factor to consider is whether the serialisation code is intrusive or not. Being intrusive can be an issue for legacy codebases, and IMO is not paricularly welcome regardless.
Everyone loves to hate on XML just because it can be verbose and some of the WS* protocols are complicated. But as a human-readable format it’s no harder to read or parse than JSON and legacy application support is orders of magnitude better than JSON. I’m not saying we should all use it but we shouldn’t be hating it either.
The issue with XML is that it was mostly pushed upon developers without any deeper understanding.
So there's a ton of utterly crap XML applications out there with no rhyme or reason, and these crap XML enterprise apps are most developer's understanding of "real world XML". Similar to how terrible C++ code being all over the place is making people hate C++ the language.
-------
XML can be used very well however: its use in Jabber / XMPP is clear and concise. XML Namespaces are needed due to the extensions of XMPP, and various programs can interact with the protocol. Specifications are clear, the ability to "stream" data is proven, different parser strategies have been implemented. Its clear that XML in this instance works, and works pretty well.
Around 2009, I worked on an application whose web interface and API were the same url -- it was served as an XML data file, with an XSLT transform to make it pretty for the browser. It was so comfortable.
I really like the separation between the semantic content and the visualization there. HTML/JS/CSS throw everything together instead. But XSLT is nothing I want to have to work with..
>crap XML enterprise apps are most developer's understanding of "real world XML"
I mean if there's enough of it, it kinda becomes "real world XML". Right or wrong human's ability to actually use the tool or tech well is often where the bar is set. Could be a great idea but if folks can't do it... won't fly.
Markup character and namespaces aside, I think what makes JSON easier to read is that it is more explicit about whether a child object is a singleton or a list, where XML requires you to consult the schema for minOccurs and maxOccurs.
The primary problem with XML is that it's a document markup language being used to do data serialization. Schemas were bolted on after the fact to add enough metadata for data serialization, but that's a whole lot of added complexity on top of the already complex process of parsing a lightly specced document.
So you have an already complex document parser (which you are using only a fraction of the full functionality!) with the added complexity of a schema system layered on top. This is why JSON took off so fast, it ditched the whole document part and just gave you the data.
It's not something that an IBM committee would have come up with but it fits the needs of an application. If XML has an "intrinsic fault" it's related to DTDs and namespaces which allow for the creation of horrible, complex monstrosities. XML can be every bit as stripped down and to-the-point as JSON, trading the curley braces for angle brackets. JSON's best point is that it doesn't support any of this needless complexity but it's still easy enough, with enough nested objects and arrays, to make something that's effectively not readable by a human.
> It's not something that an IBM committee would have come up with but it fits the needs of an application.
Items can be order-dependent in XML. So the "id=1" thing is still superfluous. See xs:sequence. (In contrast: xs:all declares something to be order independent).
If you were doing XML for your own apps, I think xs:sequence items simplify parsing grossly.
Ex: if you have Address, maybe you always want "Street", then "Zip Code", then "State".
<Address>
<Street> 1234 Blah Street </Street>
<City> Springfield </City>
<State> Whatever </State>
<Zip> 55555 </Zip>
</Address>
By declaring that Street -> City -> State -> Zip must happen in that particular order, you can grossly simplify the parser's job when reading such data in. (Or writing it out).
--------
If there are multiple People at this address, you could (and probably should) just list them out in order.
<Address>
<Street> 1234 Blah Street </Street>
<City> Springfield </City>
<State> Whatever </State>
<Zip> 55555 </Zip>
<People>
<p> Marge </p>
<p> Homer </p>
<p> Lisa </p>
<p> Bart </p>
<p> Maggie </p>
</People>
</Address>
It’s made for documents. It just doesn’t fit to general data structures. For instance when do you put class members in elements and when in attributes?
It's also highly redundant (so it basically can't be hand written), undesrpecified (do I put this value as an atribute, value, or sub-element?), underpowered (how does it represent numbers? Byte streams?), complicated (have you ever created a DTD by hand?), and just out badly designed.
Of course, JSON and YAML share some of those same problems, and add some different ones. But the complaints about XML aren't some superfluous opinions about how it looks.
The under-specified thing I think is funny because it’s the source of pretty much all the bad XML you see in the wild. For being an entirely tree oriented format it’s tragic flaw is that XML can’t actually represent nested dicts (or arrays for that matter) which turned out to be the data structures people actually wanted to work with.
Oh there are lots of ways to beat XML into looking like it can do it but they’re all bad, break all the actual semantics of XML and turn a nice deserialization story of a tree where each node is a complete thing into mess.
Turns out that very few people actually needed a tree of heterogeneous string k-v pairs.
The attribute versus child conundrum in XML specification is largely absent in JSON. Ordered versus unordered children is still a puzzle to sort out but more of a special case.
In XML, there was always some motherfucker trying to stuff CSV into an attribute.
And the difference between ID as specified and ID as implemented lead to a lot of problems, reaching its zenith (or maybe nadir?) in the XML Signature spec.
Or an array of arrays with a TOC containing the keys to index mapping. Keeps the column order and it's more compact because it omits the keys in the data.
JSONB is a private internal data representation in PostgreSQL, designed to be faster to index than textual JSON. As I understand it, it isn’t guaranteed to be stable, it isn’t exposed outside PostgreSQL, nor designed for data interchange.
I've been using it lately but i'm i'll qualified to review it to any meaningful degree. It's neat, (reportedly) fast, but has seeming zero traction and thus makes me uneasy.
(I'm currently (configurably) using it in place of bincode, for a content addressed store. for some context)
All of them use schemas, the only difference is if the syntax is specified in the language and can be machine verified or if you're left to figure it out on your own. Actually the latter will be machine verified too, but the results of a mismatch is a core dump or an error in your own ad-hoc schema verification system.
> Not sure anyone really knows how XML happened. It’s basically the W3C’s fault, I think?
Doesn't instill confidence in that content.
fyi: XML was subset by W3C (the SGML "extended review board" specifically) from SGML to become the generic syntax for web vocabularies (replace HTML syntax by XTHML, and define new SVG, MathML vocabularies). Didn't work out on the web, though.
SVG did. XHTML is incompatible with traditionally lazy nature of webdev: broken markup happens regularly, XML would make it non-displayable, so HTML keeps accessibility of broken pages.
SBE (Simple Binary Encoding) also makes some interesting performance claims.
My pet peeve with both protobuf and capnp is the lack of an option type; they seem to be designed for languages with type systems that include null, rather than more modern languages like Rust or Kotlin.
Another vote for CBOR. I had actually designed my own binary JSON encoding out of frustration with BSON (which nobody should use IMO). My format was focused on compactness, but I dropped it when I found CBOR which was very similar and even slightly better in places. One notably thing about CBOR is the support for both Indefinite and Definite-Length Arrays and Maps (typically streaming generators need the former whereas parsing the latter can be more efficient).
Interestng read, didn't know most of this. Not that it matters in the end: at one point we just needed something simple which would just work and MessagePack does just that :)
Btw JSON is often a popular choice, but although this is not explicitly mentioned in the main article as a con which is an oversight imo, the actual JSON standard does not support NaN/+-Inf for floating point. That's insane, which is why most implementations have an option to allow that anyway, but not all, which can be quite the showstopper.
If you follow the link, you'll see that there are no examples of data in the format and the README declares things like that it will provide a BNF when the specification ready. The context seems to imply that it is in an early implementation stage and not ready for non-enthusiast consumption. Also, last updated in 2014.
From recollection it’s the native data transmission format for Datomic (which is driven by Cognitect/Nubank). I think there is a little bit more to the bias’ of EDN (mostly driven by language preferences in this case).
EDN has a few limitations regarding encoding custom types (tagged literals), and also performance in general. I understand both Transit and Fressian are possible evolutions addressing these points.
I would love edn if it was viable as a standalone format. In practice, programming in clojure it's plain JSON vs transit vs fressian. I tend to go with just JSON.
The kiss of death. It's a graveyard of things from a previous era, pure maintenance mode. I think library authors get tired of maintaining for free then hand it off to these life support organizations. The new way for open source software is a decentralized ecosystem where people fix software they actually use, so the cream rises to the top on its own and you don't need the sponsorship of these digital museums.
After half a decade of "better than JSON" mentality, fiddling with thrift, protobufs, custom serialization methods, I came to realize JSON/HTTP is usually the right tool for 90% of jobs.
Maybe, if you're not paying for data transmission costs.
Binary formats win for the following cases:
- payload size
- serialisation/deserialisation speed
- simplicity of client / server code (e.g. replacing a full blown HTTP server with a simple ZeroMQ one)
Sure, having payloads that are human readable is great for initial debugging/verification, but once things are put live it's just an unnecessary expense.
This is true. There usually are a few cases an org would have where latency costs, de-serialization and deserialization costs outweigh the simplicity of http/json, and it usually happens in a business critical part of the company. This leads to a standardization policy where ALL projects now use grpc/thrift/etc... as an RPC standard, even if when it doesn't make sense.
"simplicity of http/json" always elicits snickering from me.
HTTP, by itself, is one of the more complex layer-7 protocols known to man. UTF-8 is not simple. Converting a decimal string representation of a floating point value into IEEE 754 is not simple (particularly compared to... memcpy). In a bizarre example of the tail wagging the dog, the ARM ISA has added an instruction specifically for parsing JSON...
There's really not a lot of value in having different serialization formats, so the "even if it doesn't make sense" requires a specific context for determining what makes sense.
You think an instruction with Javascript in the name, that converts JavaScript floating-point to signed fixed-point, rounding towards zero, isn't used when parsing JSON?
It is, but oddly enough Javascript numbers and JSON numbers are different and mutually incompatible. JS allows ±INF and the NANs, since it uses IEEE754 double-precision floating point. JSON doesn't, and technically allows arbitrary precision floating point (though implementations are allowed to define truncation behavior to a format at least as large as an IEEE754 double).
> and technically allows arbitrary precision floating point
technically, "arbitrary precision decimal"; floating point is a storage representation, to which JSON is formally agnostic, though RFC 8259 recommends that limiting to the range and precision representable with IEEE 754 doubles is a good idea for interoperability.
Yup. One of many ways that parsing JSON is anything but a simple thing is the ambiguity around how numbers, which really isn't a hard thing to get right.
With compression, payload size differences aren't that significant, and don't necessarily favour other formats (e.g. almost none of the formats have particularly compact representations of strings, and single decimal digit integer values are common enough that integer fields are not that wasteful in JSON).
I totally hear you on serialization/deserialization speed, though it is amazing how inefficient some of the "machine readible" implementations can be, and how efficient people have been able to get JSON parsing, its horrible design disadvantage in this area is lethal.
The simplicity argument is an amusing one to me. I think a lot of people build simple client/server code for JSON, but these "simple" implementations have all kinds of rough edges that prove problematic long term. Once you start doing things right, JSON proves to be far more complex.
Are you including the size of the typically useless HTTP headers when you talk about 'payload size', or just the actual payload in the HTTP request itself?
I really mean 'typically useless' in the scenario where the communication is being done internally, not when you're actually communicating over the public internet. There can be quite a bit of wastage in this scenario. I really don't care about the 'Date' header or the 'Server' header, but the HTTP spec does, so they're there.
Thanks to all that complexity in HTTP (particularly HTTP2), there is no need to pay that headers size cost in most cases, particularly ones where bandwidth tends to be a concern. Yes, naive and "simple" implementations totally waste a TON of space in headers, and even if you exploit all that complexity HTTP is still inefficient in terms of network bandwidth consumption, much as even compressed JSON is still inefficient with its use of bytes.
All I'm saying is that while JSON is horribly inefficient for some payloads, the reality is that most serialization formats tend to be efficient for specific kinds of payloads, but still have plenty of common cases they don't attempt to be efficient with. For example, machine readable formats like protobuf, thrift, avro, etc. employ some variation on varint encoding to keep small integers compactly represented, they encode strings relatively inefficiently; They encode field type & number, followed by a varint for length, followed by uncompressed UTF-8 encoded payload... and I've found that if anything it seems once you go through the compression ringer it often isn't much different in terms of space consumption vs. JSON... and sometimes it can be less efficient. Given how often fields end up being encoded as strings (even though they should be enums, ints, etc.), this has resulted in a lot of surprising cases for teams who have tried switching away from JSON to reduce payload sizes. If you cared about efficiency, you'd probably have a special encoding mechanism for short strings, and then a reasonably compact (but fast to parse) representation for longer strings, probably using something like SCSU encoding instead of UTF-8.
...and I've reached the exact opposite conclusion. You aren't unusual, but I've never been able to understand how people get there any more than they understand how I have come to my conclusion.
> Not sure anyone really knows how XML happened. It’s
> basically the W3C’s fault, I think? It’s okay for some
> things but in the end I’m not sure it’s something anyone
> actually wants to use, it’s just going to be one more of
> those mistakes of the past.
Look, I was doing web dev when XMLRPC was in. For simple API stuff, JSON ended up being worlds better. No fiddly XML preamble, no schemas, no envelopes, just enough structure to be able to quickly pull out a couple of fields and we're set. Beautiful.
But let me tell you how much I appreciated XML and schema validation when we were working with fifteen different partners all using a standardized XML format with a schema. These are cats who were putting XSDs into their IDEs, clicking generate to get a class, and then just not caring.
So when our integrations broke because they were sending XML payloads that failed schema validation, our conversations were so easy.
Before you say "yes but technically you could have done the same with $OTHER_THING" -- sure it's possible, but XML is ubiquitous. Shit, our tools were in Ruby.
Much agreement here, and from the exact same usecase. Probably the easiest B2B API integration scenario I worked with in the past was XML + XML Schema validation. And I still think XML is more readable in many cases than JSON, because while verbose the hierarchical structure is very easy to grasp, sometimes I misread an entire JSON document at first because I missed a quick "[" or "[{" (yes I have also misread XML documents because of the sheer weight of text you are presented with, so I'll call that one a wash).
I do agree with the XML criticisms around writability and complexity, though I think if you made a Venn diagram of the actual complexities inherent in managing a multi-client document-exchange system and the complexities of XML+Schema, you'd find that they're similar--and that many people who start building such an exchange with JSON documents end up stumbling into the same amount of complexity except in an ad hoc fashion.
It's funny, I wonder if other people have the same surreal experience: as the mass zeitgeist moved away from XML I was more or less like, okay, gotta go with it, as the driftwood goes with the currents. It's interesting how the zeitgeist functions. You can tell it's happening when people look at your functional system and are like, "Uhh, why aren't you using X...?" as though it's self-evident that you should drop everything and rewrite. Where X was, over the years (dating myself) {"Java", "PHP", "Node", "Rails", "Jquery", "REST", "Angular", "React", "Thrift", "NoSQL", "Protobuf", "Hadoop"}. Some of which I quite happily used, BTW, so the interesting part is the penetrative nature of the conversation, e.g. how the quite-functional-but-not-hot technology becomes the sick gazelle falling behind the herd, even as it is not empirically sick.
The problem I think is the extreme cargo-culting that exists with any popular technology -- half the arguments against the current flavor of the month really just begin with "no, this is not sent from god, and there do exist tradeoffs". Which is a fairly unreasonable starting point for a conversation between educated parties.
And then the ratchet turns, and the pendelum swings in reverse, and suddenly that thing sent from god was instead discovered to be sent from satan... and you're still trying to explain that tradeoffs exist
This coming from someone who hates XML, avoids it at nearly all costs - there are times it's the right tool for the job. And you list that time. If you have a 1-1 API contract, don't use XML. If you have 1-N, don't use XML. But if you have N-N API contracts? Where you're talking to half a dozen other services, and they're all talking to you? XML is an excellent choice.
One of my very, very, very first jobs (1999) was at a startup that was leaning very heavily into XML for dealing with structured data. A massive amount of time was spent dealing with issues related to hand-editing XML, so I went deep into building schemas, utilities to validate XML according to the schema, and finally building an editor that was capable of providing some XSD-enabled automagic.
No, with Protobuf, I send my counterparts a .proto file, and they 1) Don't know what the hell to do with it, not just because they don't know Protobuf but because 2) it wasn't included in the docs/dictionaries from the standards organization that set this all up in the first place.
I hate xml so much. But I had a exam at my university a year ago where they would let me type a validation schema for a xml file in notepad++ and a checker on a computer with no internet. The fail rate for that exam was high and the course was outdated, api am glad they updated the curriculum now. I just hope to never work with xml data again.
It's a shame the original article didn't mention XPath which is a great feature when you're in the XML ecosystem. (Obviously JSON has jq, but that's not a standard - so there's no library version - and has some weirdness compared to XPath).
Basically, use the best tool for the job. If you're working with complex data structures from many sources, having data, validation and transformation languages (XML, XSD and XSLT) which have been co-developed and refined to basically be able to validate any third-party tool is invaluable. Sometimes extra safety is worth both the extra space and time.
I was asked by a company to sort out complaints which were coming from recipients of their XML. The big win was doing validation against the schema on the outgoing XML!
No trailing commas and multi-line strings are my other two big complaints. The ability to leave keys unquoted if they don't have any spaces would also be nice.
The best part is that these are a superset of JSON and could easily be added if there would just be more buy-in (lots of unofficial support already exists).
I'm a huge fan of jsonnet, but this article is specifically about data serialization formats, not configuration languages. It could've been a worthwhile mention though.
Good point, though I think Arrow is optimized for larger amounts of data. Not sure how it would hold up for small messages. Still, lots of people use JSON, etc. to store large amounts of data, so Arrow (or maybe more accurately, Parquet) is a fair comparison here.
There are a bunch more possible entries in the human-readable category, one of the more popular contenders is TOML. https://github.com/toml-lang/toml .
There are more. I don't know if java dot-properties file is worth including.
Hm. Would you call YAML a "serialization format" but not TOML? I'm not sure where to draw the lines of what is a "serialization format" (you can serialize to both yaml and toml, no?) but YAML and TOML feel to me like they are in the same space, human-readable representations of hash/array json-like data, ie serializations; not to you?
One key element that I think is missing in this analysis is whether the format supports cyclic data out of the box. This is an important feature for many languages since it most languages besides Rust make it very difficult to determine if the data may contain cycles or not and handling cycles can be very expensive in terms of both time and memory for serialization.
In some sense JSON is unimprovable, because some properties of JSON are reached their limits.
The idea of JSON is to offer two important ways to aggregate, which are different enough that they deserve separate treatment - ordered array and associative array. They are economically represented, with [] and {}, and internal structure is also made with small costs - commas (,) between elements and colons (:) for keys in associative array. We can argue that both of those features are required.
Chosen primitives - numbers, strings, booleans and null - are also selected from what's regularly and consistently used, and the choice is supported by two decades of JSON application. Roots of this selection are in design of JavaScript, which in turn relied on common practice for basic primitives.
If JSON is considered as what I think it is, structure out of basic primitives, it's at the optimal point. Crockford's decision to avoid JSON versioning makes a good sense.
Applications of JSON - human-readable texts, performance-optimized communications - can suffer from e.g. lack of built-in comments, non-extensible "type system", non-optimal bandwidth usage (if direct ASCII or even UTF-8 is used for JSON encoding, not some other compressed approach). They however can be fixed by building on top of JSON. For example, parties can agree upon reserved keys and compression schemas, constraints in form of grammars (JSON-Schema). This flexibility stems from the fact that JSON is at the very foundation of data structures, so "better than JSON" is to an extent like "better than two's complement number representation".
I think https://github.com/rentzsch/lich gets the basic primitives right as well, and in a binary-friendly format, but it has never been pushed by any organization, so it’s essentially unknown.
Again, I think this is based on some starting assumptions about what is an ideal state, and a lot of people would argue they are flawed assumptions. It's easy to get trapped into a context when considering what the ideal solution would look like.
People forget this, but ASCII (and therefore UTF-8) already has characters in it for field/record/etc. separation. One increases complexity significantly (and introduces inefficiency) by overloading other characters with their semantics.
There are much more simple & efficient ways to demarcate ordered arrays in particular (e.g. simple length prefixed structures).
Nulls are highly problematic for any number of reasons that are widely documented, and JSON unfortunately compounds this by having two ways of representing them (explicitly and by omission).
One might quibble with the notion that a format that didn't even exist in 2000 doesn't really have two decades worth of applications working on it, but setting that aside, the design constraints of JavaScript and the challenges that have emerged from it are far from ideal. The work arounds that have emerged (the whole Number fiasco, the "standardizing" on UTF-8 while using UTF-16 when escaping, line terminators, the backslash escapign of control characters, the inconsistencies between JSON & JSONP, the binary64 "good interoperability" rule, and comments being disallowed, and of course the various JSON Schema efforts).
While I agree that an ideal serialization standard would not have versioning, and Crockford's decision to avoid JSON versioning might consequently seem to make good sense... in practice this has lead to significant inconsistencies between implementations, many of which lead to interoperability problems that could at least be somewhat mitigated by versioning. (http://seriot.ch/parsing_json.php)
"Fixing" a bad abstraction by layering on a bad abstraction is a common house of cards meme. It is highly problematic to say the least. It's how "simple" JSON rapidly evolves to be at least as complex (and at least as error prone) as ASN.1.
152 comments
[ 2.9 ms ] story [ 184 ms ] thread> “CAR”/“CDR” are not a part of the expression syntax
If you were implementing them for a serialization format, you'd leave out cons cells and car / cdr.
But in late-binding LISPs, which is where they're typically used, (a . b) is part of the syntax. It's worth calling out for why you don't want it: it's badly typed since it could be a 2-tuple, or it could be a list depending on what 'b' is.
[1]: https://amzn.github.io/ion-docs/docs/spec.html#sexp
So S-expressions suffer a similar problem to XML: there is not a clear way to represent many data structures.
(jk)
Very interesting article and I like the style in which it was written.
Another factor to consider is whether the serialisation code is intrusive or not. Being intrusive can be an issue for legacy codebases, and IMO is not paricularly welcome regardless.
So there's a ton of utterly crap XML applications out there with no rhyme or reason, and these crap XML enterprise apps are most developer's understanding of "real world XML". Similar to how terrible C++ code being all over the place is making people hate C++ the language.
-------
XML can be used very well however: its use in Jabber / XMPP is clear and concise. XML Namespaces are needed due to the extensions of XMPP, and various programs can interact with the protocol. Specifications are clear, the ability to "stream" data is proven, different parser strategies have been implemented. Its clear that XML in this instance works, and works pretty well.
I mean if there's enough of it, it kinda becomes "real world XML". Right or wrong human's ability to actually use the tool or tech well is often where the bar is set. Could be a great idea but if folks can't do it... won't fly.
I can't count how often I've seen XML that looked like this:
Which was, technically, XML (it even conformed to a schema!)... almost like "malicious compliance".I think that's the main problem of XML it's not meant to be read by humans but people thought it was.
So you have an already complex document parser (which you are using only a fraction of the full functionality!) with the added complexity of a schema system layered on top. This is why JSON took off so fast, it ditched the whole document part and just gave you the data.
Items can be order-dependent in XML. So the "id=1" thing is still superfluous. See xs:sequence. (In contrast: xs:all declares something to be order independent).
If you were doing XML for your own apps, I think xs:sequence items simplify parsing grossly.
Ex: if you have Address, maybe you always want "Street", then "Zip Code", then "State".
By declaring that Street -> City -> State -> Zip must happen in that particular order, you can grossly simplify the parser's job when reading such data in. (Or writing it out).--------
If there are multiple People at this address, you could (and probably should) just list them out in order.
It's also highly redundant (so it basically can't be hand written), undesrpecified (do I put this value as an atribute, value, or sub-element?), underpowered (how does it represent numbers? Byte streams?), complicated (have you ever created a DTD by hand?), and just out badly designed.
Of course, JSON and YAML share some of those same problems, and add some different ones. But the complaints about XML aren't some superfluous opinions about how it looks.
Oh there are lots of ways to beat XML into looking like it can do it but they’re all bad, break all the actual semantics of XML and turn a nice deserialization story of a tree where each node is a complete thing into mess.
Turns out that very few people actually needed a tree of heterogeneous string k-v pairs.
For comparison, time formats in JSON are all over the place.
In XML, there was always some motherfucker trying to stuff CSV into an attribute.
And the difference between ID as specified and ID as implemented lead to a lot of problems, reaching its zenith (or maybe nadir?) in the XML Signature spec.
If you need three values use an array (or in xml, child element)
https://news.ycombinator.com/item?id=7457645
I've been using it lately but i'm i'll qualified to review it to any meaningful degree. It's neat, (reportedly) fast, but has seeming zero traction and thus makes me uneasy.
(I'm currently (configurably) using it in place of bincode, for a content addressed store. for some context)
We'd also have a use case where we think about using bincode, this seems worth to check out/compare as a possible alternative.
Anything using a schema that is agreed upon between participants isn't like JSON at all.
Doesn't instill confidence in that content.
fyi: XML was subset by W3C (the SGML "extended review board" specifically) from SGML to become the generic syntax for web vocabularies (replace HTML syntax by XTHML, and define new SVG, MathML vocabularies). Didn't work out on the web, though.
It is pretty clear by the language used that it's all written tongue in cheek.
https://diziet.dreamwidth.org/6568.html
SBE (Simple Binary Encoding) also makes some interesting performance claims.
My pet peeve with both protobuf and capnp is the lack of an option type; they seem to be designed for languages with type systems that include null, rather than more modern languages like Rust or Kotlin.
Btw JSON is often a popular choice, but although this is not explicitly mentioned in the main article as a con which is an oversight imo, the actual JSON standard does not support NaN/+-Inf for floating point. That's insane, which is why most implementations have an option to allow that anyway, but not all, which can be quite the showstopper.
[0] https://github.com/edn-format/edn
[1] https://github.com/cognitect/transit-format
[2] https://github.com/Datomic/fressian/wiki
[3] https://www.youtube.com/watch?v=JArZqMqsaB0&ab_channel=Cloju...
This made me laugh more than it probably should have.
The kiss of death. It's a graveyard of things from a previous era, pure maintenance mode. I think library authors get tired of maintaining for free then hand it off to these life support organizations. The new way for open source software is a decentralized ecosystem where people fix software they actually use, so the cream rises to the top on its own and you don't need the sponsorship of these digital museums.
Binary formats win for the following cases:
- payload size
- serialisation/deserialisation speed
- simplicity of client / server code (e.g. replacing a full blown HTTP server with a simple ZeroMQ one)
Sure, having payloads that are human readable is great for initial debugging/verification, but once things are put live it's just an unnecessary expense.
HTTP, by itself, is one of the more complex layer-7 protocols known to man. UTF-8 is not simple. Converting a decimal string representation of a floating point value into IEEE 754 is not simple (particularly compared to... memcpy). In a bizarre example of the tail wagging the dog, the ARM ISA has added an instruction specifically for parsing JSON...
There's really not a lot of value in having different serialization formats, so the "even if it doesn't make sense" requires a specific context for determining what makes sense.
It is nothing to do with JSON.
technically, "arbitrary precision decimal"; floating point is a storage representation, to which JSON is formally agnostic, though RFC 8259 recommends that limiting to the range and precision representable with IEEE 754 doubles is a good idea for interoperability.
I totally hear you on serialization/deserialization speed, though it is amazing how inefficient some of the "machine readible" implementations can be, and how efficient people have been able to get JSON parsing, its horrible design disadvantage in this area is lethal.
The simplicity argument is an amusing one to me. I think a lot of people build simple client/server code for JSON, but these "simple" implementations have all kinds of rough edges that prove problematic long term. Once you start doing things right, JSON proves to be far more complex.
I really mean 'typically useless' in the scenario where the communication is being done internally, not when you're actually communicating over the public internet. There can be quite a bit of wastage in this scenario. I really don't care about the 'Date' header or the 'Server' header, but the HTTP spec does, so they're there.
All I'm saying is that while JSON is horribly inefficient for some payloads, the reality is that most serialization formats tend to be efficient for specific kinds of payloads, but still have plenty of common cases they don't attempt to be efficient with. For example, machine readable formats like protobuf, thrift, avro, etc. employ some variation on varint encoding to keep small integers compactly represented, they encode strings relatively inefficiently; They encode field type & number, followed by a varint for length, followed by uncompressed UTF-8 encoded payload... and I've found that if anything it seems once you go through the compression ringer it often isn't much different in terms of space consumption vs. JSON... and sometimes it can be less efficient. Given how often fields end up being encoded as strings (even though they should be enums, ints, etc.), this has resulted in a lot of surprising cases for teams who have tried switching away from JSON to reduce payload sizes. If you cared about efficiency, you'd probably have a special encoding mechanism for short strings, and then a reasonably compact (but fast to parse) representation for longer strings, probably using something like SCSU encoding instead of UTF-8.
JSON Schema http://json-schema.org/ is widely used for things like Swagger / OpenAPI.
But let me tell you how much I appreciated XML and schema validation when we were working with fifteen different partners all using a standardized XML format with a schema. These are cats who were putting XSDs into their IDEs, clicking generate to get a class, and then just not caring.
So when our integrations broke because they were sending XML payloads that failed schema validation, our conversations were so easy.
Before you say "yes but technically you could have done the same with $OTHER_THING" -- sure it's possible, but XML is ubiquitous. Shit, our tools were in Ruby.
XML can be a bear, but sometimes you need a bear.
I do agree with the XML criticisms around writability and complexity, though I think if you made a Venn diagram of the actual complexities inherent in managing a multi-client document-exchange system and the complexities of XML+Schema, you'd find that they're similar--and that many people who start building such an exchange with JSON documents end up stumbling into the same amount of complexity except in an ad hoc fashion.
It's funny, I wonder if other people have the same surreal experience: as the mass zeitgeist moved away from XML I was more or less like, okay, gotta go with it, as the driftwood goes with the currents. It's interesting how the zeitgeist functions. You can tell it's happening when people look at your functional system and are like, "Uhh, why aren't you using X...?" as though it's self-evident that you should drop everything and rewrite. Where X was, over the years (dating myself) {"Java", "PHP", "Node", "Rails", "Jquery", "REST", "Angular", "React", "Thrift", "NoSQL", "Protobuf", "Hadoop"}. Some of which I quite happily used, BTW, so the interesting part is the penetrative nature of the conversation, e.g. how the quite-functional-but-not-hot technology becomes the sick gazelle falling behind the herd, even as it is not empirically sick.
And then the ratchet turns, and the pendelum swings in reverse, and suddenly that thing sent from god was instead discovered to be sent from satan... and you're still trying to explain that tradeoffs exist
That same saying would also work replacing XML for soap (but not SOAP).
This coming from someone who hates XML, avoids it at nearly all costs - there are times it's the right tool for the job. And you list that time. If you have a 1-1 API contract, don't use XML. If you have 1-N, don't use XML. But if you have N-N API contracts? Where you're talking to half a dozen other services, and they're all talking to you? XML is an excellent choice.
Good times.
Fine, but with for example, Protobuf, you share a .proto file and so there's zero percentage chance of this happening.
The best part is that these are a superset of JSON and could easily be added if there would just be more buy-in (lots of unofficial support already exists).
There are more. I don't know if java dot-properties file is worth including.
[0]: https://en.wikipedia.org/wiki/Canonical_S-expressions
It's both binary and text, and supports all the common data types, and doesn't require a schema or extra compilation steps.
Reality: Author somehow overlooked the fact that MS Office documents are all bundled xml files behaving exactly as intended
Lesson: One guy’s experience is not enough to decide which languages you use or how you build software
The idea of JSON is to offer two important ways to aggregate, which are different enough that they deserve separate treatment - ordered array and associative array. They are economically represented, with [] and {}, and internal structure is also made with small costs - commas (,) between elements and colons (:) for keys in associative array. We can argue that both of those features are required.
Chosen primitives - numbers, strings, booleans and null - are also selected from what's regularly and consistently used, and the choice is supported by two decades of JSON application. Roots of this selection are in design of JavaScript, which in turn relied on common practice for basic primitives.
If JSON is considered as what I think it is, structure out of basic primitives, it's at the optimal point. Crockford's decision to avoid JSON versioning makes a good sense.
Applications of JSON - human-readable texts, performance-optimized communications - can suffer from e.g. lack of built-in comments, non-extensible "type system", non-optimal bandwidth usage (if direct ASCII or even UTF-8 is used for JSON encoding, not some other compressed approach). They however can be fixed by building on top of JSON. For example, parties can agree upon reserved keys and compression schemas, constraints in form of grammars (JSON-Schema). This flexibility stems from the fact that JSON is at the very foundation of data structures, so "better than JSON" is to an extent like "better than two's complement number representation".
People forget this, but ASCII (and therefore UTF-8) already has characters in it for field/record/etc. separation. One increases complexity significantly (and introduces inefficiency) by overloading other characters with their semantics.
There are much more simple & efficient ways to demarcate ordered arrays in particular (e.g. simple length prefixed structures).
Nulls are highly problematic for any number of reasons that are widely documented, and JSON unfortunately compounds this by having two ways of representing them (explicitly and by omission).
One might quibble with the notion that a format that didn't even exist in 2000 doesn't really have two decades worth of applications working on it, but setting that aside, the design constraints of JavaScript and the challenges that have emerged from it are far from ideal. The work arounds that have emerged (the whole Number fiasco, the "standardizing" on UTF-8 while using UTF-16 when escaping, line terminators, the backslash escapign of control characters, the inconsistencies between JSON & JSONP, the binary64 "good interoperability" rule, and comments being disallowed, and of course the various JSON Schema efforts).
While I agree that an ideal serialization standard would not have versioning, and Crockford's decision to avoid JSON versioning might consequently seem to make good sense... in practice this has lead to significant inconsistencies between implementations, many of which lead to interoperability problems that could at least be somewhat mitigated by versioning. (http://seriot.ch/parsing_json.php)
"Fixing" a bad abstraction by layering on a bad abstraction is a common house of cards meme. It is highly problematic to say the least. It's how "simple" JSON rapidly evolves to be at least as complex (and at least as error prone) as ASN.1.