I'll probably listen to this at some point and genuinely like hearing about the origins of JSON from Crockford himself.
Apart from that, I'd be surprised if this adds anything to what I've already heard and read ad nauseam, including from Crockford himself.
JSON is just a nice, simple serialization format that compresses well and interoperates acceptly with most programming languages. Most importantly with the language that it was designed to be consumed by: Javascript.
If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf, and/or ASN.1 (was that even correctly spelled?) or any of the countless other formats I have no clue about.
If you need better schema restrictions and interconnected entities + circular references: use a database, I guess.
At least everything XML has done in this space has more or less failed (e.g. SOAP). Well, failed might be too strong a word.
But all the work on defining data types and schema seems almost independent of the underlying syntax.
Since I've now spent too much time on this uninformed first comment, I will now listen what Douglas Crockford wants to say. But I think I've already read this story in another form a couple of years ago.
JSON is really a good example of "worse is better". E.g. an OpenAPI spec with automated client code and TypeScript definitions is not much different from SOAP on the surface level.
1: text with markup. Does not need all the XML machinery, but tags make a lot of sense here.
1b: Arbitrarily nested stuff
2: extensability by third parties without coordination. Namespaces as a central concept help out a lot here, and this use case can even justify some of the more insane features of XML (downloading remote schemas etc)
If you don't need either of those things, you won't get much value out of XML. If you do need them, XML can be neat. Certainly easier than trying to allow people to add arbitrary data (with validation) to your protobuf. It's just that these use cases are somewhat rare
I was recently downloading and parsing the US Patent application database, which is emitted in XML.
In 2023, Python lxml can't handle multiple documents per file. And no one has any suggestions on how to handle this other then "break on the new document tag". Other libraries didn't look promising in other languages.
Then if you try to read it, you need to disable all the DTD stuff because it immediately tries to open other local files on disk...which it can't find. There's some corruption in how it parses DTDs because I had to implement a custom DTD handler in my jupyter-notebook to read it.
I could blame the dominant XML processing library of Python, I could blame the lack of attention XML gets, but really? This is just another contribution in a long line of XML not working in some way in every language as soon as you're not using that languages specific XML flavor.
I haven't had to work with XML all that often. For the few programs I've made that need to speak XML, I've always used the XML module in Python's stdlib. Is that bad practice? What does lxml provide as a benefit?
XML tends to be more feature-rich than people need, but what's really lacking in JSON is any support for using custom types to disambiguate polymorphism.
I can't seem to find it, but there's a serialization that's basically Python, so
Stuff(a=10, b={"foo": bar})
It makes it easy to have a list with objects of different types in it.
> what's really lacking in JSON is any support for using custom types to disambiguate polymorphism
Unless I misunderstood you, you can use tagged unions[1] for this so not sure JSON needs any change there. Slightly more verbose but shouldn't be an issue in practice.
Exactly, they could have added an extra wire type to signify messages without costing any extra bytes. Nothing is stopping ProtoBuf from being self describing except incompetence.
Yeah I had SVG in mind when I wrote that post but I thought of different use cases.
Of course every document format can also be called a serialization format.
I enjoy JSX as well (and even SOAP!). That's why I weakened the quoted sentence in my comment.
When I hear "JSON vs XML" I can't help but mainly think of data payloads in web applications, especially non-static client/server communication.
It's worth listening to, or at least reading the transcript, just for the Dave Winer burn. (And so much more!)
>Adam: That was it. That was the creation of JSON, which everyone is using today. But back then, everyone rejected it.
>In some ways it was a marketing problem. On one side, you had Doug, trying to convince customers that they can build interactive applications on the web using JavaScript and this simple thing called JSON. But on the other side, you had XML that had these big companies behind it, IBM, Microsoft, and big consultants. And later they even had some tech influencers like Dave Winer.
>Douglas: He’s someone who should have known better. He had a website called scripting.com. His style of scripting came from a clever program that he had written for the Macintosh called Frontier, in which he had a scripting language and an outliner and a word processor and a database, all in one program. And the idea was that you could do virtually anything in Frontier with a little bit of scripting. And he was also one of the big promoters of SOAP, the Simple Object Annoying Protocol.
>I don’t remember what the A was, but it might have been atrocious or abominable, I don’t know.
>But SOAP was a big deal at the time. They were right in wanting simplicity. They didn’t accomplish it, but they put simplicity in the name as sort of an aspirational thing. And so, when I started showing how JSON works, he was really threatened by that. And on his website, which was well-read at the time, he complained that, “this isn’t even XML. We should find who did this in string them up now”, which was a really ugly thing to say.
>Fortunately, nobody listens to Dave Winer, so I’m still here.
Dave's done some brilliant influential stuff, which Doug credits and I've written about before, but being annoying is Dave's brand, so Doug's "SOAP" joke is dead on. It's just as fair as referring to Marc Canter's "People Aggregator" as "People Aggravator".
I did listen to the podcast lateron and was pleasantly surprised by the breadth of topics, although I have to admit I fell asleep halfway through (not because of the contents).
Thanks a lot for the money quotes! Casual conversation in English is still sometimes hard to listen to for me without focusing a lot.
> a clever program that he had written for the Macintosh called Frontier, in which he had a scripting language and an outliner and a word processor and a database, all in one program
Fascinating concept. I found a bit more info in an article about UserLand Software, which Dave Winer founded after leaving Symantec.
> In January 1992 UserLand released version 1.0 of Frontier, a scripting environment for the Macintosh which included an object database and a scripting language named UserTalk. At the time of its original release, Frontier was the only system-level scripting environment for the Macintosh, but Apple was working on its own scripting language, AppleScript, and started bundling it with the MacOS 7 system software. As a consequence, most Macintosh scripting work came to be done in the less powerful, but free, scripting language provided by Apple.
> UserLand responded to Applescript by re-positioning Frontier as a Web development environment, distributing the software free of charge with the "Aretha" release of May 1995. In late 1996, Frontier 4.1 had become "an integrated development environment that lends itself to the creation and maintenance of Web sites and management of Web pages sans much busywork," and by the time Frontier 4.2 was released in January 1997, the software was firmly established in the realms of website management and CGI scripting, allowing users to "taste the power of large-scale database publishing with free software."
> Frontier's NewsPage suite came to play a pivotal role in the emergence of blogging through its adoption by Jorn Barger, Chris Gulker, and others in the 1997–98 period.
> UserLand launched a Windows version of Frontier 5.0 in January 1998 and began charging for licenses again with the 5.1 release of June 1998.
> Frontier subsequently became the kernel for two of UserLand's products, Manila and Radio UserLand, as well as Dave Winer's OPML Editor, all of which support the UserTalk scripting language.
I have worked with a supplier / service who advertised us their brand new shiny REST/JSON based API that we could use to replace the drop a csv in a sftp workflow they offered previously.
The API was a post with a single JSON field called data, with a CSV string value. You just know they were asked by management to deliver an API and decided to do quite literally the minimum viable.
It’s MVP because it (1) is an API and (2) is doing nothing more than writing it’s output to the same sftp server the client used to hit directly. Not debating it’s dumb, but almost by definition enterprise tech ossifies in weird, dumb ways.
I've seen similar in the wild with XML, except the charset was wrong and escaping was incomplete.
So you get the XML header with UTF8 encoding, a root tag, an indented child tag. 3 lines. Then ye olde CSV, with the 5 required entities for ascii escaped, but if you manage to put accented chars in it, they end up in ISO 8859-1. Then 2 lines footer.
If you wrote out the file, the header and footer had to be byte for byte identical. Entities encoding non ascii just crashed the parser. Of course, the CSV lines were sensitive to indentation, they just dropped the first 8 chars of each line.
So many seem to have "base64 damage", they get confused when they don't have to base64 encode something. To attach documents (e.g. scans) to a given order in our API, you just POST the raw file directly to the endpoint with appropriate content type. Simple.
Yet I get so many questions like "but we base64-encode them first?" or similar. One dev mentioned he'd never done such a "raw" POST.
This interview is as much about the history of early web development and the serendipitous process by which JSON won out. After reading, it's apparent that JSON could have easily lost and been looked over entirely. Crockford's persistence in evangelizing it even after the company he worked at when it was created went bankrupt led to its eventual adoption.
> Crockford's persistence in evangelizing it even after the company he worked at when it was created went bankrupt led to its eventual adoption.
I don't think this is true at all. The reason that JSON won is stated quite clearly in the interview: you do not need a software stack to handle JSON. This feature sells itself for any software development team who needs to have frontends running in a browser consuming data from a backend. You do not need an evangelist to convince people to use a feature that is ubiquitous, free, extensively tested, and more performance than anything in the market.
There's plenty of historical examples of sub-optimal standards developing enough traction and network effects that the whole industry is stuck with them for decades. Initially you did need to pull in a library to parse JSON, it wasn't part of the JS language until Crockford pushed for it (via his later position of authority at Yahoo working on YUI).
Particularly in the early web, the need for async "liveness" of pages didn't exist, so pages didn't consume backends directly. Everything was handled on refresh or navigation and server rendered as HTML. It was only after AJAX-style web apps took off (post-Gmail, really), that JSON started to get real traction. There were no JSON parsers in C#, Java, Python, etc initially. XML was far more common and SOAP was generally used for cross-endpoint communication.
I've made do with .to_xml, use xlst on results, then .to_json. It definably works (with some caveats) but would love to cut out the middleman. I believe xlst 3.0 added explicit json support but I've yet to give it a try: https://www.saxonica.com/papers/xmlprague-2016mhk.pdf
It was a really good listen for the historical perspective alone. I did not know how much bad blood there was between Dave Winer and Crockford. Winer was a co-creator of XML-RPC, but REST and RPC are two different use cases. I guess they were not so clearly delineated at the time.
In practice they're not so delineated today either. How often do you find RPCs labeled "REST-like API". Few "REST-like" APIs actually apply HATEOAS, etc.
While I have no inarguable proof to refute you, and maybe it's easy to sound cool when history has proven one correct, my characterization of "humble genius" is based on DC's low-key presentation within the podcast.
At no point did I perceive him to assert any sort of personal superiority.
While I use his JSON creation heavily, JavaScript as such is the second least attractive language, after PHP, in my view.
at no point did you think that him saying "I was right all along and everyone who opposed me was wrong" is not a message of personal superiority? giving that message is to claim superiority over others, whether it is intentional or not.
this podcast is the most humble I have ever heard him, to be fair, but make no mistake; DC is not a humble person.
A humble person would not have said "JSON was my/our idea" or "I was right"; a humble person would not be on a podcast talking about themselves at all.
Side note-- I can just feel the frontend dev's juice running out as they put the finishing touches on the volume slider on that audio player. It suspiciously only updates the attenuation on mouseup.
Here's my hypothesis: they initially implemented it the obvious way on mousemove (or whatever similar immediate event). Then they a) noticed the horrible zipper noise that came from doing that, b) felt dread at the idea of plumbing into the webaudio API, c) said "aw fuck it" out loud and d) bound the event to mouseup.
Does the dev of that framework post here? If so, am I close on this one? If so I can commiserate from the opposite end-- I spent a long time getting audio stuff like this just right and then fudging text alignment for lack of juice.
Any changes in waveforms that are not smooth can cause percussive sounds (that's what percussions are). Playing audio suddenly causes a pop, and changing audio levels abruptly causes a weird zipper sound (the sound of a thousand MIDI drums playing in a few milliseconds).
The reasons are very interesting: your brain does a Fourier Transform (sort of) in real time to listen to audio. It allows you to separate two people or two instruments producing sounds at the same time, and abrupt changes will cause this transform to product artefacts.
The right way of doing it is to always make progressive changes to the gain level. This property is often used in audio engineering to "add or remove punch" in recordings: you can decrease the attack level to make the gain changes more aggressive and add punch, or lower it to make it smoother, over do it and it sounds muffled. In more concrete words: use the setTargetAtTime WebAudio API.
It's more the cochlea performing some kind of convolution of roughly sinusoidal wavelets with the audio signal, bringing the signal from temoporal to frequency domain, right? Does more neurological post-processing the cochlear output bring it closer to a Fourier transform?
Both are very similar and have a lot of parallels, enough that I generally call what humans do "kind of Fourier Transform", but "convolution of roughly sinusoidal wavelets" is definitely more correct.
For example, you can explain to someone with not much signal knowledge why a 15kHz square wave sounds exactly like a 15kHz sine wave to humans by looking at the harmonics through a Fourier Transform.
I do believe most of the magic happens in the brain though, the cochlea is a super advanced biological microphone, but the brain is responsible for harmonic and rhythmic processing. The brains figures out what is a voice, an instrument, or a percussion. My understanding is that it stores short-term temporal information needed to make sense of realtime information: a note alone might be perceived differently depending on the harmonic and rhythmic context ("wrong note").
Said another way, an instantaneous change in gain causes a step change in the audio signal. A step change isn't physically possible, but is equivalent to the summation of harmonic frequencies up to infinity. Your speakers try their best, and end up playing all the harmonic frequencies within their range of 20Hz-20Khz or so. This happens to produce a similar sound as a percussion instrument.
It's simpler than that. It's an <input type="range"> element with a 'change' event listener, which is the most obvious way to implement a volume slider. The 'change' event fires on mouseup.
> I think Object-Oriented Programming was an important advance over procedural programming, which is what we had been doing, but we’re stuck. We’re supposed to change paradigms every generation, and we haven’t. So we’re way behind now. Object-Oriented Programming hit the mainstream in 1980. We’re in the 2020s and we’re still there. We should have moved on.
Functional programming is the next big thing. Ideas from functional programming have gone mainstream, and are found in almost all major languages: lambdas, higher order functions, and immutability / purity. Monads have made some minor inroads (although they get bastardized by OO people, see Java and Javascript).
There are still remnants of ideological OOP use around and they are terrible indeed, but remnants on the language level are perfectly fine as long as they are not used as originally intended. Paraphrasing Crockford, we have found OOP, the good parts. It's used for structuring code now, not for modelling the world. That's a huge difference.
Well if people wouldn't keep brutally deconstructing it [1] then serving it up as a delicious la carte menu [2] of tempting indulgences, maybe object programming will finally die!
Well, we're heading for a future where no one programs and everyone just prompts chatgpt. So it seems the future of programming languages is no more programming languages.
So, "in the future" how do people create programs expressing their intent? They'd write or speak in some natural language, right?
1. In order to avoid ambiguities they'd need to use concrete vocabulary.
2. Any non-trivial program will need to consist of multiple input statements. Those input statements will have some kind of relationship to each other. This is roughly what a grammar is about.
3. Those future programmers will somehow need to check if the program does what it's supposed to do. They'd run it with test data.
Combine 1-3 and see that we just described, what a programming language and its ecosystem (compiler, runtime, etc.) do. Thus, programming languages are nowhere near to die out, they will just shift things higher up the layer lasagne.
We already have that, considering gpt requires a lot of code to train itself, the transition to a new language might not be worth it, so we might be stuck with the ones we have now.
That argument sounds like change for the sake of change. Programmed obsolescence for programming languages.
We should move on when we find something that is objectively better, not because some years have passed.
And honestly, the only new thing that seems to be better than the current paradigm is Rust borrow model, which is not against object orientation, it compliments it.
While this is true, I'd like to point out that the switch to the currently prevailing flavor of OOP also didn't really fit that description all that much. Note that by now, other advancements like assembly vs. raw machine code would be pretty much a no-brainer.
The 90s-flavor OOP we're stuck with in many places is objectively bad for performance (see e.g. Mike Acton's takes on what he calls "Data-Oriented Design") as well as readability/understandability (where I'd say languages with a culture around more FP concepts and immutability have a way better track record) and concurrency (see Erlang/Elixir).
And, as always, you can simulate the useful properties of other paradigms, but that only goes so far. I think what the original quote laments is that the median software developer is leaning way more on "we've always done it this way" as a way of guidance instead of better ideas replacing the old ones.
Edit: Also: I totally agree with you about Rust, but I can't help but notice that Rust mainly stops you from doing dumb things (no inheritance, borrow checker…), which effectively dials back many of the… more unfortunate concepts/idioms of 90s OOP.
Sounds like you have missed the "programming in XML" phase that triggered Crockford to do JSON: huge configuration languages, usually with almost good schemas. And nothing in there could ever fail before runtime. It was driven by a deep fear of programming, a desperate hope that maybe you could build something so configurable that you would reach a state of done after which you would never ever have to send the source through a compiler again.
In reality however, often the best way to understand a problem in those configuration monsters (or the only way...) was to recompile the non-xml part with some tailored ad-hoc diagnostics. But the idea that the configuration was supposed to change a lot while the code is immutable proved surprisingly robust against ongoing demonstrations of exactly the opposite happening.
>Douglas: Some of the people who were doing JavaScript, were doing it badly and didn’t want to be told that they were doing it badly. There is this notion among second rate programmers that the most important thing they do is express themselves. It’s not making programs that work well and are free of error. That’s way down on the list. What’s much more important is expressing themselves, that, I’m an artist and I express my arts by leaving out semicolons. It’s that kind of thing. So what I was saying did not resonate with that at all. So that audience was not happy to hear it, and JavaScript was just not being used.
If you could have this instead:
[{"columns": {"field1", "field2", "field3"}, "rows": [{"value1,"value2","value3"},{"value1,"value2","value3"},{"value1,"value2","value3"}]
If json parser could then have the smarts to return the values by position for the various field names even though each row does not include the field names.
It's JavaScript Object Notation. An array of objects looks like that, if you want to have another data structure you can do it and have the accompanying code to consume it.
The only thing holding you back from doing that is 1kb of JS as the second argument to JSON.parse
JSON is not foundational, actually harmful because it ditches the main idea of web (linking things) for pushing around schemaless hodgepodge data structures...
There is no contradiction. Json simply asserts that the basic exchange blob should be as simple as possible.
Linking and schemas and validation and all that stuff are terribly important but a good and simple approach would ADD them when needed, not inflict you with maximum complexity for even the simplest thing.
In this respect json-ld has shown one way this can be done but overall its a problem that still asks for a brilliant innovator to solve.
There is no complexity in linking, there's nothing simpler than RDF. The whole reason you think it's complex is... drumroll... JSON. It made these bespoke unlinkable structures proliferate and de facto standard. Later JSON-LD tried to bring it back to sanity but because it still has to be JSON (because devs now hate everything else) the end result is complex, inscrutable (contexts?? @reverse???) and usually misunderstood.
Linking is a conceptual requirement, not a format feature. It is a more complex requirement so it naturally needs more complex semantics to implement. So far so unavoidable.
A good design would allow progressively adding semantic complexity iff needed. I think the popularity of json was just because xml violated this very simple design aspect.
Unfortunately it is true that linking, metadata, ontologies and all that stuff are required as the ambition of information exchange increases. People who deny that obviously have never worked on any domain that requires significant data exchanges between many independent actors.
So now we are stuck with two sub-optimal solutions: a simple but incomplete json approach that must reinvent the wheel without destroying its good bits and a complex and hard to use xml approach that is only loved by software bureaucrats...
Absolutely loved that interview. So much of current tools we use seem to come from nowhere, but no, there are talented visionaries behind them with very clear ideas of what good looks like and an ability to materialize it. Thankfully some of them succeed.
But its a painful slow process as there are not enough of them practical innovators. Big corp and consultants have an amazing ability to keep things stagnant for ages and milk complexity.
But the json revolution is definitely unfinished. Standards are important and so many domains have adopted XML already. A practical way to jsonify all these specs would be really useful.
Assuming you mean maps where the keys are ordered, then that is something JSON Schema could, in theory, specify[1].
However they seem to be against it primarily because it's not JSON (keys are explicitly unordered) and that means some generators might be unable to produce valid output, so interoperability is tricky.
A lot of this reminds me of present day web frameworks.
"But you HAVE TO have a virtual DOM. You HAVE TO write all this boilerplate and ship a fat runtime. You HAVE TO manage the caching details to these performance problems and abstraction leaks (looking at you, useMemo!). You have to WRAP AND HIDE the HTML and CSS—which breaks accessibility, so you need to learn all the implementation details about a11y and manually add it back one ARIA role at a time (or just assert it's too hard, so less accommodation)."
All this when there are alternatives that are much simpler, so much easier, so much less code, and so much more consistent, folks constantly downplay it. "It CAN'T be used for large projects. You HAVE TO make this large object hierarchy to make web apps. You CAN'T rely on a compiler, because that's magic and no longer 'pure' JavaScript."
I especially love the "magic" argument since it completely glosses over the fact that we're providing info to processed sand we hit with lightning to make it think. Magic is already part of the bargain from your first "Hello World."
Only now it's not "is that a standard?" It's "everyone's doing it the unnecessarily hard way, and we've invested too much time learning all the ins and outs of the unnecessarily hard way, so if you want a job, you'd better get used to the hard way and suffer as we have."
105 comments
[ 3.3 ms ] story [ 152 ms ] threadI'll probably listen to this at some point and genuinely like hearing about the origins of JSON from Crockford himself.
Apart from that, I'd be surprised if this adds anything to what I've already heard and read ad nauseam, including from Crockford himself.
JSON is just a nice, simple serialization format that compresses well and interoperates acceptly with most programming languages. Most importantly with the language that it was designed to be consumed by: Javascript.
If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf, and/or ASN.1 (was that even correctly spelled?) or any of the countless other formats I have no clue about.
If you need better schema restrictions and interconnected entities + circular references: use a database, I guess.
At least everything XML has done in this space has more or less failed (e.g. SOAP). Well, failed might be too strong a word.
But all the work on defining data types and schema seems almost independent of the underlying syntax.
Since I've now spent too much time on this uninformed first comment, I will now listen what Douglas Crockford wants to say. But I think I've already read this story in another form a couple of years ago.
JSON is really a good example of "worse is better". E.g. an OpenAPI spec with automated client code and TypeScript definitions is not much different from SOAP on the surface level.
1: text with markup. Does not need all the XML machinery, but tags make a lot of sense here.
1b: Arbitrarily nested stuff
2: extensability by third parties without coordination. Namespaces as a central concept help out a lot here, and this use case can even justify some of the more insane features of XML (downloading remote schemas etc)
If you don't need either of those things, you won't get much value out of XML. If you do need them, XML can be neat. Certainly easier than trying to allow people to add arbitrary data (with validation) to your protobuf. It's just that these use cases are somewhat rare
In 2023, Python lxml can't handle multiple documents per file. And no one has any suggestions on how to handle this other then "break on the new document tag". Other libraries didn't look promising in other languages.
Then if you try to read it, you need to disable all the DTD stuff because it immediately tries to open other local files on disk...which it can't find. There's some corruption in how it parses DTDs because I had to implement a custom DTD handler in my jupyter-notebook to read it.
I could blame the dominant XML processing library of Python, I could blame the lack of attention XML gets, but really? This is just another contribution in a long line of XML not working in some way in every language as soon as you're not using that languages specific XML flavor.
Java and JSON is a different story. There was a window when the only JSON parser Java shipped with was JSON.parse() in its Javascript interpreter.
https://www.reddit.com/r/programming/comments/eaqgk/java_is_...
"I Wanna Be <![CDATA[" Sung to the tune of “I Wanna Be Sedated”, with apologies to The Ramones.
https://donhopkins.medium.com/i-wanna-be-cdata-3406e14d4f21
I can't seem to find it, but there's a serialization that's basically Python, so
It makes it easy to have a list with objects of different types in it.Unless I misunderstood you, you can use tagged unions[1] for this so not sure JSON needs any change there. Slightly more verbose but shouldn't be an issue in practice.
[1]: https://en.wikipedia.org/wiki/Tagged_union
ProtoBuf is a horrible format. its not self describing like JSON, when it easily could be, without costing any extra bytes.
like if I have a message inside of a message vs just a bytes field that happens to have a valid protobuf?
Bwhahahaha good one
SVG
DOCX
SAML
JSX
The legacy of XML incalculable and expanding.
Xml excels when the content must be human consumed and machine readable.
I enjoy JSX as well (and even SOAP!). That's why I weakened the quoted sentence in my comment.
When I hear "JSON vs XML" I can't help but mainly think of data payloads in web applications, especially non-static client/server communication.
I wouldn't want JSON instead of SVG either :)
>Adam: That was it. That was the creation of JSON, which everyone is using today. But back then, everyone rejected it.
>In some ways it was a marketing problem. On one side, you had Doug, trying to convince customers that they can build interactive applications on the web using JavaScript and this simple thing called JSON. But on the other side, you had XML that had these big companies behind it, IBM, Microsoft, and big consultants. And later they even had some tech influencers like Dave Winer.
>Douglas: He’s someone who should have known better. He had a website called scripting.com. His style of scripting came from a clever program that he had written for the Macintosh called Frontier, in which he had a scripting language and an outliner and a word processor and a database, all in one program. And the idea was that you could do virtually anything in Frontier with a little bit of scripting. And he was also one of the big promoters of SOAP, the Simple Object Annoying Protocol.
>I don’t remember what the A was, but it might have been atrocious or abominable, I don’t know.
>But SOAP was a big deal at the time. They were right in wanting simplicity. They didn’t accomplish it, but they put simplicity in the name as sort of an aspirational thing. And so, when I started showing how JSON works, he was really threatened by that. And on his website, which was well-read at the time, he complained that, “this isn’t even XML. We should find who did this in string them up now”, which was a really ugly thing to say.
>Fortunately, nobody listens to Dave Winer, so I’m still here.
Dave's done some brilliant influential stuff, which Doug credits and I've written about before, but being annoying is Dave's brand, so Doug's "SOAP" joke is dead on. It's just as fair as referring to Marc Canter's "People Aggregator" as "People Aggravator".
https://news.ycombinator.com/item?id=20780928
https://news.ycombinator.com/item?id=21170440
https://news.ycombinator.com/item?id=16224154
https://www.gawker.com/183778/people-aggravator-marc-canter-...
Thanks a lot for the money quotes! Casual conversation in English is still sometimes hard to listen to for me without focusing a lot.
Fascinating concept. I found a bit more info in an article about UserLand Software, which Dave Winer founded after leaving Symantec.
https://en.wikipedia.org/wiki/UserLand_Software#Frontier
> In January 1992 UserLand released version 1.0 of Frontier, a scripting environment for the Macintosh which included an object database and a scripting language named UserTalk. At the time of its original release, Frontier was the only system-level scripting environment for the Macintosh, but Apple was working on its own scripting language, AppleScript, and started bundling it with the MacOS 7 system software. As a consequence, most Macintosh scripting work came to be done in the less powerful, but free, scripting language provided by Apple.
> UserLand responded to Applescript by re-positioning Frontier as a Web development environment, distributing the software free of charge with the "Aretha" release of May 1995. In late 1996, Frontier 4.1 had become "an integrated development environment that lends itself to the creation and maintenance of Web sites and management of Web pages sans much busywork," and by the time Frontier 4.2 was released in January 1997, the software was firmly established in the realms of website management and CGI scripting, allowing users to "taste the power of large-scale database publishing with free software."
> Frontier's NewsPage suite came to play a pivotal role in the emergence of blogging through its adoption by Jorn Barger, Chris Gulker, and others in the 1997–98 period.
> UserLand launched a Windows version of Frontier 5.0 in January 1998 and began charging for licenses again with the 5.1 release of June 1998.
> Frontier subsequently became the kernel for two of UserLand's products, Manila and Radio UserLand, as well as Dave Winer's OPML Editor, all of which support the UserTalk scripting language.
We truly live in a world
The API was a post with a single JSON field called data, with a CSV string value. You just know they were asked by management to deliver an API and decided to do quite literally the minimum viable.
They had 'no plans' to expand it.
So you get the XML header with UTF8 encoding, a root tag, an indented child tag. 3 lines. Then ye olde CSV, with the 5 required entities for ascii escaped, but if you manage to put accented chars in it, they end up in ISO 8859-1. Then 2 lines footer.
If you wrote out the file, the header and footer had to be byte for byte identical. Entities encoding non ascii just crashed the parser. Of course, the CSV lines were sensitive to indentation, they just dropped the first 8 chars of each line.
Still, XML was officially supported.
But... What do would you do instead?
Yet I get so many questions like "but we base64-encode them first?" or similar. One dev mentioned he'd never done such a "raw" POST.
Not surprisingly, they didn't put any example JSON in their API documentation.
I don't think this is true at all. The reason that JSON won is stated quite clearly in the interview: you do not need a software stack to handle JSON. This feature sells itself for any software development team who needs to have frontends running in a browser consuming data from a backend. You do not need an evangelist to convince people to use a feature that is ubiquitous, free, extensively tested, and more performance than anything in the market.
Particularly in the early web, the need for async "liveness" of pages didn't exist, so pages didn't consume backends directly. Everything was handled on refresh or navigation and server rendered as HTML. It was only after AJAX-style web apps took off (post-Gmail, really), that JSON started to get real traction. There were no JSON parsers in C#, Java, Python, etc initially. XML was far more common and SOAP was generally used for cross-endpoint communication.
[0]: https://news.ycombinator.com/item?id=35467711 [1]: https://news.ycombinator.com/item?id=35452861 [2]: https://news.ycombinator.com/item?id=35429976 [3]: https://news.ycombinator.com/item?id=35427394
genius? lol
he is certainly not humble and he's no smarter than anyone else. right place, right time.
his LACK of humility is what gave him the persistence required.
At no point did I perceive him to assert any sort of personal superiority.
While I use his JSON creation heavily, JavaScript as such is the second least attractive language, after PHP, in my view.
this podcast is the most humble I have ever heard him, to be fair, but make no mistake; DC is not a humble person.
A humble person would not have said "JSON was my/our idea" or "I was right"; a humble person would not be on a podcast talking about themselves at all.
Here's my hypothesis: they initially implemented it the obvious way on mousemove (or whatever similar immediate event). Then they a) noticed the horrible zipper noise that came from doing that, b) felt dread at the idea of plumbing into the webaudio API, c) said "aw fuck it" out loud and d) bound the event to mouseup.
Does the dev of that framework post here? If so, am I close on this one? If so I can commiserate from the opposite end-- I spent a long time getting audio stuff like this just right and then fudging text alignment for lack of juice.
The reasons are very interesting: your brain does a Fourier Transform (sort of) in real time to listen to audio. It allows you to separate two people or two instruments producing sounds at the same time, and abrupt changes will cause this transform to product artefacts.
The right way of doing it is to always make progressive changes to the gain level. This property is often used in audio engineering to "add or remove punch" in recordings: you can decrease the attack level to make the gain changes more aggressive and add punch, or lower it to make it smoother, over do it and it sounds muffled. In more concrete words: use the setTargetAtTime WebAudio API.
For example, you can explain to someone with not much signal knowledge why a 15kHz square wave sounds exactly like a 15kHz sine wave to humans by looking at the harmonics through a Fourier Transform.
I do believe most of the magic happens in the brain though, the cochlea is a super advanced biological microphone, but the brain is responsible for harmonic and rhythmic processing. The brains figures out what is a voice, an instrument, or a percussion. My understanding is that it stores short-term temporal information needed to make sense of realtime information: a note alone might be perceived differently depending on the harmonic and rhythmic context ("wrong note").
> I think Object-Oriented Programming was an important advance over procedural programming, which is what we had been doing, but we’re stuck. We’re supposed to change paradigms every generation, and we haven’t. So we’re way behind now. Object-Oriented Programming hit the mainstream in 1980. We’re in the 2020s and we’re still there. We should have moved on.
[1] https://www.info.ucl.ac.be/~pvr/decon.html
[2] http://www.paulgraham.com/reesoo.html
1. In order to avoid ambiguities they'd need to use concrete vocabulary.
2. Any non-trivial program will need to consist of multiple input statements. Those input statements will have some kind of relationship to each other. This is roughly what a grammar is about.
3. Those future programmers will somehow need to check if the program does what it's supposed to do. They'd run it with test data.
Combine 1-3 and see that we just described, what a programming language and its ecosystem (compiler, runtime, etc.) do. Thus, programming languages are nowhere near to die out, they will just shift things higher up the layer lasagne.
We should move on when we find something that is objectively better, not because some years have passed.
And honestly, the only new thing that seems to be better than the current paradigm is Rust borrow model, which is not against object orientation, it compliments it.
The 90s-flavor OOP we're stuck with in many places is objectively bad for performance (see e.g. Mike Acton's takes on what he calls "Data-Oriented Design") as well as readability/understandability (where I'd say languages with a culture around more FP concepts and immutability have a way better track record) and concurrency (see Erlang/Elixir).
And, as always, you can simulate the useful properties of other paradigms, but that only goes so far. I think what the original quote laments is that the median software developer is leaning way more on "we've always done it this way" as a way of guidance instead of better ideas replacing the old ones.
Edit: Also: I totally agree with you about Rust, but I can't help but notice that Rust mainly stops you from doing dumb things (no inheritance, borrow checker…), which effectively dials back many of the… more unfortunate concepts/idioms of 90s OOP.
When the hot new thing in web dev is...
...
...
JavaScript XML (JSX)
> https://facebook.github.io/jsx/
So yes, it's not XML.
It's JavaScript XML.
This line made me think about the many discussions about Python syntax and the value of having brackets to know when a code block ends and starts.
Balanced named tags are another step to make something easier to visually parse, at the cost of having much more text.
In reality however, often the best way to understand a problem in those configuration monsters (or the only way...) was to recompile the non-xml part with some tailored ad-hoc diagnostics. But the idea that the configuration was supposed to change a lot while the code is immutable proved surprisingly robust against ongoing demonstrations of exactly the opposite happening.
>Douglas: Some of the people who were doing JavaScript, were doing it badly and didn’t want to be told that they were doing it badly. There is this notion among second rate programmers that the most important thing they do is express themselves. It’s not making programs that work well and are free of error. That’s way down on the list. What’s much more important is expressing themselves, that, I’m an artist and I express my arts by leaving out semicolons. It’s that kind of thing. So what I was saying did not resonate with that at all. So that audience was not happy to hear it, and JavaScript was just not being used.
Since when did Yahoo lose its "!"?
example: instead of the following:
[{"field1":"value", "field2":"value", "field3":"value"}, {"field1":"value1", "field2":"value2", "field3":"value3"}, {"field1":"value3", "field2":"value3", "field3":"value4"}]
If you could have this instead: [{"columns": {"field1", "field2", "field3"}, "rows": [{"value1,"value2","value3"},{"value1,"value2","value3"},{"value1,"value2","value3"}]
If json parser could then have the smarts to return the values by position for the various field names even though each row does not include the field names.
The only thing holding you back from doing that is 1kb of JS as the second argument to JSON.parse
JSON is not foundational, actually harmful because it ditches the main idea of web (linking things) for pushing around schemaless hodgepodge data structures...
Linking and schemas and validation and all that stuff are terribly important but a good and simple approach would ADD them when needed, not inflict you with maximum complexity for even the simplest thing.
In this respect json-ld has shown one way this can be done but overall its a problem that still asks for a brilliant innovator to solve.
There is no complexity in linking, there's nothing simpler than RDF. The whole reason you think it's complex is... drumroll... JSON. It made these bespoke unlinkable structures proliferate and de facto standard. Later JSON-LD tried to bring it back to sanity but because it still has to be JSON (because devs now hate everything else) the end result is complex, inscrutable (contexts?? @reverse???) and usually misunderstood.
A good design would allow progressively adding semantic complexity iff needed. I think the popularity of json was just because xml violated this very simple design aspect.
Unfortunately it is true that linking, metadata, ontologies and all that stuff are required as the ambition of information exchange increases. People who deny that obviously have never worked on any domain that requires significant data exchanges between many independent actors.
So now we are stuck with two sub-optimal solutions: a simple but incomplete json approach that must reinvent the wheel without destroying its good bits and a complex and hard to use xml approach that is only loved by software bureaucrats...
Ah, but we have schemas now! https://json-schema.org/
I've been able to convert our XSD's 1:1 to JSON Schemas, including choice elements. I'm sure there's not a 100% overlap but most XSDs could I think.
But its a painful slow process as there are not enough of them practical innovators. Big corp and consultants have an amazing ability to keep things stagnant for ages and milk complexity.
But the json revolution is definitely unfinished. Standards are important and so many domains have adopted XML already. A practical way to jsonify all these specs would be really useful.
But XML is not necessarily that bad for many things.
I still use this XML editor I made in 2008: http://move.rupy.se/file/logic.html
However they seem to be against it primarily because it's not JSON (keys are explicitly unordered) and that means some generators might be unable to produce valid output, so interoperability is tricky.
[1]: https://github.com/json-schema-org/json-schema-vocabularies/...
"But you HAVE TO have a virtual DOM. You HAVE TO write all this boilerplate and ship a fat runtime. You HAVE TO manage the caching details to these performance problems and abstraction leaks (looking at you, useMemo!). You have to WRAP AND HIDE the HTML and CSS—which breaks accessibility, so you need to learn all the implementation details about a11y and manually add it back one ARIA role at a time (or just assert it's too hard, so less accommodation)."
All this when there are alternatives that are much simpler, so much easier, so much less code, and so much more consistent, folks constantly downplay it. "It CAN'T be used for large projects. You HAVE TO make this large object hierarchy to make web apps. You CAN'T rely on a compiler, because that's magic and no longer 'pure' JavaScript."
I especially love the "magic" argument since it completely glosses over the fact that we're providing info to processed sand we hit with lightning to make it think. Magic is already part of the bargain from your first "Hello World."
Only now it's not "is that a standard?" It's "everyone's doing it the unnecessarily hard way, and we've invested too much time learning all the ins and outs of the unnecessarily hard way, so if you want a job, you'd better get used to the hard way and suffer as we have."