JSON encoding is a huge impediment to interprocess communication in NodeJS.
Sooner or later is seems like everyone gets the idea of reducing event loop stalls in their NodeJS code by trying to offload it to another thread, only to discover they’ve tripled the CPU load in the main thread.
I’ve seen people stringify arrays one entry at a time. Sounds like maybe they are doing that internally now.
If anything I would encourage the V8 team to go farther with this. Can you avoid bailing out for subsets of data? What about the CString issue? Does this bring faststr back from the dead?
Based off of my first ever forays into node performance analysis last year, JSON.stringify was one of the biggest impediments to just about everything around performant node services. The fact that everyone uses stringify to for dict keys, the fact that apollo/express just serializes the entire response into a string instead of incrementally streaming it back (I think there are some possible workarounds for this, but they seemed very hacky)
As someone who has come from a JVM/go background, I was kinda shocked how amateur hour it felt tbh.
Yeah. I think I've only ever found one situation where offloading work to a worker saved more time than was lost through serializing/deserializing. Doing heavy work often means working with a huge set of data- which means the cost of passing that data via messages scales with the benefits of parallelizing the work.
> JSON encoding is a huge impediment to communication
I wonder how much computational overhead JSON'ing adds to communications at a global scale, in contrast to just sending the bytes directly in a fixed format or something far more efficient to parse like ASN.1.
> If anything I would encourage the V8 team to go farther with this.
That feels the wrong way to go. I would encourage the people that have this problem to look elsewhere. Node/V8 isn't well suited to backend or the heavier computational problems. Javascript is shaped by web usage, and it will stay like that for some time. You can't expect the V8 team to bail them out.
The Typescript team switched to Go, because it's similar enough to TS/JS to do part of the translation automatically. I'm no AI enthousiast, but they are quite good at doing idiomatic translations too.
When it comes time to do serious work in Node, that's when you start using TypedArrays and SharedArrayBuffers and work with straight binary data. stringifying is mostly for toy apps and smaller projects.
If I took all the coworkers I’ve had in 30 years of coding whom I could really trust with the sort of bit fiddling array buffers, I could populate maybe two companies and everyone else would be fucked.
TypedArray is a toy. Very few domains actually work well with this sort of data. Statistics, yes. Games? Do all the time, but also games: are full of glitches used by speed runners, due to games playing fast and loose to maintain an illusion they are doing much more per second than they should be able to.
DataView is a bit better. I am endlessly amazed at how many times I managed to read people talk about TypedArrays and SharedByteArrays before I discovered that DataView exists and has existed basically forever. Somebody should have mentioned it a lot, lot sooner.
> Sooner or later is seems like everyone gets the idea of reducing event loop stalls in their NodeJS code by trying to offload it to another thread, only to discover they’ve tripled the CPU load in the main thread.
Why not use structuredClone to communicate with the worker? So long as your object upholds all the rules you can pass it into postMessage directly.
It's a major problem for JVM performance as well. Json encoding is simply fundamentally an expensive thing to do.
One thing that improves performance for the JVM that'd be nice to see in node realm is that JSON serialization libraries can stream out the serialization. One of the major costs of JSON is the memory footprint. Strings take up a LOT more space in memory than a regular object does.
Since the JVM typically only uses JSON as a communication protocol, streaming it out makes a lot of sense. The IO (usually) takes long enough to give a CPU reprieve while simultaneously saving memory.
I really like seeing the segmented buffer approach. It's basically the rope data structure trick I used to hand-roll in userland with libraries like fast-json-stringify, now native and way cleaner. Have you run into the bailout conditions much? Any replacer, space, or custom .toJSON() kicks you back to the slow path?
The part that was most surprising to me was how much the performance of serializing floating-point numbers has improved, even just in the past decade [1].
Did you run any tests/regressions against the security problems that are common with parsers? Seems like the solution might be at risk of creating CVEs later
I confess that I'm at a bit of a loss to know what sort of side effects would be common when serializing something? Is there an obvious class of reasons for this that I'm just accidentally ignoring right off?
The SWAR escaping algorithm [1] is very similar to the one I implemented in Folly JSON a few years ago [2]. The latter works on 8 byte words instead of 4 bytes, and it also returns the position of the first byte that needs escaping, so that the fast path does not add noticeable overhead on escape-heavy strings.
If you want to optimize it a little more, you can combine isLess(s, 0x20) and isChar('"') into isLess(s ^ (kOnes * 0x02), 0x21) (this works since '"' is 0x22).
An important question that was not addressed is whether the general path will be slower to account for what is needed to check first if the fast path can be used.
V8 is extremely good, but (maybe due to JS itself?) it still falls short of LuaJIT and even JVM performance. Although at least for JVM it takes way longer to warm up than the other two.
JSON serialization is already quite fast IMO so this is quite good. I think last time I compared JSON serialization to Protocol Buffers, JSON was just a little bit slower for typical scenarios but not materially so. These kinds of optimizations can shift the balance in terms of performance.
JSON is a great minimalist format which is both human and machine readable. I never quite understood the popularity of ProtoBuf; the binary format is a major sacrifice to readability. I get that some people appreciate the type validation but it adds a lot of complexity and friction to the transport protocol layer.
I will need to check this vs the usual safe-stable-stringify package I usually use in my projects now when I'm back on the computer, but it is always nice to see the speedup.
> No indexed properties on objects: The fast path is optimized for objects with regular, string-based keys. If an object contains array-like indexed properties (e.g., '0', '1', ...), it will be handled by the slower, more general serializer.
Slightly related: I die a little inside each time I see `JSON.parse(JSON.stringify(object))` thinking about the inefficiencies involved compared to how you'd do this in a more efficient language.
Unrelated but I have the easiest design improvement for your website: change <html> background color to #4285f4 (same as in topbar) so it looks like a sheet of paper on a blue base
How is this compared to other ecosystems? I've been serializing JSON for about a decade and it's been so fast I haven't really thought about it. Simdjson can do gigabytes per second per core.
Once you factor in prefetching, branch prediction, etc., a highly optimized JSON serializer should be effectively free for most real world workloads.
The part where json sucks is IO overhead when modifying blobs. It doesn't matter how fast your serializer is if you have to push 100 megabytes to block storage every time a user changes a boolean preference.
Not that I doubt the value of the work, and the reasoning of its performance directly affecting common operations makes intuitive sense, but I would have liked to hear more about what concrete problems were being solved. Was there any interesting data across the V8 ecosystem about `JSON.stringify` dominating runtimes?
> No replacer or space arguments: Providing a replacer function or a space/gap argument for pretty-printing are features handled exclusively by the general-purpose path. The fast path is designed for compact, non-transformed serialization.
Do we get this even if you call `JSON.stringify(data, null, 0)`? Or do the arguments literally have to be undefined?
38 comments
[ 3.2 ms ] story [ 47.0 ms ] threadSooner or later is seems like everyone gets the idea of reducing event loop stalls in their NodeJS code by trying to offload it to another thread, only to discover they’ve tripled the CPU load in the main thread.
I’ve seen people stringify arrays one entry at a time. Sounds like maybe they are doing that internally now.
If anything I would encourage the V8 team to go farther with this. Can you avoid bailing out for subsets of data? What about the CString issue? Does this bring faststr back from the dead?
As someone who has come from a JVM/go background, I was kinda shocked how amateur hour it felt tbh.
Or I guess you can do it without the WebAssembly step.
> JSON encoding is a huge impediment to communication
I wonder how much computational overhead JSON'ing adds to communications at a global scale, in contrast to just sending the bytes directly in a fixed format or something far more efficient to parse like ASN.1.
That feels the wrong way to go. I would encourage the people that have this problem to look elsewhere. Node/V8 isn't well suited to backend or the heavier computational problems. Javascript is shaped by web usage, and it will stay like that for some time. You can't expect the V8 team to bail them out.
The Typescript team switched to Go, because it's similar enough to TS/JS to do part of the translation automatically. I'm no AI enthousiast, but they are quite good at doing idiomatic translations too.
TypedArray is a toy. Very few domains actually work well with this sort of data. Statistics, yes. Games? Do all the time, but also games: are full of glitches used by speed runners, due to games playing fast and loose to maintain an illusion they are doing much more per second than they should be able to.
DataView is a bit better. I am endlessly amazed at how many times I managed to read people talk about TypedArrays and SharedByteArrays before I discovered that DataView exists and has existed basically forever. Somebody should have mentioned it a lot, lot sooner.
Why not use structuredClone to communicate with the worker? So long as your object upholds all the rules you can pass it into postMessage directly.
One thing that improves performance for the JVM that'd be nice to see in node realm is that JSON serialization libraries can stream out the serialization. One of the major costs of JSON is the memory footprint. Strings take up a LOT more space in memory than a regular object does.
Since the JVM typically only uses JSON as a communication protocol, streaming it out makes a lot of sense. The IO (usually) takes long enough to give a CPU reprieve while simultaneously saving memory.
[1] https://github.com/jk-jeon/dragonbox?tab=readme-ov-file#perf...
[1] https://source.chromium.org/chromium/_/chromium/v8/v8/+/5cbc...
[2] https://github.com/facebook/folly/commit/2f0cabfb48b8a8df84f...
So array list instead of array?
JSON is a great minimalist format which is both human and machine readable. I never quite understood the popularity of ProtoBuf; the binary format is a major sacrifice to readability. I get that some people appreciate the type validation but it adds a lot of complexity and friction to the transport protocol layer.
Any idea why?
There's structuredClone (https://developer.mozilla.org/en-US/docs/Web/API/Window/stru... https://caniuse.com/?search=structuredClone) with baseline support (93% of users), but it doesn't work if fields contain DOM objects or functions meaning you might have to iterate over and preprocess objects before cloning so more error-prone, manual and again inefficient?
Once you factor in prefetching, branch prediction, etc., a highly optimized JSON serializer should be effectively free for most real world workloads.
The part where json sucks is IO overhead when modifying blobs. It doesn't matter how fast your serializer is if you have to push 100 megabytes to block storage every time a user changes a boolean preference.
Do we get this even if you call `JSON.stringify(data, null, 0)`? Or do the arguments literally have to be undefined?