I wonder if they had chosen to compile to the standard DOM implementation if there would still be a huge performance gain over the current execution speed?
DOM operations, in JavaScript, on Firefox already occur at near assembly speed at billions of operations per second. Comparatively abstractions over the native DOM implementation, in JavaScript, are many orders of magnitude slower. While I understand this is a C++ application compiled to WASM it still begs the question if this application is potentially sacrificing performance for a more popular implementation such as the nonstandard JSX vDOM.
while clearly, effective instructions per cycle depend on the workload. That said, https://en.wikipedia.org/wiki/Instructions_per_second mentions AMD Ryzen 7 1800X reaching "304,510 MIPS at 3.6 GHz", which would give you a budget of 152 instructions in order to achieve 2 billions of operations per second (I assumed you took the plural to mean "integer plural", i.e. at least 2 billions :-) )
Now, I agree that it's a bit of a stretch to assume DOM operations won't hit some memory fetch bottleneck or something else, but parent's estimate is not miles off.
DOM operations in this context is single threaded and definitely not deeply pipelined cached friendly code. I benchmark web code using cycles and instructions and I'm finding in practice that we're averaging in the order of 0.5-2 IPC in chrome for the web render process's main thread when running real world code. 152 IPC would be amazing!
My understanding is that's CSS reflows what makes DOM operations expensive (plus maybe re-building redundant/cached DOM structures such as document.links and informing event listeners), rather than trivial tree insertions. You could also batch multiple DOM updates by just constructing an off-screen DOM element hierarchy and then finally attach it to the live DOM, replacing anything below a given node in a single atomic operation. AFAICS, DOM diffing really only adds value if you have DOM updates from many sources/components to merge for very complex React-like apps. Happy to be corrected by actual performance figures/experts.
My understanding is that's CSS reflows what makes DOM operations expensive
This is why CSS contain property is such a big performance revolution.
There is also if I remember well, a special kind of Dom that is only for logic, therefore being unaffected to reflows.
I don't remember how it's called.
Only a simple memory allocation is an order of magnitude slower. And I am not even speaking about the cost of boundary crossing between the JS engine and the C++ DOM/render API.
On my really old desktop computer I am running about 800 million operations per second in Firefox 74 on Windows 10. See what numbers you can get up to with modern hardware or a different operating system.
I'm getting about 2.5 bn op/s, which suggests this is rather down to the JIT's ability to elide microbenchmarking code.
Edit: Worth pointing out that the linked page runs on an eight year old version of the "benchmarkjs" package, which is described as a robust benchmarking package, but while the linked blog posts discusses some issues with JS benchmarking, how to trick the JIT into actually still running your NOPs is not discussed.
You make a couple of assumptions and I am not sure they are correct.
1. JIT's ability to elide microbenchmarking code
This can be easily debunked using a combination of DOM instructions and taking note of how dramatically the ops/s count drops and then comparing that difference in ops/s against the same difference from another browser like Chrome. If both browsers report wildly different base numbers but similar ratios then there is no cheating for micro-benchmarking numbers.
2. Worth pointing out that the linked page runs on an eight year old version of the "benchmarkjs" package
I found a different site that reports similar numbers so either they are using the same old code or the numbers valid without regard for the old code. https://jsbench.me/
> If both browsers report wildly different base numbers but similar ratios then there is no cheating for micro-benchmarking numbers.
I don't have a lot of experience with JS benchmarking, but for the JVM microbenchmarking is genuinely difficult and relatively worthless because the JIT is really good at killing off huge swaths of code if the results aren't used and side effects aren't observed. And even if you do, it's not unusual for it to optimize your code for the microbenchmark in a way that's not directly applicable to production use.
Unless the benchmarking code around this does some transformations on the source, I would expect a JIT (or an optimizing interpreter) to simply remove a side-effect-free statement. I would certainly expect a state-of-the-art JIT that's heavily bound to the DOM implementation to infer "document.getElementById("#foo");" is a NOP.
> This can be easily debunked using a combination of DOM instructions and taking note of how dramatically the ops/s count drops and then comparing that difference in ops/s against the same difference from another browser like Chrome.
To get back to your particular example, since getElementById in your microbench runs at approximately 100+ times the speed of querySelector (~15 mio op/s, which seems like what you'd expect) I would suspect that querySelector() is not optimized out, because it can actually throw an exception depending on the input.
Each approach is equally capable of resulting in an error state if unexpected input is supplied, for example pass an array into the document.getElementById. The largest important difference is that the input to the querySelector must be parsed as a coded instruction, a selector, while the input to getElementById is a string literal that does not get parsed.
So that's your DOM operation... You literally are just accessing the same memory over and over again and because of the super-pipelined nature of CPUs, the fetch itself ends up being pipelined.
It is a sandbox site. You can enter any different custom instructions you wish to see how it performs. If you would like something that is not cached in memory perhaps this instruction isn't:
Then perhaps there is something you aren't doing correctly. It could be your abstraction. Even on old hardware I don't notice this blocking, unless the output is millions of items, when the code is using the native DOM methods.
Here is an application I wrote last year that can output a large table to the DOM pretty quickly even with hundreds of thousands of records: https://sparser.io/demo/
Here is one I am currently working on that can output thousands of file system artifacts to the DOM without any blocking to the user interface. http://mailmarkup.org/sharefile/demo1.mp4
The UI is not developed with a phone in mind, assuming you are talking about the Sparser application. I just attempted this in my IPhone for the first time. Here are the numbers according to the tool (the numbers output to the title box in the top left corner of the screen):
I've pasted that snippet and while it did indeed render fast, adding a newline anywhere caused the whole thing to hang for 12 seconds on my phone. Try it - you'll get a similar effect.
It's easy to just render a large page from scratch. Reflowing half of it by e.g. adding some vertical space is what's really hard.
EDIT: Actual numers are lower, but still troubling:
I called you out specifically because you cited "billions of DOM operations per second." Since I work at or near the assembly level often, I found this a bit hard to believe. The query you just sent would also be cached for the same reason. You haven't mutated any state! But at least now I know the initial claim as it stands is complete bogus as suspected and I can move on with my day.
Firefox is a terrible browser to use if you want to draw conclusions from microbenchmarks; it's extremely aggressive at eliminating code. In my laptop, `document.getElementById("canvas")` runs at 1b ops/s (roughly the same ops/s as not running any code at all), which should tell you something fishy is going on. For context, in Chrome, I get 200M ops/s for the DOM calls, vs 1b ops/s for doing nothing.
Also worth noting that benchmark.js (the underlying benchmarking library in jsbench) hasn't really aged very well. It doesn't account, for example, for JIT warmup (i.e. if you try to graph time measurements of a payload over time, you should see orders of magnitude changes in numbers between the first few, first few hundreds and fully warmed up calls in a modern js engine.)
It's all about batching. Sometimes doing a tiny change can cause the entire page to re-render. If you render your templates directly to the DOM you might start a rerendering chain that needlessly wastes performance. The other benefit is that the vDOM reduces the number of changes to a minimum. That's good but it's the rendering that kills performance. Changing 1000 or 10 DOM nodes doesn't matter that much.
DOM already has a "batching" interface: construct an arbitrary complex new node off-screen, then use eg. replaceChild() on the live DOM (possibly using importNode()/adoptNode()) to attach it as the last step of your event handler.
This technique should be O(n) in every browser (where n is the size of the inserted tree), unless large parts of the tree are hidden with display:none.
After replaceChild(), the inserted tree must be analysed in context for CSS, layout and rendering. That will take O(n) unless parts can be skipped.
Do you mean that IE does a separate CSS or layout calculation for each node as they are inserted one by one?
> Sometimes doing a tiny change can cause the entire page to re-render.
What kind of change? If you are talking about repainting the visual CSS rendering to the viewport that isn't the DOM. That is your hardware. This is why modern browsers use GPU acceleration.
Changes to the DOM are changes to the DOM tree and the markup represented by that tree. These changes can have second order performance implications if the change to the tree results in adding/removing many nodes of it results in execution of events, which is outside the DOM, that result in the execution of additional DOM instructions.
> If you render your templates directly to the DOM you might start a rerendering chain that needlessly wastes performance.
I have never encountered this. Could you provide an example?
Wait... Wasn't one of React virtual DOM's original motivation "DOM operations are so expensive so we need to use virtual DOM to reduce those operations"?
One does not simply "compile to the standard DOM" to get performance gains. A virtual dom is a binding abstraction, meaning it provides a "declarative" interface and then figures out at runtime what values have changed. This costs allocating memory for a virtual dom representation of the view. If you want to compile away the virtual DOM, then the compiler has to be aware of how data changes propagate through your model. Svelte, for example does this by tracking assignments in a flat component file, but since it doesn't introspect very deeply, if you put something like redux on top of it (or even if you simply assign a deep data structure), you're going to get a relatively inefficient redraw load (this is essentially the dirty-checking problem from angular 1). Yes, a compiler could in theory be smart about it, but it then needs to be deeply aware of the runtime semantics of the language to mount the reactive model on top of it, and this is far more complex than what you can do w/ just babel-style parsing.
I'm not aware of any js compiler today that is able do the level of language introspection required to be the elusive "sufficiently smart compiler"[1].
> If you want to compile away the virtual DOM, then the compiler has to be aware of how data changes propagate through your model
That makes the assumption that a virtual DOM is needed for efficiency because the model exposed by a framework instance would be doing extra work otherwise. That is a red herring for two reasons:
1. You don't need any framework, whether MVC or otherwise, to have a fully expressive application written in JavaScript. If there is no framework and no model there is nothing superficial to be aware of or propagate through. Perhaps the vDOM provides performance improvements compared with other framework techniques. The compiler doesn't need to be smart when unnecessarily complex considerations are taken off the table.
2. There are only two classes of APIs exposed by the browser to JavaScript: The Web APIs and the DOM. Regardless of how clever or organized any framework is it still ultimately interacts with the page using the same basic set of instructions as everything else. For example a JavaScript framework isn't exposing new access points to hardware, memory, or markup artifacts that aren't already available as web standards. The performance benefit of bypassing a virtual DOM is simply due to executing fewer instructions, which means not executing a giant framework in addition to bypassing its virtual DOM.
As an analogy I don't need to blow up a dam and change the course of a river to put out a camp fire, but if I were going to put out a campfire by diverting a river I now have many additional performance factors to consider.
It sounds like you're getting caught up on the loadedness of the term "MVC".
Model means anything that is conceptually related to data. You don't need a framework to have a model. If you have a vanilla js app and call fetch and assign to a variable, that's your model. Unless you carefully write procedural code by hand to mutate specific DOM targets based on a relevant state change, the abstraction (be it a vdom or a compiler) will have to do it for you.
What view frameworks/libraries do is expose interfaces for a developer and turn code written against those interfaces into execution plans. The abstraction trend-du-jour is to have declarative style have reactive semantics (i.e. `foo = 1` triggers a change to any views that read `foo`). The challenge is if you have N state values and M DOM bindings, how do you efficiently translate changes to a subset of state values to a subset of DOM calls for only the bindings that are affected. Something has to do this determination. Either a vdom, a dirty checker, a KVO, a compiler or you.
> If there is no framework and no model
Yeah, of course you can write raw DOM calls manually. But people don't want to, citing maintainability concerns or whatever. VDOMs, dirty checkers and KVOs have already been explored and have known trade-offs. What I'm saying is that compilers leveraging today's state-of-art will necessarily compile to one of those techniques, because there's no implementation that does reactivity analysis at compile time.
> I don't need to blow up a dam and change the course of a river to put out a camp fire
To borrow your analogy, you don't need to change the course of a river to put out a camp fire, but if you want to build a system to put out fires in a city, you'll most likely use a plumbing system to divert water from your local water sources, because manually bringing buckets of water from the river will usually not cut it, and water teleportation hasn't been invented yet.
A virtual DOM is a copy of the DOM that is usually diffed and updated based on the differences.
> can we have SPA without it?
Yes. You can set divs to be visiable/hidden based on routing with vanilla js. And there's a framework called Svelte that is a SPA framework without a vDOM.
As already stated, the idea is that you have an object that represents your DOM. When your code makes changes to the DOM, it actually affects the virtual DOM, this allows for lots of quick changes to the vDOM without much overhead, then at some point the vDom is synchronised with the actual DOM. This reduces the overhead between wasm->JS as you don't have to do it as often, it also can improve the performance as DOM changes can be slow.
Yes, however it is important to know one will loose accessibility for people using assistive technologies. For background try W3C's Web Accessibility Initiative (WAI) Accessible Rich Internet Applications (WAI-ARIA) https://www.w3.org/WAI/standards-guidelines/aria/
This is a bit misleading; you're losing accessibility only if you're never rendering to the actual DOM. A virtual DOM is just a way to avoid interacting with the real DOM until the last second, it doesn't really have anything to do with your final render target.
Svette avoids a virtual DOM while still rendering to the real DOM. It keeps itself fast by compiling your code to optimize unnecessary operations beforehand. Svette can still be used for accessible web apps.
JQuery and pure Javascript just manipulate the real DOM. This can come with performance consequences, but for many apps, the performance consequences don't matter. JQuery/Javascript can still be accessible.
React/Preact/etc... all use a virtual DOM. They can be accessible as well.
The virtual DOM is not the thing that gives you accessibility. You could have a virtual DOM that rendered to canvas and it would not meet accessibility standards. Accessibility is determined by your final render target, not by how you get there.
I understand WAI for articles and "document" web pages, but does anyone who uses these technologies use them on web apps? Is this even remotely the audience for most of these SaaS SPAs?
Nice, but I think that only in rather extreme situations it would be worthwhile to write web app in C++.
Can someone knowledgeable enough about wasm tell how difficult is to call libraries written in another language compared to normal nonweb programming (say calling C library from Python)?
48 comments
[ 2.6 ms ] story [ 79.3 ms ] threadDOM operations, in JavaScript, on Firefox already occur at near assembly speed at billions of operations per second. Comparatively abstractions over the native DOM implementation, in JavaScript, are many orders of magnitude slower. While I understand this is a C++ application compiled to WASM it still begs the question if this application is potentially sacrificing performance for a more popular implementation such as the nonstandard JSX vDOM.
Citation needed. At 1 retired instruction per cycle that's 1 instruction per DOM operation to hit 2G operations / sec on a 2 Ghz clock.
Now, I agree that it's a bit of a stretch to assume DOM operations won't hit some memory fetch bottleneck or something else, but parent's estimate is not miles off.
There is also if I remember well, a special kind of Dom that is only for logic, therefore being unaffected to reflows. I don't remember how it's called.
Call me skeptical
Yep. I call that FCA/FMA numbers [1].
Only a simple memory allocation is an order of magnitude slower. And I am not even speaking about the cost of boundary crossing between the JS engine and the C++ DOM/render API.
[1]: https://www.youtube.com/watch?v=Y21xzVH8NGY
Test
against On my really old desktop computer I am running about 800 million operations per second in Firefox 74 on Windows 10. See what numbers you can get up to with modern hardware or a different operating system.Edit: Worth pointing out that the linked page runs on an eight year old version of the "benchmarkjs" package, which is described as a robust benchmarking package, but while the linked blog posts discusses some issues with JS benchmarking, how to trick the JIT into actually still running your NOPs is not discussed.
1. JIT's ability to elide microbenchmarking code
This can be easily debunked using a combination of DOM instructions and taking note of how dramatically the ops/s count drops and then comparing that difference in ops/s against the same difference from another browser like Chrome. If both browsers report wildly different base numbers but similar ratios then there is no cheating for micro-benchmarking numbers.
2. Worth pointing out that the linked page runs on an eight year old version of the "benchmarkjs" package
I found a different site that reports similar numbers so either they are using the same old code or the numbers valid without regard for the old code. https://jsbench.me/
I don't have a lot of experience with JS benchmarking, but for the JVM microbenchmarking is genuinely difficult and relatively worthless because the JIT is really good at killing off huge swaths of code if the results aren't used and side effects aren't observed. And even if you do, it's not unusual for it to optimize your code for the microbenchmark in a way that's not directly applicable to production use.
Unless the benchmarking code around this does some transformations on the source, I would expect a JIT (or an optimizing interpreter) to simply remove a side-effect-free statement. I would certainly expect a state-of-the-art JIT that's heavily bound to the DOM implementation to infer "document.getElementById("#foo");" is a NOP.
> This can be easily debunked using a combination of DOM instructions and taking note of how dramatically the ops/s count drops and then comparing that difference in ops/s against the same difference from another browser like Chrome.
To get back to your particular example, since getElementById in your microbench runs at approximately 100+ times the speed of querySelector (~15 mio op/s, which seems like what you'd expect) I would suspect that querySelector() is not optimized out, because it can actually throw an exception depending on the input.
It's the DOM modifications that can easily hang the browser for a noticeable time if the reflow they cause is complex enough.
Here is an application I wrote last year that can output a large table to the DOM pretty quickly even with hundreds of thousands of records: https://sparser.io/demo/
Here is one I am currently working on that can output thousands of file system artifacts to the DOM without any blocking to the user interface. http://mailmarkup.org/sharefile/demo1.mp4
It's not "my abstraction" - it's a general problem that many people have been trying to tackle - vDOM is just one of many approaches.
* 108ms raw execution time
* 857ms total browser visual render time
I used this code: https://raw.githubusercontent.com/prettydiff/prettydiff/mast... which generated a table of 23010 records.
The execution performance was better than I expected for a phone and I am not seeing any delay in user interaction. What were your numbers?
It's easy to just render a large page from scratch. Reflowing half of it by e.g. adding some vertical space is what's really hard.
EDIT: Actual numers are lower, but still troubling:
Parsing time: 771.7 milliseconds.
Browser time: 12078.1 milliseconds.
Also worth noting that benchmark.js (the underlying benchmarking library in jsbench) hasn't really aged very well. It doesn't account, for example, for JIT warmup (i.e. if you try to graph time measurements of a payload over time, you should see orders of magnitude changes in numbers between the first few, first few hundreds and fully warmed up calls in a modern js engine.)
1) this technique doesn't work if you need to keep part of the DOM (e.g. a focused input or a running video)
2) this technique is O(n) in IE
After replaceChild(), the inserted tree must be analysed in context for CSS, layout and rendering. That will take O(n) unless parts can be skipped.
Do you mean that IE does a separate CSS or layout calculation for each node as they are inserted one by one?
What kind of change? If you are talking about repainting the visual CSS rendering to the viewport that isn't the DOM. That is your hardware. This is why modern browsers use GPU acceleration.
Changes to the DOM are changes to the DOM tree and the markup represented by that tree. These changes can have second order performance implications if the change to the tree results in adding/removing many nodes of it results in execution of events, which is outside the DOM, that result in the execution of additional DOM instructions.
> If you render your templates directly to the DOM you might start a rerendering chain that needlessly wastes performance.
I have never encountered this. Could you provide an example?
One does not simply "compile to the standard DOM" to get performance gains. A virtual dom is a binding abstraction, meaning it provides a "declarative" interface and then figures out at runtime what values have changed. This costs allocating memory for a virtual dom representation of the view. If you want to compile away the virtual DOM, then the compiler has to be aware of how data changes propagate through your model. Svelte, for example does this by tracking assignments in a flat component file, but since it doesn't introspect very deeply, if you put something like redux on top of it (or even if you simply assign a deep data structure), you're going to get a relatively inefficient redraw load (this is essentially the dirty-checking problem from angular 1). Yes, a compiler could in theory be smart about it, but it then needs to be deeply aware of the runtime semantics of the language to mount the reactive model on top of it, and this is far more complex than what you can do w/ just babel-style parsing.
I'm not aware of any js compiler today that is able do the level of language introspection required to be the elusive "sufficiently smart compiler"[1].
[1] https://wiki.c2.com/?SufficientlySmartCompiler
That makes the assumption that a virtual DOM is needed for efficiency because the model exposed by a framework instance would be doing extra work otherwise. That is a red herring for two reasons:
1. You don't need any framework, whether MVC or otherwise, to have a fully expressive application written in JavaScript. If there is no framework and no model there is nothing superficial to be aware of or propagate through. Perhaps the vDOM provides performance improvements compared with other framework techniques. The compiler doesn't need to be smart when unnecessarily complex considerations are taken off the table.
2. There are only two classes of APIs exposed by the browser to JavaScript: The Web APIs and the DOM. Regardless of how clever or organized any framework is it still ultimately interacts with the page using the same basic set of instructions as everything else. For example a JavaScript framework isn't exposing new access points to hardware, memory, or markup artifacts that aren't already available as web standards. The performance benefit of bypassing a virtual DOM is simply due to executing fewer instructions, which means not executing a giant framework in addition to bypassing its virtual DOM.
As an analogy I don't need to blow up a dam and change the course of a river to put out a camp fire, but if I were going to put out a campfire by diverting a river I now have many additional performance factors to consider.
Model means anything that is conceptually related to data. You don't need a framework to have a model. If you have a vanilla js app and call fetch and assign to a variable, that's your model. Unless you carefully write procedural code by hand to mutate specific DOM targets based on a relevant state change, the abstraction (be it a vdom or a compiler) will have to do it for you.
What view frameworks/libraries do is expose interfaces for a developer and turn code written against those interfaces into execution plans. The abstraction trend-du-jour is to have declarative style have reactive semantics (i.e. `foo = 1` triggers a change to any views that read `foo`). The challenge is if you have N state values and M DOM bindings, how do you efficiently translate changes to a subset of state values to a subset of DOM calls for only the bindings that are affected. Something has to do this determination. Either a vdom, a dirty checker, a KVO, a compiler or you.
> If there is no framework and no model
Yeah, of course you can write raw DOM calls manually. But people don't want to, citing maintainability concerns or whatever. VDOMs, dirty checkers and KVOs have already been explored and have known trade-offs. What I'm saying is that compilers leveraging today's state-of-art will necessarily compile to one of those techniques, because there's no implementation that does reactivity analysis at compile time.
> I don't need to blow up a dam and change the course of a river to put out a camp fire
To borrow your analogy, you don't need to change the course of a river to put out a camp fire, but if you want to build a system to put out fires in a city, you'll most likely use a plumbing system to divert water from your local water sources, because manually bringing buckets of water from the river will usually not cut it, and water teleportation hasn't been invented yet.
> can we have SPA without it? Yes. You can set divs to be visiable/hidden based on routing with vanilla js. And there's a framework called Svelte that is a SPA framework without a vDOM.
Svette avoids a virtual DOM while still rendering to the real DOM. It keeps itself fast by compiling your code to optimize unnecessary operations beforehand. Svette can still be used for accessible web apps.
JQuery and pure Javascript just manipulate the real DOM. This can come with performance consequences, but for many apps, the performance consequences don't matter. JQuery/Javascript can still be accessible.
React/Preact/etc... all use a virtual DOM. They can be accessible as well.
The virtual DOM is not the thing that gives you accessibility. You could have a virtual DOM that rendered to canvas and it would not meet accessibility standards. Accessibility is determined by your final render target, not by how you get there.
The WAI looks mostly aimed at applications anyway, I suspect in part because there aren't a ton of compelling reasons to use WASM for a static page.
Can someone knowledgeable enough about wasm tell how difficult is to call libraries written in another language compared to normal nonweb programming (say calling C library from Python)?
Maybe useful for getting existing desktop applications running in the browser?