Nice, a benchmark that tries to measure something else than JS speed (because it's not really JS that is the actual bottleneck in webpages people actually use). There's also Dromaeo but for some reason it has never gained enough mindshare.
It is not a well balanced benchmark so it is hard to know which part are statistically relevant. Doubling the speed of Dromaeo would not double the speed of normal Web Apps.
It is also very easy to cheat on some of Dromaeo's tests. Cheating is rampant in certain engines, modern benchmarks need to perform real workload to avoid that.
This is a good benchmark for stressing rendering, layout, and style. We should be testing the real frameworks being used on the web and Speedometer is as good as any I have seen. Unlike IE's testdrive, Speedometer's source does not look like it was written by cherry picking examples that perform well in a specific browser.
I'm skeptical of a minor point though: synthetic events do not accurately simulate end-user responsiveness, especially when it comes to frame and event scheduling that occurs in a different process. There's a difference between simulated and real events even on the renderer side (see: http://trac.webkit.org/changeset/164871).
Fair point. Synthetic events are not a fully accurate model of real user events. But they are as close as you can get in a benchmark that runs purely in the browser (as opposed to needing an external OS-specific harness). Making it easy to run everywhere has a lot of value.
> Note: Speedometer is not meant to compare the performance of different JavaScript frameworks. The mechanism we use to simulate user actions is different for each framework, and we’re forcing frameworks to do more work synchronously than needed in some cases to ensure run time can be measured.
That's too bad. I was mostly curious how the different framework implementations + tests performed.
I noticed that the React implementation was by far the slowest. The jQuery implementation was the second slowest.
6 comments
[ 5.2 ms ] story [ 31.8 ms ] threadIt is not a well balanced benchmark so it is hard to know which part are statistically relevant. Doubling the speed of Dromaeo would not double the speed of normal Web Apps.
It is also very easy to cheat on some of Dromaeo's tests. Cheating is rampant in certain engines, modern benchmarks need to perform real workload to avoid that.
Some tests of Dromaeo are great thought.
I'm skeptical of a minor point though: synthetic events do not accurately simulate end-user responsiveness, especially when it comes to frame and event scheduling that occurs in a different process. There's a difference between simulated and real events even on the renderer side (see: http://trac.webkit.org/changeset/164871).
That's too bad. I was mostly curious how the different framework implementations + tests performed.
I noticed that the React implementation was by far the slowest. The jQuery implementation was the second slowest.