This vaguely reminds me of John Resig's TestSwarm project (http://ejohn.org/blog/test-swarm-alpha-open/). Although I'm not sure what the current status of the project is since the links to the site and github project are now dead:
The slickest way I can imagine to do it would be be using HTML5 Webworkers on the clients and Websockets on NodeJS for the server. I've been thinking of doing some experiments in this direction, I'd be surprised if at least a few folks aren't already doing this.
Are there any JITs for JS that end up exposing SIMD instructions on the host processor? Without that, and combined with the fact that JITs still don't necessarily get to C speed even on simple math on raw floats/integers, you're going to take a stiff performance penalty for this, even on the latest JS VMs. User will still be paying for their electricity but will be doing a vanishing fraction of the work... rather than being a great idea it would actually sort of hostile to even offer this option.
NaCl, maybe. Google isn't helping me figure out if it can do SIMD instructions.
I'd really rather use NaCl. And, I think SIMD is in the plan for NaCl, eventually...
But, you might be a bit surprised by the raw float performance of the latest JS VMs. Running this benchmark, http://stepheneb.github.com/webgl-matrix-benchmarks/matrix_b... on my 3.4Ghz AMD in FF4, CanvasMatrix can multiply 20,000 pairs of 4x4 matrices in 1.5 ms. That's about 1.5 double-precision gigaflops on 1 core. 4 webworkers would give 6 gigaflops. That's low compared to the 50 gigaflops that native, pure-cpu-burn benchmarks get on that processor (when overclocked to 3.6 Ghz) http://www.overclock.net/amd-cpus/499526-official-phenom-ii-... but it's not terrible.
This is what wowd (http://www.wowd.com/ [no longer relevant]) tried to do for realtime search before they had to pivot. Current browsers with ECMAScript as the only API are simply too insular, too limited and too slow for any useful distributed computing to ever be genuinely feasible.
It would be nice if the browser was a stronger vm specifically suited for executing untrusted objects, yet still with a more traditional os api, filesystem access, and real network interface. But the web still sucks for now.
I've been lurking hn for a while now but finally decided to register for this post.
I actually did something similar to this for my undergrad honours thesis last year. The main difference being that my client was written in actionscript instead of javascript.
The difficulty with this type of application is breaking up the computation into chunks large enough to offset the overhead of getting the data to and from the client to net a "profit" in terms of computation time. As the author mentioned, you have to limit the amount of CPU time that the client uses, otherwise, you'll end up potentially impacting the user negatively. Couple this with the limited amount of time between a user's browser requests and the amount of computational power at your disposal is pretty limited.
Although my implementation was far from optimal (it was purely academic in order to see if the concept would work), I found that in every case, it was on the order of at least 100 times slower than simply doing the computation serially(!) on the server machine. It was actually costing more time to manage the connections between the various clients and the server than the amount of computational work that I was getting from the clients.
Now, it's not to say that something like this couldn't potentially be made to work. But the types of computations that this sort of grid is also severely limited. Due to things like sandboxing and socket restrictions you cannot (easily) have clients collaborate together to perform more complex computations, therefore you are effectively limited to embarrassingly parallel algorithms.
8 comments
[ 4.1 ms ] story [ 27.8 ms ] threadhttp://testswarm.com/
https://github.com/jeresig/testswarm
This seems like it would be an ideal starting point for a project like this.
NaCl, maybe. Google isn't helping me figure out if it can do SIMD instructions.
But, you might be a bit surprised by the raw float performance of the latest JS VMs. Running this benchmark, http://stepheneb.github.com/webgl-matrix-benchmarks/matrix_b... on my 3.4Ghz AMD in FF4, CanvasMatrix can multiply 20,000 pairs of 4x4 matrices in 1.5 ms. That's about 1.5 double-precision gigaflops on 1 core. 4 webworkers would give 6 gigaflops. That's low compared to the 50 gigaflops that native, pure-cpu-burn benchmarks get on that processor (when overclocked to 3.6 Ghz) http://www.overclock.net/amd-cpus/499526-official-phenom-ii-... but it's not terrible.
"You've heard of SETI@home? We're SETI@Chrome." http://2010.nodeknockout.com/teams/anansi http://news.ycombinator.com/item?id=1645520
It would be nice if the browser was a stronger vm specifically suited for executing untrusted objects, yet still with a more traditional os api, filesystem access, and real network interface. But the web still sucks for now.
The difficulty with this type of application is breaking up the computation into chunks large enough to offset the overhead of getting the data to and from the client to net a "profit" in terms of computation time. As the author mentioned, you have to limit the amount of CPU time that the client uses, otherwise, you'll end up potentially impacting the user negatively. Couple this with the limited amount of time between a user's browser requests and the amount of computational power at your disposal is pretty limited.
Although my implementation was far from optimal (it was purely academic in order to see if the concept would work), I found that in every case, it was on the order of at least 100 times slower than simply doing the computation serially(!) on the server machine. It was actually costing more time to manage the connections between the various clients and the server than the amount of computational work that I was getting from the clients.
Now, it's not to say that something like this couldn't potentially be made to work. But the types of computations that this sort of grid is also severely limited. Due to things like sandboxing and socket restrictions you cannot (easily) have clients collaborate together to perform more complex computations, therefore you are effectively limited to embarrassingly parallel algorithms.