Nice! Ebay's running webwokers. Do you guys have any plans to start implementing app cache manifests to reduce load off the clusters? Even though ebay's loading time is fast as hell, I could still imagine this would reduce not only load for static content, but also decrease http sizes for end users.
"Our idea was to build a bare minimum boilerplate Node.js server that scales; we did not want to bloat the application by introducing a framework. The first four node modules we added as dependencies were express, cluster, request, and async."
I'm curious how you're using Express if not as a framework.
That doesn't really change what MatthewPhillips said. "Framework" can mean everything from Rails to startup.sh that boots up your node process. Express really doesn't do much, and is closer to middleware than a full framework.
Not to stop there, it's in the page title too, so anyone coming in from Google (for example) is basically required to click on the phrase "web application framework" and acknowledge that the creators, at least, think that it's a framework.
in a quick blurb on express v restify:
"I get asked this more than anything else, so I'll just get it out of the way up front.
Express' use case is targeted at browser applications and contains a lot of functionality, such as templating and rendering, to support that. Restify does not."
FWIW I am a backend guy and have enjoyed prototyping and working with restify a great deal ( dtrace ftw :) ), just a user and YMMV of course.
I'd be interested in actually seeing the Wiki myself. It sounds like some research went into properly addressing concerns; it'd fit the public domain well!
I really hate posts like this. It's full of vague statements like:
"When we found that Java did not seem to fit the
project requirements (no offense), we began exploring
the world of Node.js"
and
"By the end of the exercise, people understood the core
value of Node.js; indeed, some of the con arguments proved
to be part of the beauty of the language."
What's wrong with this? The pros and cons of Java versus Node.js aren't explained. In fact, I'm hard pressed to find any specific information in this article at all about what drove their decision making, or for that matter, anything in this article that's remotely technical.
This is a fluff piece that won't inform you about anything besides the fact that eBay is using Node. Cool story? But you're not telling us why
They talk about how under "Scalability" how "Java did not seem to fit the project requirements"... I'm sure a Java implementation can be less scalable than a node.js implementation, but every anecdotal piece of evidence I have experienced suggests that a good Java implementation will be faster than a good node.js implementation.
If we look to popular micro-benchmarks, they support my anecdotal observation:
On a side node: node.js also suffers from the issue that while a small node.js program might be fast, what is actually fast is the C code and as the program grows larger and increasing amounts of time are spent executing actual javascript its performance characteristics change dramatically. This fact seems often lead to unrealistic understanding of node.js performance due to people benchmarking very small examples.
This is less an issue with "very small examples" and more an issue of people throwing the practices out when they build more complex software.
Highly performant JS ends up looking a lot like C, and most of the off-the-shelf JS libraries that people use to build software are littered with '.call', '.apply', 'arguments', null/undefined/true/false usage, implicit type conversions, unnecessary closures, etc. All of these are cases that V8 doesn't optimize well, and because the libraries are built to be highly abstract and useful by using these, performance drops off precipitously. It's completely possible to write a large JS application that's competitive with Java in performance -- but nobody does it.
V8 hasn't even really begun optimizing these things, and they're going to be incredibly hard to optimize well. So perhaps a more apt question is: when JS VMs have as much time in the oven as the JVM has had, what will the performance look like?
The article was about the How more than the Why (ie. "How We Built eBay's First Node.JS Application). It would be interesting to know more about the pro/con debates, but it did a pretty good job of providing high level detail on their development stack. With new technologies people often ask 'what large production environments has this been used in?' Articles like this help to answer that. And since there are so many modules and options for node developers, articles that highlight some of the more popular and mature modules can really help narrow the field for people who are wondering where to start. By the way TechEmpower has some great comparisons of performance metrics that are probably targeted more to readers like yourself who want detailed comparisons and analysis of technologies under particular use cases.
"This being a normal scenario in any event loop-based model like Node.js, the logs are crossed between multiple URL transactions, and the reporting tool shows scrambled output. We have worked out both short-term and long-term resolutions for this issue."
which sounds like a serious problem for pretty much anybody.
I know they're not obliged to tell us their solutions, but come on, they're a big company with lots of smart engineers and billions of dollars. We already know they can solve problems, telling us they've done so is neither interesting nor useful.
I still dont get it, sounds like: 'look! ebay employees are also dabbling with node'.
What did you actually build? Is it your hackathon project 'Talk' that you build on node, or was it something else?
Given that they went with Node.js and talked about scalability, I assume they wanted the async nature of Node.js to maintain a large number of connections. I'm surprised they didn't try out Netty since they are a Java shop. The last time I investigated Node.js for async connection support in a project, the benchmarks I did showed that Netty could scale substantially better than Node.js in term of number of connections, memory usage, thread utilization, and raw performance on the same hardware. That was a year and half ago. Things might be different now, but I doubt some of fundamentals have changed much.
Anyway, it's good to try out new things. Kudos to them for doing something out of their mold.
27 comments
[ 3.0 ms ] story [ 66.8 ms ] threadI'm curious how you're using Express if not as a framework.
http://mcavage.github.io/node-restify/
Both self describe as frameworks.
in a quick blurb on express v restify: "I get asked this more than anything else, so I'll just get it out of the way up front.
Express' use case is targeted at browser applications and contains a lot of functionality, such as templating and rendering, to support that. Restify does not."
FWIW I am a backend guy and have enjoyed prototyping and working with restify a great deal ( dtrace ftw :) ), just a user and YMMV of course.
I think they may have meant that they didn't want to write their own.
This is a fluff piece that won't inform you about anything besides the fact that eBay is using Node. Cool story? But you're not telling us why
If we look to popular micro-benchmarks, they support my anecdotal observation:
http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
http://www.techempower.com/benchmarks/
On a side node: node.js also suffers from the issue that while a small node.js program might be fast, what is actually fast is the C code and as the program grows larger and increasing amounts of time are spent executing actual javascript its performance characteristics change dramatically. This fact seems often lead to unrealistic understanding of node.js performance due to people benchmarking very small examples.
Highly performant JS ends up looking a lot like C, and most of the off-the-shelf JS libraries that people use to build software are littered with '.call', '.apply', 'arguments', null/undefined/true/false usage, implicit type conversions, unnecessary closures, etc. All of these are cases that V8 doesn't optimize well, and because the libraries are built to be highly abstract and useful by using these, performance drops off precipitously. It's completely possible to write a large JS application that's competitive with Java in performance -- but nobody does it.
V8 hasn't even really begun optimizing these things, and they're going to be incredibly hard to optimize well. So perhaps a more apt question is: when JS VMs have as much time in the oven as the JVM has had, what will the performance look like?
"This being a normal scenario in any event loop-based model like Node.js, the logs are crossed between multiple URL transactions, and the reporting tool shows scrambled output. We have worked out both short-term and long-term resolutions for this issue."
which sounds like a serious problem for pretty much anybody.
I know they're not obliged to tell us their solutions, but come on, they're a big company with lots of smart engineers and billions of dollars. We already know they can solve problems, telling us they've done so is neither interesting nor useful.
Anyway, it's good to try out new things. Kudos to them for doing something out of their mold.
No CPS pyramid of doom garbage though. To get that, Node.js is your one-stop-shop.