> The original essay's point was to say that people over-weigh the pros, that there are cons, and that some of those cons have considerable real-world significance. > > It wasn't to say that there are no pros. It wasn't to provide a balanced analysis of all the pros and cons. It had a specific point, it made it, and it acknowledged that the point wasn't universal. > > This new essay misinterprets the point in order to set itself up as a more realistic viewpoint, when in fact it's arguing against a straw interpretation of the original essay.

I don't see the pros mentioned, at all.

What I see is another "Hadoop is fast, Hadoop is slow" comparison based entirely on speed which on its own isn't either a "pro" or a "con" is just a factor among many.

Without the complete picture we'll keep misinterpreting and arguing over this technology for the wrong reasons.

> Yes, fault-tolerance can be really important. And sometimes not. Such a discussion is orthogonal the point of the original. A single node is quite reliable. There are many ways to get redundancy on top of that. For the same price as a Hadoop cluster, you could buy a machine with more reliable hardware, plus a hot backup, plus a load balancer. This may be all that's needed.

Redundancy and fault-tolerance are not the same thing, you're also thinking in the context of web development and this is a different scenario.

In web you can use redundancy to improve reliability because you're either serving the same content to many clients or performing independent computations to each of them; data processing frameworks are used to perform a single computation with several different inputs over many machines, replicating it may give you some safety regarding hardware failures but none for software issues and you'll be also wasting a lot of power thus limiting the range of applications.

Big Data frameworks partition the input, they do not replicate it; they also keep track of the results and allow you to retry failed chunks, it's a different approach than those used on web.

> If you need 100 machines then you are still in the realm where you "really need distributed processing", which was one of the times where the original essay said that Hadoop could be the right solution. > > If you have 100 machines, and the single node can go 200x faster, then you really only need 1 machine.

Like I said the speed is just one factor, you can't determine the right solution based exclusively on it and that's what the original article missed, you're disregarding all the other advantages that a cluster and a Big Data framework provide.

Also the very concept of "machine" can vary wildly, there's an infinite range of combinations of disk, ram and processor so what is "a machine" and how can you say when you need one, ten or 100?

The real choice here is a single "powerful enough" machine vs. a set of "commodity" hardware, then you can compare prices, speed, scalability and reliability of each scenario in all the possible situations you expect to encounter and hopefully some unforeseen ones as well.