You have a point, but it's also a fact that lots of people use "Hadoop" interchangeably with "MapReduce". And Spark can in fact replace the Hadoop infrastructure entirely, as it's not a component of that ecosystem. Just…
Long winded in more ways than one. I would still use Spark even if it were slower than Hadoop, just to get a sane API.
It can also spill to disk if needed, which is what Hadoop always does.
You have a point, but it's also a fact that lots of people use "Hadoop" interchangeably with "MapReduce". And Spark can in fact replace the Hadoop infrastructure entirely, as it's not a component of that ecosystem. Just…
Long winded in more ways than one. I would still use Spark even if it were slower than Hadoop, just to get a sane API.
It can also spill to disk if needed, which is what Hadoop always does.