Parent is talking big-O and so is ignoring that it isn't exactly k^2. Depending on how you count (do 2 nodes require 2 connections or 1 bi-directional connection) it's k(k-1).
The paper contains the motivation: "For our needs at Twitter, we needed a stream processing platform that was open-source, high-performance, scalable, and was compatible with the current Storm API."
Now I get it. It's not just "I didn't write this, therefore shit, so I need to rewrite it from scratch." This was a clear explanation in a bite sized chunk which illustrated the reasons for Heron. Thanks.
SIGMOD 2015 proceedings are open access, meaning the conference has arranged with ACM to provide the content to pretty much everyone.
That being said, the submission is to someone's (Adrian's) interpretation of the work, and is arguably more useful than just linking at the pdf itself.
14 comments
[ 2.8 ms ] story [ 44.1 ms ] threadshouldn't that be !(k-1)? Or am I missing something about the architecture?
If you have k=2 stream managers you have 1 connection, not 4 . Even if the connections weren't bidirectional you'd have 2 connections.
In every metric except perhaps the most important one for actual users:
Storm is open source, Heron isn't.
Also, the @TwitterOSS account indicated that there are "no plans to open source in the short-term":
https://twitter.com/TwitterOSS/status/605460461761396736
I guess 3/4 isn't bad.
That being said, the submission is to someone's (Adrian's) interpretation of the work, and is arguably more useful than just linking at the pdf itself.