Would love to see a non JVM kafka alternative. But, I wish people would stop building infrastructure with C/C++ when there are safer alternatives like rust and go.
I'm not to defend Heka, as I don't use it, but those "design flaws" only
concerned development, not operations. If you read the message you quote, you
would see that Mozilla is still using it.
And I take it you have measured Fluentd's performance, so you say so from
experience, not flail around the typical opinion about language that can
safely be attached to Java as well?
This is a message broker. You can publish messages to it, and it will retain them for as long as you want(it supports retention and compaction semantics), and you can consume the messages, starting from any message(messages are identified by sequence numbers). This is similar to Apache Kafka and - to a lesser extent - Azure Events Hub )
Wish the post talked real numbers, as "very high throughput and very low latency" means very different things to different people. For example, a 1 millisecond latency is "very low latency" for web related things, but might be a lifetime when comparing it to a top of the line trading application.
You are right, I should have elaborated on those claims, and I plan to add a relevant Wiki page to address this shortcoming.
However, you can use the tank-cli bm command to benchmark client to broker to client (end to end) latency.
We get sub-ms (a few hundred microseconds) latency here, and even on a very busy Tank broker (it takes effort to get there), I don't foresee situations where you can get over 1ms latency for any operation.
For throughput, Tank relies on sendfile() and readahead() for very high throughput - and thanks to the very tight packing of the messages, you will likely saturate your disk/network easily.
13 comments
[ 4.0 ms ] story [ 82.1 ms ] threadAnd I take it you have measured Fluentd's performance, so you say so from experience, not flail around the typical opinion about language that can safely be attached to Java as well?
We get sub-ms (a few hundred microseconds) latency here, and even on a very busy Tank broker (it takes effort to get there), I don't foresee situations where you can get over 1ms latency for any operation.
For throughput, Tank relies on sendfile() and readahead() for very high throughput - and thanks to the very tight packing of the messages, you will likely saturate your disk/network easily.