Goka is not a replacement for Kafka Streams, they can be used together. For example, you can use streams and tables created with Kafka Streams inside services built with Goka.
There are several differences. Here are some of them.
- Kasper is micro-batched, Goka consumes one message at a time.
- Kasper requires a centralized state storage (such as Redis or ElasticSearch), Goka only depends on Kafka: processors persist their state (key-value tables) in a compacted topic in Kafka. That is really nice because processors can then join streams with tables of other processors.
- Kasper statically assigns partitions to processors. Goka automatically splits the input and state partitions among processors.
- By default Goka employs sarama and sarama-cluster libraries to access Kafka, but that can be easily replaced with Confluent's go library. Similarly, the local storage is by default LevelDB, but can be replaced even with some remote storage if necessary.
10 comments
[ 27.7 ms ] story [ 177 ms ] threadIs the work in progress? If so, please tell us where.
http://eng.uber.com/cherami/
https://github.com/uber?utf8=%E2%9C%93&q=cherami&type=&langu...
It supports consumption of kafka topics:
https://github.com/uber/cherami-server/commit/226fb2c55b7c51...
- Kasper is micro-batched, Goka consumes one message at a time.
- Kasper requires a centralized state storage (such as Redis or ElasticSearch), Goka only depends on Kafka: processors persist their state (key-value tables) in a compacted topic in Kafka. That is really nice because processors can then join streams with tables of other processors.
- Kasper statically assigns partitions to processors. Goka automatically splits the input and state partitions among processors.
- By default Goka employs sarama and sarama-cluster libraries to access Kafka, but that can be easily replaced with Confluent's go library. Similarly, the local storage is by default LevelDB, but can be replaced even with some remote storage if necessary.