KSQL feels like the enterprise equivalent of tesla autopilot. Confluent is trying to generate a ton of hype behind it at their keynotes, yet I've yet to see it work successfully in production. Apache Flink and Spark Streaming seem to be miles ahead.
In fact, I try my best to not use Kafka at all. Most of the applications don't have that kind of load and adopting Kafka leads to heavyweight architecture.
In my experience, it is easy to emulate Kafka topic behaviour on modern SQL databases (for example LISTEN/UNLISTEN/NOTIFY in PostgreSQL), rather than doing whole gymanstic of KSQL layer on Kafka beast and still fall short of SQL2016 spec by a large margin. I always look for databases which application uses as a primary DB with support for Streams/CDC (Change Data Capture) either natively OR it is easy to implement thin layer over the DB within 2-3 days.
2 comments
[ 4.4 ms ] story [ 18.0 ms ] threadIn fact, I try my best to not use Kafka at all. Most of the applications don't have that kind of load and adopting Kafka leads to heavyweight architecture.
In my experience, it is easy to emulate Kafka topic behaviour on modern SQL databases (for example LISTEN/UNLISTEN/NOTIFY in PostgreSQL), rather than doing whole gymanstic of KSQL layer on Kafka beast and still fall short of SQL2016 spec by a large margin. I always look for databases which application uses as a primary DB with support for Streams/CDC (Change Data Capture) either natively OR it is easy to implement thin layer over the DB within 2-3 days.