A better description is Apache Kafka utilises zero copy if you do not use SSL/TLS. Almost every Kafka cluster I’ve set up, SSL/TLS has been an absolute requirement to tick the uses SSL checkbox for security sing-off.
It’s been a good while since I used Kafka but I wouldn’t describe it as generally performant. Again it’s one of those, in specific use cases/setups it can be performant. The throughput/latency decreased as the sum of partitions grew. As the level of parallelism is on the partition you can notice this quickly with a fairly low number of topics. I know they where working on partition scaling so not sure how that looks today.
Interestingly Confluent releases a blog post last week saying their cloud product is now no longer Kafka, it’s something entirely different speaking the Kafka protocol. I suspect this is because of partition scaling/multi tenant clusters amongst other things.
6 comments
[ 3.0 ms ] story [ 28.9 ms ] threadA better description is Apache Kafka utilises zero copy if you do not use SSL/TLS. Almost every Kafka cluster I’ve set up, SSL/TLS has been an absolute requirement to tick the uses SSL checkbox for security sing-off.
It’s been a good while since I used Kafka but I wouldn’t describe it as generally performant. Again it’s one of those, in specific use cases/setups it can be performant. The throughput/latency decreased as the sum of partitions grew. As the level of parallelism is on the partition you can notice this quickly with a fairly low number of topics. I know they where working on partition scaling so not sure how that looks today.
Interestingly Confluent releases a blog post last week saying their cloud product is now no longer Kafka, it’s something entirely different speaking the Kafka protocol. I suspect this is because of partition scaling/multi tenant clusters amongst other things.
Nowadays, Smart NICs can handle encryption and IOMMU provides (some) security isolation.
The sendfile() syscall uses these if available. (https://people.kernel.org/kuba/tls-1-3-rx-improvements-in-li...)
I never used Kafka. I'd expect that using Cloud HSM for TLS/SSL offloads in any cloud should get you zero copy with encryption.