When Kafka deals with large volumes of data, scaling out or in requires large data replication of the partitions, which is a very heavy operation. With the architecture based on S3 using shared storage, everything becomes much simpler. In my view, the trade-off is that we additionally use EBS as a persistent write cache to work with S3, which to some extent eliminates the latency issue of accessing S3.