Stateless Kafka-compatible broker: shifting durability to the storage layer
Tansu proposes rebuilding the Kafka model: removing state from the brokers and delegating reliability to external storage. This changes the system’s behavior under load and simplifies the operational model. The problem manifests at the operational level. A classic Kafka broker is a stateful component: replication, leader elections, persistent state, long uptime. Such nodes are hard … Read more