We are excited to announce that 2.8 introduces an early-access look at Kafka without ZooKeeper! The implementation is not yet feature complete and should not be used in production, but it is possible to start new clusters without ZooKeeper and go through basic produce and consume use cases.
At a high level, KIP-500 works by moving topic metadata and configurations out of ZooKeeper and into a new internal topic named
@metadata. This topic is managed by an internal Raft quorum of “controllers” and is replicated to all brokers in the cluster. The leader of the Raft quorum serves the same role as the controller in clusters today. A node in the KIP-500 world can serve as a controller, a broker, or both, depending on the new
process.rolesconfiguration. See the README for quickstart instructions and additional details.
In addition to the headline item, there are plenty of other bugfixes and additions as well.