One critical aspect of operating Kafka clusters is load balancing ensuring data and traffic are evenly distributed amongst the brokers. Uneven load leads to performance issues, such as increased latencies, or reduced throughput. It also makes routine maintenance, like a simple rolling restart, complicated because overloaded brokers are unable to take over when other brokers restart. In short, it’s an operational nightmare you want to avoid in your production clusters!

Luckily for us, there is Cruise Control, an open source tool designed to dynamically rebalance workloads. This tool is really powerful and offers a lot of options, but it can be a bit hard to get started with it.

In this session, we’ll do a quick overview of Cruise Control to present its main concepts and features. We’ll then demonstrate how to use it alongside Strimzi, an open source operator for Kafka, to handle common operations like scale up or down, and how to keep your clusters balanced and healthy. You’ll leave this session with a good understanding of how to use Cruise Control with any Kafka cluster, as well as a convenient way to get going with operating Kafka on Kubernetes.