Apache Kafka is more than just the de facto event streaming platform. It's becoming the universal platform for data in motion. But there’s one aspect which we wanted to make easier.

Perhaps you have struggled with how best to consume data from Kafka when your application just wants a bunch of messages rather than a stream of events. If you're using consumer groups today, you'll know that you need to worry about balancing the number of partitions and consumers. And what’s the best way to cope with individual bad records? These are the reasons which motivated KIP-932: Queues for Kafka.

KIP-932 introduces the concept of share groups. Share groups let your applications consume data from regular Kafka topics with fine-grained sharing and per-message acknowledgement, just like a traditional queue. Now you can bring your queuing workloads to Apache Kafka without doing all the hard work yourself.

KIP-932 is in preview in Apache Kafka 4.1 but it's fast progressing towards production-readiness. Come and learn how to build applications using share groups, how to monitor share groups using the broker and client metrics, and how to tune them so you're ready to take your first workloads into production.