Session
Kafka Flow: Taming Distributed Data with Kotlin Coroutines
Let's be real for a minute. People usually just ignore the Future returned from the standard Apache Kafka producer. Plus, if your microservices are pumping out events super fast, trying to handle all that thread management inside your Consumer.poll() loops can get messy really fast. While the classic Java Kafka client is solid, it often doesn't play nicely with modern, non-blocking Kotlin setups.
Good news, though! Kotlin Flows and Coroutines are here to be your event-driven superheroes. They transform that messy, callback-ridden setup into beautiful, thread-efficient data streams—which is one place where Kotlin shines. This whole declarative style—handling streams, backpressure, and lifecycle like a total pro—is what we're calling Kafka Flow.
We're going to build the ultimate Flow-based Kafka wrapper and reveal the essential "bridge" patterns. We'll show you the magic of using suspendCoroutine to turn those producer acknowledgments into a clean suspend function, and how to safely tuck the slow-poke consumer polling inside a smooth, declarative Flow. The real kicker is how Flow operators let you tackle the seriously hard stuff: fine-tuning backpressure, building rock-solid distributed retries, and managing the entire consumer lifecycle—all with elegant, functional code that actually looks awesome.
Swing by and see how "Kafka Flow" can finally get you out of the thread management business and help your event-driven services scale up effortlessly. Trust me, future you will be sending thank you notes.
Sandon Jacobs
Confluent, an IBM Company - Senior Developer Advocate
Raleigh, North Carolina, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top