Message queues are essential for real-time use cases like payment processing, fraud detection, and AI-powered support systems—but traditional queues often lack scalability, durability, and replayability. In this talk, we explore how Kafka 4.0 brings native queue semantics to the world of distributed streaming, enabling fair, concurrent, and isolated message processing at scale.

We’ll show how Apache Flink’s LLM integration (using Opensearch) leverages this queue model to perform real-time Large Language Model (LLM) inference—like sentiment analysis or summarization—and how enriched results can be written directly to Apache Iceberg, a powerful data lakehouse for long-term analytics, data versioning & time travel.

Through a demo and architecture walkthrough, you’ll learn how to build intelligent, scalable pipelines that combine Kafka queues, Flink, LLMs, and Iceberg into a unified real-time analytics stack.