With the rapid evolution of connected Internet of Things devices, achieving real-time operability and interoperability has been challenging. Collecting and integrating different asset types of data from billions of connected devices and processing them at different paces are part of these challenges.

Combining MQTT and Apache Kafka is an excellent start to tackle most of the issues in these scenarios, but maintaining interoperability between them takes work. An everyday use case from Strimzi users is to use MQTT-based devices and a Kafka cluster as the event and storage platform running on Kubernetes through Strimzi to produce messages. That’s why we built the MQTT Bridge.

This talk introduces a new cloud-native MQTT to Apache Kafka Bridge, enabling MQTT-based devices to produce messages to Kafka clusters flexibly and coherently. I will start by talking about both MQTT and Apache Kafka, how they are used in an ideal IoT/IIoT scenario, and how each alone is insufficient to build a scalable, fault-tolerant and resilient IoT/IIoT Data pipeline. In between, I will be covering all the most crucial aspects of the Bridge and its working. Finally, I finish the talk with an exciting demo where I will show the working of a real-time IoT/IIoT data pipeline for an anomaly detection ML model training using the Strimzi MQTT Bridge.