Redis is at the heart of modern data architectures. Thanks to Redis Modules, we have a multi-model database with capabilities such as Graph, JSON, SQL, etc. Once such module is RedisTimeSeries, that brings first-class support for time series data in Redis (no need to use Sorted Sets anymore!). This talk is all about how to combine the power of RedisTimeSeries and Apache Kafka to build scalable solutions that handle time series data.

Time Series databases are used to solve a variety of real-world problems such as monitoring fleet of apps/infrastructure, analytics for thousands of IoT device data, and much more. Apache Kafka is an open-source event streaming platform with large-scale production deployments across a wide range of organisations and industries. One of its key benefit is the rich ecosystem of projects consisting of client APIs, connectors and stream processing capabilities.

Given the value of Kafka as a platform, it's worth exploring how it can work together with RedisTimeSeries and the implications at an architectural level. This talk will cover use cases and integration patterns. These include topics such as streaming real-time data from external systems into Kafka and landing them into RedisTimeSeries for aggregation and analysis. For visual interpretation of this data, what options do we have in terms of integrating RedisTimeSeries with downstream systems? We also need to think about long term retention of the raw input data and kind of architectures that help facilitate this.

To better illustrate these concepts, we will also dive into an end to end implementation of a data pipeline on Azure and see it in action.