Big Data Analytics in Near-Real-Time with Apache Kafka Streams

Quit waiting on data analytics and achieve near-real-time with Apache Kafka streaming.

As companies generate and store increasing amounts of data, using and distributing that data can quickly become a challenge. The saying is "knowledge is power" and extracting knowledge from our data is the new norm. As the volume of data increases and the number of sources where we gather our data also grows, the standard ETL processes we've used for the past several decades are starting to break down in terms of their effectiveness and timeliness.

In this talk we'll review the standard batch ETL processes most people are familiar with, and then we'll step into the world of streaming data where we'll start processing data in real-time leveraging Apache Kafka as the shared backbone for connecting our streaming data with SQL Server and ElasticSearch. We'll be demonstrating real-time data aggregations and horizontal scaling using a combination of Kafka, Kafka Connect, KSQL and Kafka Streams.

If you're curious what it might be like to live in a world where you're not waiting for your batch processes to complete and you'd rather have instant access to your processed data as it comes in, come join us in the evolution of ETL.

Learn how Apache Kafka can take you from batch processing to real time streaming and connect different data systems such as SQL Server and ElasticSearch.