Session

Real-Time Everything: Blending Streaming and Batch on a Modern AWS Stack

Most data teams end up running streaming and batch side by side — not by design, but because batch never fully disappears. The fraud pipeline needs sub-second latency. The financial report still runs overnight. And nobody warned you that adding streaming to an existing batch platform means owning two systems, two operational models, and twice the on-call surface area. This session gives you the decision framework and architecture to make them coexist. We start with a workload classification model for deciding when real-time is worth the complexity — over-engineering toward streaming is one of the most expensive mistakes a data platform team makes, and this framework helps you avoid it before the infrastructure is already in place. From there we walk through the hybrid AWS architecture: Kinesis Data Streams and MSK for ingestion, Lambda and Managed Flink for real-time processing, Glue and Step Functions for batch orchestration, and Apache Iceberg on S3 as the unified storage layer that keeps both worlds consistent. The 60-minute format lets us go deep on the operational problems most talks skip: exactly-once semantics across paradigm boundaries, late-arriving data that silently corrupts aggregates, schema evolution that breaks consumers downstream, and unified observability across streaming and batch. Every pattern is drawn from production systems processing petabytes of data.

Anusha Kovi

Business Intelligence Engineer, Amazon

Seattle, Washington, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top