In this workshop, we explore an end to end example that combines batch and streaming aspects in one uniform Apache Beam pipeline. We start to analyze incoming taxi trip events in near real time with an Apache Beam pipeline.
We show how to archive the trip data to Amazon S3 for long term storage. We subsequently explain how to read the historic data from S3 and backfill new metrics by executing the same Beam pipeline in a batch fashion.
Along the way, you also learn how you can deploy and execute the Beam pipeline with Amazon Kinesis Data Analytics in a fully managed environment.

Key areas covered in the workshop are:

Create Ingestion Infrastructure
Deploy Streaming Pipeline
Beam on KDA
Deploy Batch Pipeline
Monitoring and Profiling

The workshop is aimed at engineers and technical/solutions architects and the overall duration for the workshop will be 2 hrs.