Session

Airflow and Data Mesh - Architecture, Setup & Monitoring

Ever thought how airflow could play a pivotal role in data mesh architecture, hosting thousands of DAGs and hundreds of thousands daily running tasks, let's find out!

- In this workshop we will review how an organisation can setup data mesh architecture. What requirements should be met to adapt this architecture.
- Airflow and it's role in Data Mesh
- How to setup monitoring for running scalable airflow
- How to manage cross team DAG dependencies
- How to manage multi-tenancy
- Challenges and factors to keep in mind to make it a success

This workshop should be suitable for any Architect, Devops and Data Engineers. At the end of this workshop you would have solid understanding for using airflow as pivotal part in an organisation's data mesh framework.

M. Waqas Shahid

Delivery Hero, Principal Data Platform Engineer

Zürich, Switzerland

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top