Session
Airflow and Data Mesh - Architecture, Setup & Monitoring
Ever thought how airflow could play a pivotal role in data mesh architecture, hosting thousands of DAGs and hundreds of thousands daily running tasks, let's find out!
- In this workshop we will review how an organisation can setup data mesh architecture. What requirements should be met to adapt this architecture.
- Airflow and it's role in Data Mesh
- How to setup monitoring for running scalable airflow
- How to manage cross team DAG dependencies
- How to manage multi-tenancy
- Challenges and factors to keep in mind to make it a success
This workshop should be suitable for any Architect, Devops and Data Engineers. At the end of this workshop you would have solid understanding for using airflow as pivotal part in an organisation's data mesh framework.
M. Waqas Shahid
Delivery Hero, Principal Data Platform Engineer
Zürich, Switzerland
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top