M. Waqas Shahid
Delivery Hero, Principal Data Platform Engineer
Zürich, Switzerland
Actions
Engineer and Architect with more than 10 years of experience, working to create seamless data journey for thousands of users across the organisation. Helping to deliver simplified data platform implementing data mesh. Passionate about building easy to manage and fast data platform.
Area of Expertise
Topics
The Platform Team Said No — dbt won with “Yes” - Story of Governance, Autonomy, and Scale
At Delivery Hero, one of the world's largest food delivery platforms, we run a decentralised data engineering org — 50+ teams, 70+ countries, 60+ PB data, 300k+ tables. Teams running autonomous pipelines, and domain engineers wanting to create data products rapidly enabling data driven business decisions. Complex setup, but initially all without dbt!
It won’t be a talk, but rather a live dialogue between two practitioners from opposite sides of that negotiation. One sat on the "central data platform team", responsible for standardisation, quality and governance. The other sat in a "domain team" owning their pipelines. We'll recreate the real friction — the "you can't do that" moments — and show how dbt gave both sides a shared language to work with.
How dbt forced the platform team to enable it: With its ease of use, small effort to migrate, suite of features like lineage and dbt tests. The features we've leaned on, the ones we're still exploring, and what we haven't figured out yet.
Enterprise-Grade Airflow Upgrade: Strategies & Deep Dive
Upgrading to Apache Airflow in large, production-grade environments can be complex—especially in enterprise setups with hundreds of DAGs, custom plugins, and mission-critical pipelines. The challenge grows even more complex in decentralized setups, where platform teams are responsible for the system's stability, but the DAG code lives across multiple teams you don’t directly control.
You will have the chance for personalised review of your current organizational setup, assess testing coverage, and identify concrete ways to improve your upgrade process. This hands-on workshop will provide:
- Environment Health Check & Audits (dependency checks, resource sizing)
- DAG refactoring for deprecated features and optimizations
- Database migrations and backward-compatibility strategies
- Improving CI/CD validation using GenAI to increase reliability
- Self-managed and Astronomer upgrade (with no downtime)
Supported by battle-tested approach and guided exercises. Recommended for platform teams, data engineers, and architects managing production Airflow deployments. At the end of this workshop participants will gain actionable strategies tailored to their specific upgrade challenges.
Airflow 3 - An Open Heart Surgery
Curious how code truly flows inside Airflow? Join me for a unique visualisation journey into Airflow’s inner workings (first of its kind) — code blocks and modules called when certain operations are running.
A walkthrough that unveils task execution, observability, and debugging like never before. Scaling of Airflow in action, showing performance comparison b/w Airflow 3 vs 2. This session will demystify Airflow’s architecture, showcasing real-time task flows and the heartbeat of pipelines in action.
Perfect for engineers looking to optimize workflows, troubleshoot efficiently, and gain a new perspective on Airflow’s powerful upgraded core. See Airflow running live with detailed insights and unlock the secrets to better pipeline management!
Airflow and Data Mesh - Architecture, Setup & Monitoring
Ever thought how airflow could play a pivotal role in data mesh architecture, hosting thousands of DAGs and hundreds of thousands daily running tasks, let's find out!
- In this workshop we will review how an organisation can setup data mesh architecture. What requirements should be met to adapt this architecture.
- Airflow and it's role in Data Mesh
- How to setup monitoring for running scalable airflow
- How to manage cross team DAG dependencies
- How to manage multi-tenancy
- Challenges and factors to keep in mind to make it a success
This workshop should be suitable for any Architect, Devops and Data Engineers. At the end of this workshop you would have solid understanding for using airflow as pivotal part in an organisation's data mesh framework.
Airflow at Delivery Hero: Running a data mesh with ~500 airflows instances
Ever thought how airflow could play a pivotal role in data mesh architecture, hosting thousands of DAGs and hundreds of thousands daily running tasks, let's find out!
Delivery Hero delivers food in 70 countries with 12 different brands and platforms. With thousands of engineers, analysts and data scientists spread across many countries running analytics and ML services for all these orders delivered. Serving the workflow orchestration needs for such a massive group becomes a challenge.
This is where airflow and data mesh comes to rescue, by running more than 500 airflow instances to empower different teams to own and curate data products. This presentation will explain how to efficiently setup and monitor airflow at massive scale. New feature of launching dynamic airflow staging and development environments dedicated for each developer. Demo about "Workspace" concept in direction of multi-tenancy management.
Airflow and Optimised Data Platform: Setup & Customisations
Airflow has been used by many companies as a core part of their internal data platform. Would you be interested in finding out how airflow could play a pivotal role in achieving data engineering excellence and efficiency using modern data architecture. The best practices, tools and setup to achieve a stable but yet cost effective way of running small or big workloads, let's find out!
In this workshop we will review how an organisation can setup data platform architecture around airflow and necessary requirements.
- Airflow and it's role in Data Platform
- Different ways to organise airflow environment enabling scalability and stability
- Useful open source libraries and custom plugins allowing efficiency
- How to manage multi-tenancy, cost savings
- Challenges and factors to keep in mind using Success Matrix!
This workshop should be suitable for any Architect, Data Engineer or Devops aiming to build/enhance their internal Data Platform. At the end of this workshop you would have solid understanding of initial setup and ways to optimise further getting most out of the tool for your own organisation.
dbt Summit 2026 Sessionize Event
Airflow Summit 2026 Sessionize Event
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top