Tzu-ping Chung
Archetecting the future for Apache Airflow
為 Apache Airflow 設計未來
Taipei, Taiwan
Actions
Staff software engineer with Astronomer, mainly working with open source Apache Airflow on new features and coordination on various contribution works. Prolific and experienced with OSS project organisation in general. Based in Taipei, Taiwan. 🇹🇼
Staff software engineer @ Astronomer Inc. 主要進行 Apache Airflow 開源專案相關工作，包含開發新功能、專案統整、以及其他貢獻。擁有豐富開源專案管理經驗。現住台灣台北。🇹🇼
Assets: Past, Present, Future
Airflow Asset originated from data lineage and evolved into its current state, being used as a scheduling concept (data-aware, event-based scheduling). It has even more potential. This talk discusses how other parts of Airflow, namely Connection and Object Storage, contain concepts related to Asset, and we can tie them all together to make task authoring flow even more naturally.
Planned topics:
- Brief history on Asset and related constructs.
- Current state of Asset concepts.
- Inlets, anyone?
- Finding inspiration from Pydantic et al.
- My next step for Asset.
Introducing Apache Airflow® 3 – The Next Evolution in Orchestration
Apache Airflow® 3 is here, bringing major improvements to data orchestration. In this keynote, core Airflow contributors will walk through key enhancements that boost flexibility, efficiency, and user experience.
Vikram Koka will kick things off with an overview of Airflow 3, followed by deep dives into DAG versioning (Jed Cunningham), enhanced backfilling (Daniel Standish), and a modernized UI (Brent Bovenzi & Pierre Jeambrun).
Next, Ash Berlin-Taylor, Kaxil Naik, and Amogh Desai will introduce the Task Execution Interface and Task SDK, enabling tasks in any environment and language. Jens Scheffler will showcase the Edge Executor, while Tzu-ping Chung and Vincent Beck will demo event-driven scheduling and data assets. Finally, Buğra Öztürk will unveil CLI enhancements for automation and debugging.
This keynote sets the stage for Airflow 3—don’t miss the chance to learn from the experts shaping the future of workflow orchestration!
Seeing Clearly with Airflow: The Shift to Data-Aware Orchestration
As Apache Airflow evolves, a key shift is emerging: the move from task-centric to data-aware orchestration. Traditionally, Airflow has focused on managing tasks efficiently, with limited visibility into the data those tasks manipulate. However, the rise of data-centric workflows demands a new approach—one that puts data at the forefront.
This talk will explore how embedding deeper data insights into Airflow can align with modern users’ needs, reducing complexity and enhancing workflow efficiency. We’ll discuss how this evolution can transform Airflow into a more intuitive and powerful tool, better suited to today’s data-driven environments.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top