Mocheng Guo
Software Engineer at Airbnb
Actions
Mocheng is currently working in Airbnb to improve the workflow orchestration infrastructure.
On-Demand DAG through the REST API
In this talk we want to present how Airbnb extends the REST api to support on-demand workload. A DAG object is created from a local environment like Jupyter notebook, serialized into binary and transported to the API. The API persists the DAG object into the meta DB and Airflow scheduler and worker are extended to process this new kind of DAG.
Mocheng Guo
Software Engineer at Airbnb
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top