Session
CI CD for Dataflow with Flex Templates and Cloud Build
The goal of this talk is showing a full example with a CI CD pipeline for Dataflow jobs.
The jobs will be based on Flex Template that is a way to standardize the deployment of Dataflow jobs and we are going to show a full example with Java and Python SDKs.
The CI CD will be orchestrated with Cloud Build based on a Github repository :
- Launch unit tests on push to the Github repository
- Manual job to deploy the Dataflow job with Flex Template
- Manual job to run the Dataflow job and template
In an extra and optional part, we will show the Dataflow Flex Template deployment with Dagger IO and Go.
Dagger is a tool that allows to write CI CD Pipeline As Code.
Mazlum Tosun
GDE Cloud and Head of Data & Cloud at GroupBees
Paris, France
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top