Data Ingestion using notebooks can require many steps and the orchestration of these steps.

Besides the other options, the notebooks can orchestrate their own execution. The steps can be broken down in multiple notebooks and one main notebook can orchestrate the execution of the all of them.

Beyond this capability, the orchestration can trigger parallel execution. Instead of executing the steps in sequence, they can be executed in parallel.

The parallel execution of notebooks bring many secrets, such as the exchange of parameters, complex parameter types and management of dependencies.

In this session we will analyse the secrets of orchestrating parallel executions with notebooks.