When developing a data & analytics platform in Azure Synapse or Azure Data Factory, you probably store data in a data lake, expose the data in a SQL Pool and have all the orchestration of the pipelines stored in a SQL database.

Imagine you have created multiple integration pipelines, all leveraging a centralized metadata database. You use this database to store your source-metadata, platform-metadata and logging for all runs. For a particular integration pipeline you need to make a change in a stored procedure, test the procedure itself, run the pipeline it is used in and all tests are successful. You deploy your software to the production-environment and suddenly other pipelines stop working.

You didn’t know the stored procedure was used there as well and didn’t test it.

Especially when working together in a team or even multiple teams, having limited insights of this impact can create breaking changes. Within this session I will show you how you can get an overview of all dependencies. Including dependencies between pipelines, datasets, linked services, notebooks etcetera within your Synapse or Data Factory platform so you know what you need to test.

Recommended knowledge:
- Familiar with Azure Synapse Analytics Workspace or Azure Data Factory
- Knowledge of building a Data platform within Azure
- Basic knowledge of SQL

Target audience:
- BI Developers/Data Engineers
- Data Architects