Unleashing the Potential of Metadata-Driven ELT Framework
A metadata-driven ELT framework in Azure Synapse Analytics or in Microsoft Fabric is a way of organizing, optimizing and managing data pipelines that involves using metadata to define and control the flow of data from source to destination. This can be useful for organizations that have a large number of data pipelines and want to have more control over how data is processed and moved between systems.
In this session, we will discuss the benefits of using a metadata-driven approach to managing data pipelines. Our discussion will include practical examples and best practices for implementing a metadata-driven ELT framework in your organization based on the Medallion Lakehouse architecture. We will also provide you with code samples and walk them through how to get started with implementing this framework in your own work.
This session is ideal for data engineers and other technical professionals who want to learn how to optimize their data pipelines in Azure Synapse Analytics or Fabric. By the end of the session, you will have a clear understanding of how to use a metadata-driven approach to manage and maintain data pipelines, enabling better control and visibility over their data processes.
Erwin de Kreuk
Data Platform MVP | Lead Data and AI |Public Speaker | InSpark | Innovate to Accelerate
Rotterdam, NetherlandsView Speaker Profile