Mastering Data Workflows: Microsoft Fabric's Metadata Pipeline Solution with the Medallion Architect

Microsoft Fabric provides both Data Lakehouse and Data Warehouse platforms for Data Analytics. In a separate post, I illustrated a Metadata Driven Pipeline pattern for Microsoft Fabric following the medallion architecture with Fabric Data Lakehouses used for both the Bronze and Gold layers and SQL views over tables for the Silver layer. Fabric Data Lakehouse is perfect for landing data into the Bronze layer. And if I can host my star schema in a Fabric Data Lakehouse, why consider Fabric Data Warehouse as the Gold layer? Because Fabric Data Warehouse has some features that may be a better fit for your organization.

Microsoft Fabric: All About The Data Analytics Platform

A breif introduction about fabric components and main concepts (OneLake, OneSecurity,V-order optimization and much more) and a short demo on how to execute ETL's using Fabric different components.

Step-by-Step Guide: Building ETL Workflows in Microsoft Fabric

In this session, we will delve into building robust ETL (Extract, Transform, Load) pipelines using Microsoft Fabric. This demonstration aims to enhance our data insights by leveraging powerful features of Microsoft Fabric.

Throughout the session, participants will explore the full capabilities of Microsoft Fabric, focusing on:

Workspace: Setting up collaborative environments for efficient data management.
Lakehouse and Warehouse: Constructing the Medallion architecture for unified data storage and processing.
Data Flow Gen2: Designing scalable and efficient data processing workflows.
Pipelines: Automating the flow of data from source to destination.
Notebooks: Using interactive notebooks for exploratory data analysis and pipeline development.
SQL Endpoints: Integrating structured query capabilities for advanced data manipulation.
Datasets: Organizing and managing data assets for streamlined access and analysis.
PowerBI Reports: Visualizing insights through interactive and insightful reports.
Data Agents: As a new frontier in self-service analytics, Microsoft Fabric Data Agents empower users to interact with data through natural language no need for SQL, DAX, or KQL. Built on Azure OpenAI, Data Agents bridge the gap between business users and complex data models by transforming everyday questions into optimized queries across lakehouses, warehouses, and Power BI datasets. In this session, we’ll explore how to configure Data Agents, define intent-driven instructions, and govern access to trusted datasets—unlocking fast, intuitive, and secure access to insights for everyone in your organization.
By the end of the session, attendees will have gained practical experience in harnessing these features to build resilient ETL pipelines, empowering them to extract maximum value from their data resources.

Prerequisites
Internet connectivity
Your own active Azure Subscribtion
Your own Microsoft Fabric Trial

Mastering the Art: Orchestrating ADF with the Power of Workflow Orchestration Manager

in-depth session where we tackle the challenge of dynamically invoking Azure Data Factory (ADF) pipelines using Apache Airflow. While ADF does not natively support this functionality, Apache Airflow provides a powerful solution to orchestrate these pipelines effectively.

What You'll Learn:
Running ADF Pipelines from Airflow:

Setup and Configuration: We will guide you through the prerequisites, including necessary tools and accounts (Azure, ADF, Airflow), and provide a step-by-step configuration process for integrating Airflow with ADF.
Creating a DAG (Directed Acyclic Graph): Learn how to create a DAG in Airflow and configure tasks to invoke ADF pipelines.
Execution and Monitoring: Experience a live demonstration of executing the DAG, monitoring, and managing the pipeline execution.
Running Custom Modules in Airflow:

Module Development: Discover how to create custom modules tailored for specific tasks.
Integration with Airflow: Understand the process of adding custom modules to Airflow DAGs.
Execution and Monitoring: See a live demo of executing custom modules and observing the execution process.

Session Highlights:
Overcome the limitation of ADF's native capabilities by leveraging Apache Airflow.
Gain hands-on experience through two detailed demos.
Enhance your data orchestration skills with practical, real-world applications.

This session is perfect for data engineers, developers, and IT professionals looking to expand their knowledge and capabilities in data pipeline orchestration using cutting-edge tools.

Building an AI-Ready Data Estate with the OneLake Catalog in Microsoft Fabric

Data is everywhere lakehouses, warehouses, cloud storage, databases, SaaS applications, and increasingly, AI agents that need access to it all.
The challenge is no longer collecting data; it's making data discoverable, trusted, and secure without creating governance bottlenecks.
In this session, we'll take a deep dive into the OneLake Catalog in Microsoft Fabric and explore how it unifies data discovery, governance, and security into a single experience. You'll learn how to organize your data estate with domains and workspaces, discover data through rich metadata and lineage, prepare data for AI experiences, and govern access with fine-grained security controls.

We'll also look at how OneLake Catalog enables a modern data mesh architecture, integrates with external catalogs and APIs, and empowers both technical and business users to confidently find and use trusted data.

Whether you're a data engineer, administrator, architect, or analytics leader, you'll leave with practical guidance for building an AI-ready, governed data platform in Microsoft Fabric.

Learn how OneLake Catalog helps you turn data chaos into trusted, discoverable, and secure data products for the era of AI and self-service analytics.