Session
Databricks Lakeflow Declarative Pipelines from Code to Flows
Data teams often spend more time stitching tools together than moving data. What if you could build batch and streaming pipelines with a few lines of SQL or Python and let the platform handle orchestration, retries and change data capture?
Databricks Lakeflow Declarative Pipelines runs on the Databricks runtime and lets you declare flows instead of writing procedural Spark code. These flows read from sources and write to managed tables or external sinks while the pipeline manages dependencies and applies data‑quality expectations.
In this session we’ll explore the core capabilities of Lakeflow Declarative Pipelines. We’ll build an end‑to‑end ETL pipeline to ingest raw data, transform it and query the results. You’ll learn how flows, streaming tables, materialized views and expectations work together and how the framework automates orchestration. We’ll also review how Databricks Asset Bundles can automate development and deployment of pipelines.
By the end of the session, you’ll understand why declarative pipelines reduce complexity and improve reliability, know how to define flows and monitor pipelines with Databricks Workflows.
This session is aimed at data engineers and architects who want to simplify orchestration and get the most out of Databricks. Whether you’re migrating from Airflow/ADF or building new pipelines, you’ll see how to deliver reliable, incremental and declarative pipelines so you can focus on your data.
Falek Miah
Principal Data Engineering Consultant at Advancing Analytics
London, United Kingdom
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top