Session
Enforcing Data Quality at Every Layer: Lakeflow Declarative Pipelines Expectations in Databricks
Bad data doesn't announce itself - it slips quietly into dashboards, breaks downstream models, and erodes trust in your Lakehouse long before anyone notices the root cause. Lakeflow Declarative Pipelines (formerly Delta Live Tables) expectations let you catch it at the source, enforcing data quality rules directly inline as data moves through bronze, silver, and gold - without bolting on a separate framework.
This session is a hands-on look at LDP expectations: how to define them, the difference between warn, drop, and fail actions, how to quarantine and inspect violating records, and how to structure expectations across pipeline layers so quality gets stricter as data gets closer to production. We'll also look at how expectations surface in pipeline event logs and metrics, so failures are visible, not silent.
You'll leave knowing exactly how to design an expectations strategy for your own pipelines - catching bad data before it ships, without leaving the Databricks-native tooling you already have.
Kamil Nowinski
Strategic data architecture for the Microsoft & Databricks stack - deliberately, not accidentally
Stevenage, United Kingdom
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top