As data engineers, our real value isn’t writing one-off pipelines - it’s delivering reliable data that drives business outcomes. In this session, we’ll explore how Databricks’ new Lakeflow Connect and Declarative Pipelines features make it easier than ever to build reusable, maintainable ELT workflows in SQL.
We’ll start by showing how Lakeflow Connect accelerates ingestion from popular sources like Salesforce and Microsoft SQL Server (on-prem or in the cloud). You’ll see how to handle schema changes, enable CDC, and selectively load objects with minimal code.
Next, we’ll move into transformation with Declarative Pipelines (formerly Delta Live Tables), demonstrating how to define medallion-layer pipelines using parameterized SQL that’s portable across environments. We’ll walk through how expectations, merges, and autoloader complexity are abstracted away, enabling your team to deliver faster.
By the end of the session, you’ll see an end-to-end demo that takes data from source systems through bronze, silver, and gold layers — ready for analytics — and you’ll leave with practical patterns you can use to accelerate your own ELT development in Databricks.