Session

The Write Way: Building Bi-Directional Data Pipelines in the Lakehouse

Most teams can read data from the Lakehouse with ease. Writing data back in a clean, governed, and repeatable way is where things usually fall apart. The result is the same story everywhere: people build side databases, spreadsheets, shadow apps, or messy one‑off scripts just to let business users make changes. This creates silos, weakens governance, and breaks the idea of a single source of truth.

This session shows a simple, practical pattern for bi‑directional data movement that avoids all of that. The approach keeps the Lakehouse as the authoritative store, while giving users a fast, friendly place to edit their data without waiting on engineering teams. Databricks Unity Catalog (Lakehouse) syncs efficiently into PostgreSQL (Lakebase) for low‑latency read/write operations. A Databricks‑hosted web app gives users an easy interface for updates. Lakeflow jobs then sync those changes back to the Lakehouse every few minutes. The loop stays tight, governed, and reliable.

I’ll walk through real scenarios like form‑based data entry, table editing, and data‑quality workflows. All built with Dash and Databricks Asset Bundles to help deploy the apps cleanly with versioning, CI/CD, and automation.

By the end, you'll have a clear blueprint for building bi-directional data flows that empower users without sacrificing governance or reliability.

Falek Miah

Principal Data Engineering Consultant at Advancing Analytics

London, United Kingdom

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top