Session
Database DevOps Safety Harness: Automating Data Quality Gates with Great Expectations and Soda
Ever had a carefully managed database release where everyone swore they had done everything right, only to discover new data issues as soon as it hit production? If you have not, please stay after the session and tell us how you've managed it. Even when schema changes and migration scripts are carefully controlled, releases are still exposed to data problems that only show up once real workloads and real data get involved.
This session shows how to treat data quality checks as first-class tests in your database DevOps pipeline, using Great Expectations and Soda. Using Azure DevOps and Azure SQL Database, we focus on common pain points such as schema drift between environments, unexpected row-count changes, broken reference data and the classic “it worked in test but…”.
We will cover:
* Designing checks that guard typical database failure modes such as drift, unexpected nulls, duplicated keys and broken reference data.
* Wiring checks into CI/CD pipelines so every database change runs through the same automated gates as your migration scripts.
* Surfacing failures in a way DBAs and engineers actually use: build breaks, dashboards and simple, shareable reports.
You will see live demos against Azure SQL Database using free-to-use tooling, including a side-by-side comparison of Great Expectations and Soda. The session is delivered from a data engineer’s perspective, with a focus on pragmatic guardrails.
What You’ll Learn (Database DevOps–aligned)
* Testing strategies for database changes: how to express data quality rules as tests (row counts, distributions, referential integrity and key business rules) and run them on every deployment.
* Schema versioning and drift detection in practice: how to combine migration tooling with Great Expectations and Soda checks to detect schema drift, breaking changes and incompatible data during rollout.
* CI/CD pipelines for SQL data platforms: concrete Azure DevOps pipeline patterns for Azure SQL Database that build, deploy and then run data quality suites, failing the build or blocking promotion when checks do not pass.
* Choosing and operating the right tool: strengths and trade-offs of Great Expectations versus Soda for database DevOps, including setup effort, how checks are defined and versioned, and how they fit alongside existing DBA tooling.
Audience Relevance
* DBAs: understand how to implement low-friction safety nets around your existing deployment process without replacing current tools or workflows.
* Data engineers: learn how to add stability to database release processes by treating data quality checks as part of the deployment and not an afterthought.
* DevOps engineers: extend standard CI/CD patterns (build - test - release) to data and schema together for more reliable database releases.
Format
A 60-minute practical session combining:
* Short framing of common database DevOps failure cases.
* Introduction to Great Expectations and Soda in a SQL-first context.
* Live demos: Azure DevOps pipeline plus Azure SQL Database plus Great Expectations and Soda checks.
* Platform-agnostic design patterns you can adapt to SQL Server and other relational platforms.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top