Clean, reliable data is essential - not just for analytics and machine learning, but for trust in every data-driven decision. As data pipelines grow more complex, ensuring data quality consistently can become a major challenge.

This talk explores how to automate data quality checks using two powerful data validation tools - Great Expectations and Soda - with a focus on embedding checks into everyday CI/CD workflows using Azure DevOps.

We begin by examining the core drivers and challenges of data quality, including schema drift, data volume shifts, and integration friction. From there, we’ll explore each tool with live demonstrations, show how they validate assumptions, reduce risk, and foster transparency. You will learn how to migrate to and automate tests in Azure DevOps pipelines and close with a comparative overview of Great Expectations vs. Soda, and discussion on how automation promotes a collaborative data culture - making teams more confident and proactive.

What You’ll Learn:
* Key data quality risks and how they arise in real-world pipelines
* How to structure data validations using Great Expectations and Soda
* Practical techniques for embedding data checks into CI/CD workflows
* How to choose the right tool for your needs
* How automated data quality supports team empowerment and trust

Audience Relevance:
* Data Teams: Improve integrity and reproducibility of datasets.
* DevOps Practitioners: Add guardrails for better data reliability in deployments.
* Engineering Leaders: See how automation drives cultural and technical resilience.

Format: A mix of concise conceptual framing, hands-on demos, and platform-agnostic implementation examples using free-to-use tooling.