Iain Dobson
Data Engineer, Quorum
Edinburgh, United Kingdom
Actions
Iain is a seasoned IT professional with over two decades of experience. Based in an employee-owned IT consultancy in the heart of Scotland, he views his role more as a vocation than a job, as it continually presents new learnings and opportunities to meet interesting, intelligent people with shared interests.
Believing in the transformative power of IT, learning, and data, Iain advocates for ethical and accessible IT solutions to drive change. He combines technical proficiency with a passion for sharing knowledge with the IT community.
Area of Expertise
Topics
Database DevOps Safety Harness: Automating Data Quality Gates with Great Expectations and Soda
Ever had a carefully managed database release where everyone swore they had done everything right, only to discover new data issues as soon as it hit production? If you have not, please stay after the session and tell us how you've managed it. Even when schema changes and migration scripts are carefully controlled, releases are still exposed to data problems that only show up once real workloads and real data get involved.
This session shows how to treat data quality checks as first-class tests in your database DevOps pipeline, using Great Expectations and Soda. Using Azure DevOps and Azure SQL Database, we focus on common pain points such as schema drift between environments, unexpected row-count changes, broken reference data and the classic “it worked in test but…”.
We will cover:
* Designing checks that guard typical database failure modes such as drift, unexpected nulls, duplicated keys and broken reference data.
* Wiring checks into CI/CD pipelines so every database change runs through the same automated gates as your migration scripts.
* Surfacing failures in a way DBAs and engineers actually use: build breaks, dashboards and simple, shareable reports.
You will see live demos against Azure SQL Database using free-to-use tooling, including a side-by-side comparison of Great Expectations and Soda. The session is delivered from a data engineer’s perspective, with a focus on pragmatic guardrails.
What You’ll Learn (Database DevOps–aligned)
* Testing strategies for database changes: how to express data quality rules as tests (row counts, distributions, referential integrity and key business rules) and run them on every deployment.
* Schema versioning and drift detection in practice: how to combine migration tooling with Great Expectations and Soda checks to detect schema drift, breaking changes and incompatible data during rollout.
* CI/CD pipelines for SQL data platforms: concrete Azure DevOps pipeline patterns for Azure SQL Database that build, deploy and then run data quality suites, failing the build or blocking promotion when checks do not pass.
* Choosing and operating the right tool: strengths and trade-offs of Great Expectations versus Soda for database DevOps, including setup effort, how checks are defined and versioned, and how they fit alongside existing DBA tooling.
Audience Relevance
* DBAs: understand how to implement low-friction safety nets around your existing deployment process without replacing current tools or workflows.
* Data engineers: learn how to add stability to database release processes by treating data quality checks as part of the deployment and not an afterthought.
* DevOps engineers: extend standard CI/CD patterns (build - test - release) to data and schema together for more reliable database releases.
Format
A 60-minute practical session combining:
* Short framing of common database DevOps failure cases.
* Introduction to Great Expectations and Soda in a SQL-first context.
* Live demos: Azure DevOps pipeline plus Azure SQL Database plus Great Expectations and Soda checks.
* Platform-agnostic design patterns you can adapt to SQL Server and other relational platforms.
Optimising Azure Architecture: A Deep Dive into Service and Private Endpoints
Knowing how to effectively implement Azure network services for security and performance requires a proper understanding of when to use Azure Service Endpoints and Private Endpoints.
This presentation provides an overview of each endpoint type and discusses the benefits and drawbacks of each. Attendees will learn how to make informed decisions on usage, avoid common pitfalls, and apply best practices for a secure Azure environment.
While the session delves deeply into the technical aspects, we will start from the basics, introducing networking components at a beginner level to ensure everyone can follow along. This approach ensures that even those new to endpoint security will grasp the foundational concepts before we explore the advanced topics.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top