"I deleted 20k items from prod" - said David (fake name), the backend team leader. He mistakenly triggered a deletion data pipeline by a wrong configuration. "Yeah David, I've once deleted an entire table - don't worry, we will help you fix this."

How could have David avoided this by designing data quality gates for his pipeline?
What are the possible patterns he could have used?
Let's build together a practical framework to help us reason about and design data quality for our data pipeline.

This talk is based on a talk I gave to backend and data engineers at Tikal.
It's also based on an article I published that got attention in the data community. It was featured in two publications, and one podcast.
Data Engineering Weekly by Ananth P.: https://lnkd.in/d7HEekk4
Modern Data Stack: https://lnkd.in/dwHMTdjB
Data eng weekly radio:
https://open.spotify.com/episode/29YPTJeeYMZqGDImGgVJba?si=jf-a3JBrSQiqm-PEwvLxFg