Most engineering effort goes into getting data in, and once the rows land, we trust them. This talk is about everything that happens after that, and why the post-write layer is where correctness actually gets decided. The case study is GovParti, a civic data platform covering all 3,127 U.S. counties, built solo with no team to catch mistakes downstream, so governance had to be engineered into the schema and storage rather than enforced by process.
The core idea is that every value carries its lineage at write time. A provenance record holds the verbatim upstream bytes next to the value, so a number never has to be refetched and survives a source going dark or changing underneath it. This is what makes data debuggable at all: when a number moves, you cannot tell your own ingestion bug apart from a genuine upstream revision unless you still hold what you ingested last time, and both wrong guesses corrupt your data in opposite directions. The source's own methodology is cached alongside the value too, because a number is only as defensible as the method behind it, and because two valid sources can contradict each other by definition rather than by value.
The talk closes on a hard lesson. A post-write quality layer, agents that judge each value against the national distribution, its own state, and the county's own history, was built, deployed, and then paused on purpose. Auditing the flagship checker, I found it was silently failing most of its core statistical tests, a type mismatch that made the database quietly return nothing rather than error. A validation agent that fails silently is worse than none, because it manufactures confidence you didn't earn. Validation is just another source, and it lies too.