I build digline (digline.dev), an open source regression gate for LLM applications. The approved run gets committed in your repo as the reference, and every change is compared against it, with the judge's own noise measured first.

Before that, thirty years of enterprise ERP and HR software, which is where I learned what a silent regression costs.

I write about the ways measurement goes wrong when the thing doing the measuring is itself a language model. Judges that stop discriminating, checks that can't fail, numbers that improve while the system gets worse. All of it from my own systems, with the run files published.