Everyone wants to put an LLM into their data pipeline. I tested that idea on a hard, practical problem: catching bank statements that were quietly edited to fool a lender. I built an open benchmark of 600 statements and pitted tuned string-matching rules against open-source LLMs. The rules won on accuracy, but they missed almost half of the payee swaps without raising a single alarm. Letting the LLM decide everything was worse. Giving it one narrow job, “is this the same payee?”, cut missed fakes sevenfold. In 15 minutes you’ll see the results, the design pattern that made it work, and a simple rule for deciding where an LLM belongs in your own data pipelines. All code is open and runs for free.