Every conference talk about AI ends with something working. This one is about the things that did not, and what they cost us before we were willing to admit it.

A tour of features that demoed beautifully and then died. The ones where accuracy plateaued just below useful. The ones whose token bill scaled faster than the value they created. The ones that worked perfectly until real users held them the wrong way round. And the one where the honest answer turned out to be a database query and an afternoon of work.

Each gets the same treatment: what we built, why it looked right at the time, what the first real signal was that it was not, how long we took to accept that signal, and what I would check first now. That includes the failures that were our own fault, through bad framing, missing evaluation, or a prompt that nobody owned, and the ones that were simply the wrong shape for the technology.

Then the part that is actually useful to you: the questions we now ask before starting anything. What is the cost per successful outcome, rather than per call. What level of accuracy would make this worth having, and is that plausible. What does this look like when it is wrong, and who finds out. What is the boring solution, and why exactly is it not enough.

No vendor pitch and no redemption arc where it all works out at the end. Some of it did not work, and finding that out early is worth more than another success story.

Takeaways

- A pre-mortem checklist for agentic features, drawn from real failures
- Cost per successful outcome as a design metric, and how it changes decisions
- Early warning signs that an accuracy ceiling sits below your usefulness threshold
- How to kill a project with the organisation's goodwill intact
- The cases where the boring non-AI solution is still the right answer

Preferred duration: 45 minutes including Q&A. Also works well as a 30 minute session or a 20 minute keynote-style slot.

Target audience: engineers, architects and engineering leaders working on AI features. Accessible to a mixed audience; very little code.

Level: all levels.

Technical requirements: my own laptop (USB-C / HDMI).

First public delivery: not yet delivered.