Session

A Better Prompt Won't Fix Your Agent. A Better Harness Will

A demo is one clean session with a friendly prompt. Production is a different problem, and the same agent that dazzled in the demo starts failing in ways the model cannot detect on its own. It hallucinates a fact and writes it to memory, where it reloads as trusted context every future session. It reads a poisoned web page and stores the attacker's instruction as if it were its own. It books a three-step trip, trusts every tool that says "confirmed", and ships a trip with a step that silently never saved. It re-reasons the same deterministic answer a thousand times, miscounting and re-paying for it on every call. The instinct is to fix it in the prompt: a sharper system prompt, more rules, a bigger model. None of that holds, because none of these is a prompt problem. They are harness problems. This talk reproduces each failure, then fixes it in the harness around the model: validate what enters memory before the write, gate the dangerous action at the tool boundary, verify each step against ground truth and retry only what failed, and turn repeated reasoning into a tool the agent writes once and reuses. You leave with a way to decide which harness-level control any given failure needs, and an honest account of what each control can and cannot catch.


Outline: • The demo lied to you • Diagnose before you fix: break it on purpose • Memory it cannot trust • Memory it should not have trusted • The step that silently failed • Work it keeps re-paying for • The decision rule + Q&A

Elizabeth Fuentes Leone

Developer Advocate

San Francisco, California, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top