Session
Keep Conversation and Context Apart, Cut Your Token Bill
Your agent fetched a large dataset to answer one question. No error. But that payload now rides along in every model call, burning your token budget. The instinct is more memory. Your agent already has two. Conversation memory holds turns and facts, recalled by meaning. Context memory holds large tool outputs like logs, recalled by an exact reference. Most token and cost failures are one stored as the other. The fix is not more memory, it is the right memory. Store large outputs outside the window and keep a short reference in context. Each memory does what it is good at, and you decide where state lives by how you recall it.
What you'll learn: • Apply the two memory model to decide where any piece of agent state belongs before you write a tool • Design context offloading with a framework plugin so large tool outputs never re enter the window and your tools stay ordinary functions • Evaluate exact reference storage against semantic recall and choose the right one per data type • Build the production split across a managed memory service and object storage without leaking large payloads into the conversation • Implement selective tools that return summaries so the offloader stays a safety net, not the whole strategy Outline: • The large payload question • An agent has two memories • Context memory: offload large data outside the window • Production: two memories on purpose • Decide placement before you build
Elizabeth Fuentes Leone
Developer Advocate
San Francisco, California, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top