Session
Keep Conversation and Context Apart, Cut Your Token Bill
Your agent fetched a large dataset to answer one question. No error. But that payload now rides along in every model call, burning your token budget. The instinct is more memory. Your agent already has two. Conversation memory holds turns and facts, recalled by meaning. Context memory holds large tool outputs like logs, recalled by an exact reference. Most token and cost failures are one stored as the other. The fix is not more memory, it is the right memory. Store large outputs outside the window and keep a short reference in context. Each memory does what it is good at, and you decide where state lives by how you recall it.
What you'll learn: • Tell conversation memory and context memory apart, and decide where each piece of agent state belongs before you write a tool • Keep large tool outputs out of the context window so they stop riding along in every model call, while your tools stay ordinary functions • Evaluate exact reference storage against recall by meaning, and pick the right one for each data type • Build the production split: conversation in a managed memory service, large data in object storage, no payloads leaking into the chat • Implement tools that return summaries so offloading stays a safety net, not your whole strategy Outline: • The large payload question • An agent has two memories • Context memory: offload large data outside the window • Production: two memories on purpose • Decide placement before you build
Elizabeth Fuentes Leone
Developer Advocate
San Francisco, California, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top