An agent is fundamentally simple: Prompt + Tools + Model + Boilerplate. The first three are where product teams create differentiated value—domain-specific instructions, purpose-built tools, model selection. The last one is everything else: state management, conversation history, context compression, streaming, cancellation, tracing, memory, persistence.

The problem is that "boilerplate" isn't trivial. It's 80% of the code but 0% of the differentiation. Every team rebuilds the same orchestration loops, the same streaming infrastructure, the same history truncation logic. At ThoughtSpot, we built an Agent Platform that draws a hard line between agent logic and agent infrastructure, letting product teams ship customer-facing agents faster by owning only what matters: their prompts and their tools.

This talk covers the infrastructure patterns behind that separation, diving deep into five hard problems:

State management across tool calls. Agents need to track state that evolves through multi-step tool interactions. We evaluated two approaches: stateless tools (state lives on the agent, passed as arguments to each tool) versus stateful tools (state lives in the tool service). Stateless gives you better testability and lets the LLM reason about available state variables; stateful avoids serialization overhead for large objects. I'll walk through concrete flow diagrams, show how we propagate state updates via tool response metadata, and discuss when each pattern makes sense.

Configuration-driven agent definitions. Our agents are defined entirely through config—templated system prompts with conditionals, tool server endpoints, sub-agent delegation rules, tool output compression strategies. This lets teams spin up new agents without writing orchestration code. I'll show the schema design and discuss how we expose runtime variables (tool call count, accumulated context from tool responses) to prompt templates.

Inter-agent communication. Complex workflows often require specialized agents collaborating. We support two patterns: agents-as-tools (parent agent calls a sub-agent like any other tool, receives structured output) and agent handoff (full conversation transfer with history). The platform handles the routing, context passing, and conversation threading—product teams just declare which agents can delegate to which, and under what conditions.

Shared memory across agents. When memory lives in the platform rather than individual agents, something interesting happens: knowledge accumulates across agent boundaries. A user's interaction with one agent informs another. We implemented tiered memory with opt-in scoping—tenant-wide knowledge, org-level rules, user-specific context—with a retrieval pipeline that surfaces relevant memory regardless of which agent captured it. I'll cover explicit memory updates (via user feedback), implicit extraction, and the tradeoffs between memory tiers.

Tool protocol design. We standardized on MCP for tool interfaces, but the protocol alone doesn't solve everything. I'll discuss patterns we developed on top: call cancellation semantics, progress message streaming, context variable propagation in tool responses, and an adapter layer for integrating existing services like protobufs to MCP.
Building for customer-facing scale adds constraints: high concurrency, conversation encryption, strict tenant isolation, auditability. I'll touch on how these requirements shaped our API design and infrastructure choices.

Takeaways:
- A mental model for separating agent value (prompt + tools) from agent infrastructure
- State management patterns: when to put state on the agent vs. in tool services
- Inter-agent communication: agents-as-tools vs. handoff patterns
- Shared memory architecture that works across agent boundaries
- MCP extension patterns for production agent systems