Session
The Infrastructure Beneath Agentic AI: Storage, Context, and the Cost of Every Decision
We spend a lot of time talking about how intelligent AI agents are becoming. But there is another side to the story that doesn't get nearly as much attention: the infrastructure they depend on.
Every time an AI agent retrieves information, searches a knowledge base, calls a tool, or makes another decision, something has to happen behind the scenes. Data must be available, storage must respond, networks must carry requests, and systems must deliver the right context at the right time.
What happens when the data is stale? When storage latency increases? When an agent makes repeated tool calls that create unexpected infrastructure load? Or when the underlying system becomes unavailable halfway through a workflow?
These are familiar problems for infrastructure engineers, but they take on new importance as organizations move from simple AI assistants to autonomous, multi-step agents.
Drawing on more than 15 years of experience with enterprise storage, distributed systems, Linux, and infrastructure automation, I'll explore the infrastructure challenges behind agentic AI and how we can design for them.
Using a hypothetical enterprise knowledge-retrieval workflow, I'll walk through a reference architecture showing how agents interact with storage, retrieval services, APIs, and operational infrastructure.
We'll examine:
- How context retrieval depends on data availability, freshness, and storage performance.
- Why traditional infrastructure metrics don't always explain poor agent performance.
- How repeated tool calls, retries, and parallel agent workloads can amplify latency and infrastructure demand.
- Where caching, observability, request limits, and failure handling fit into the architecture.
- How to think about reliability across the entire agent workflow, rather than treating each component independently.
Vinay Sherkar
Senior Storage Platform Engineer, Bloomberg
Philadelphia, Pennsylvania, United States
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top