Session

When the Runbook Runs Out: Encoding Tribal Knowledge So Agents Can Debug the Unknown

On-call runbooks and agent skills are great at one thing: known, prescriptive steps. But healthy teams fix root causes, so the incidents that actually page you are the novel ones no runbook covers, usually across several services. The knowledge to debug them lives in a handful of domain experts' heads, and LLMs on their own struggle to reason across large distributed systems.

We encode that tribal knowledge for agents instead of scripting their steps. We treat MCP servers as far more than API wrappers: they expose a service's anatomy, rationale, instrumentation, and known failure modes. Paired with portable "teaching" skills that capture how services interact, an agent fuses this encoded context with live telemetry to reason toward a root cause it was never shown.

Built on this approach, our agents help run the pipeline validating the AI behind Zoox's robotaxis: 10-20 million simulations on tens of thousands of GPUs, where one failure spans many services and no runbook can cover it.

Hiral Mehta

Site Reliability Engineer, HPC Infrastructure at Zoox

Mountain View, California, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top