Bringing AI Agents Home: Data & Infrastructure for Local AI Agents

More and more companies want AI agents, but they can't send sensitive data to public cloud APIs. That leaves one option: running agents locally, on their own infrastructure, over their own data. This is where most projects get stuck. The model is the easy part. The hard part is everything around it.

This session covers what it really takes to run AI agents in-house. On the infrastructure side: GPU sizing, serving open-weight models on Kubernetes, and the trade-offs between model size, latency and cost. On the data side: data that agents can actually understand, a semantic layer that gives business context, RAG and vector search over internal data, and connecting agents to internal systems through MCP.

We'll also cover a problem that local deployment doesn't solve by itself: security. Agents must act with the user's permissions, not as an admin. That means row-level security, auditing every query, and protecting against prompt injection hidden inside company data.

Through a live demo, we'll take a naive local agent that leaks data and gives wrong answers, and turn it step by step into a secure, production-ready setup. Attendees leave with a practical checklist for making their data and infrastructure agent-ready.