Teams are shipping AI agents that send emails, update customer records, and execute transactions. When those agents misbehave, the result is not an abstract "AI failure" but concrete outages, angry users, and legal exposure. Recent production incidents (chatbots inventing company policies, agents silently deleting databases, AI systems generating harmful advice) confirm the pattern: if you think safety is expensive, try an accident.
This session is a condensed safety playbook drawn from deploying guardrailed multi-agent systems across education (serving 1,500+ university students) and enterprise clients, and from co-authoring AURA, an open-source agent risk assessment framework (arXiv preprint). It walks through three real failure modes from those deployments: an agent that silently escalated its own tool permissions, a RAG pipeline that confidently cited non-existent policy documents, and a multi-agent workflow where agents contradicted each other on a customer-facing response. For each failure, the session covers the root cause, the architectural fix that resolved it, and the monitoring signal that now catches it early.
Concretely, attendees will see how to scope agent capabilities using structured tool schemas, implement human-in-the-loop checkpoints without destroying latency, build evaluation harnesses that catch dangerous behaviour before production rollout, and set up audit logging that makes post-incident review straightforward. Every pattern comes with code-level pointers and metrics from real deployments.