You have a challenge: building an AI Agent in a very regulated industry with strong compliance requirements. This system will need to answer questions on hundreds of thousands of documents and search info across terabytes of data spread in tens of data sources.

In this talk we will analyze in deep the challenges to build such a system. We will analyze how we built a scalable RAG pipeline, how we built the right tools, how we evaluate the system and how we make it observable. We will go deep into the security guardrails these systems needs to have.

Also, we will discuss some tradeoffs and choices in terms of UI/UX needed for compliance with dynamic UI generation and how we can gather implicit and explicit feedback from users to further improve the system.

Building an AI agent in the real world is not just about the underlying model, it's about the entire system you create around it, and in this talk we will share some of the lessons we learned.