Building a public-facing chatbot powered by Large Language Models (LLMs) comes with some unique challenges, ensuring responses are accurate, grounded, and aligned with ethical guidelines and business regulations.

In this session, we’ll explore how we leveraged Azure OpenAI to create a customer service chatbot that "plays nice"

Some key takeaways:

- Grounding responses with confidence: How we leveraged external data sources, content moderation, and prompt engineering to ensure the chatbot consistently delivers accurate and grounded responses.

- Preventing harmful interactions, techniques for mitigating risks of the chatbot engaging in inappropriate or harmful conversations, including ethical considerations.

- Automating Evaluation Pipelines: How we developed automated systems to evaluate chatbot responses, ensuring measurable improvements and avoiding regressions when updating prompts or models.

- Architectural decisions and Azure services that we used to create a scalable and resilient chatbot solution.

- Lessons learned from real world usage