AI POCs Are Easy. Production Is Hard. Evaluation Closes the Gap.

Generative AI prototypes are easy. With a few prompts and a model endpoint, teams can create impressive demos in minutes. But once these systems meet real users and real data, the cracks appear: retrieval pipelines drift, responses hallucinate, costs and latency fluctuate, and agent workflows take unexpected paths.

The gap between a compelling POC and a reliable production system is rarely the model. It’s the absence of systematic Evaluation.

This session introduces Evaluation Driven Development as a practical engineering discipline for production AI systems. Using tools like the Microsoft's Evaluation SDK and Azure AI Foundry, we’ll explore how developers can instrument AI applications with automated evaluators to measure quality and safety.

From there, we’ll examine how evaluation applies across modern AI architectures including RAG pipelines, tool-calling agents, and multi-step reasoning workflows. You’ll see how to design evaluation datasets, run automated evaluation pipelines, and integrate these checks into CI/CD so changes to prompts, retrieval, or orchestration can be validated before reaching production.

Scaling AI systems isn’t about better demos. It’s about trust. Evaluation closes the gap.

Oct 7- 12 pm CT

HAX Toolkit: Guidelines, Patterns, and Real-World Examples for Human-Centered AI

Microsoft's Human-AI eXperience (HAX) Toolkit was designed in direct response to AI practitioners who have expressed that "AI is the most ambiguous space I've ever worked in... There aren't any real rules and we don't have a lot of tools." It's a comprehensive resource designed to address the ambiguity in AI design.

The HAX Toolkit includes essential components such as the HAX Design Guidelines, HAX Design Library, HAX Workbook, and HAX Playbook. These tools are crafted to provide clarity and structure in the design of human-AI interactions.

The toolkit synthesizes over 20 years of research and was introduced in an award-winning 2019 CHI paper, ensuring that it is grounded in real-world insights from AI practitioners. This session is ideal for product designers, AI developers, UX researchers, and design strategists looking to enhance their approach to AI product development.

Key takeaways:
-Explore the HAX Design Guidelines: Learn 18 evidence-based best practices for creating intuitive AI user experiences, synthesizing over 20 years of research.

-Explore the HAX Design Library: Discover a rich database of design patterns and real-world examples to effectively implement AI interaction guidelines.

-Leverage the HAX Workbook: Facilitate cross-functional collaboration and alignment in early-stage AI product development, prioritizing design goals and resource allocation.

-Utilize the HAX Playbook: Identify and mitigate potential AI system errors, particularly in natural language processing, to create more robust and user-friendly applications.

Building Trustworthy AI: Practical Approaches with Azure AI Foundry

We are moving on from building just intelligent systems to creating responsible, ethical, and impactful solutions. Join us for a hands-on session on building ethical and safe AI solutions using Azure AI Foundry. Let's cut through the jargon and discuss practical ways to make your AI projects more responsible. In this session, you'll learn:
• The basics of Responsible AI and why it matters
• How to use Azure AI Foundry's tools for safer AI development
• Practical tips for content moderation in AI applications
• Real-world techniques for ethical AI governance

The session is intended for developers, data scientists, and tech leaders who want to create AI that's not just smart, but also trustworthy. Let's see how it's done.
Key takeaways:
• Understand what Responsible AI really means in practice
• Learn to use Azure AI Foundry's safety features effectively
• Master practical content moderation techniques