Building an AI agent that works once on your machine is a great feeling. But how do you confidently deploy it when the output is non-deterministic and changes on every run? When dealing with Generative AI, traditional unit tests like Assert.AreEqual() completely fall apart.

In this session, we are tackling the new frontier of AI development: Evaluation. Using the Microsoft Agent Framework in C#, I will introduce you to the fundamental concepts of testing AI applications, moving from "vibes-based" guesswork to actual metrics.

We will start with the basics, looking at fast, Local Evaluators to run simple sanity checks in your C# code, like verifying if your agent actually called the tool it was supposed to. Then, we will gently introduce the concept of "LLM-as-a-judge" using Azure AI Foundry, showing how you can use another AI model to score the relevance and coherence of your agent's responses.

Whether you are just starting with AI in .NET or looking for ways to stabilize your first intelligent applications, this session will give you the practical foundation you need to test with confidence.

Key Takeaways:

The AI Testing Problem: Why we need a new approach to our testing.

Local Checks in C#: How to write simple, fast rules using the Microsoft Agent Framework to verify tool execution and agent behavior.

Introduction to Cloud Evals: A clear explanation of "LLM-as-a-judge" and a practical look at using AI Foundry to measure quality and relevance.