You shipped an AI feature. The demo worked, everyone smiled, someone probably said: "This is amazing."

Now the uncomfortable question:
How do you know it still works tomorrow?

Most teams do not have an AI problem. They have a measurement problem. They have built a dashboard where the needles are painted on, and everyone is politely pretending it is a cockpit.

The usual response is: "You can't unit test an AI model."

True.

But you are not testing the model. You are testing the behaviour of the system you built around it.

Your AI feature has requirements:
* It should not leak the system prompt.
* It should return valid JSON.
* It should include the important caveat, not just the impressive number.

Those things are testable!

In this live coding session, we will build an AI evaluation harness in C# from scratch. We will create a golden dataset, test behaviours instead of exact answers, add an LLM-as-judge approach (including the calibration step everyone forgets), detect regressions, and wire it into CI.

Because an AI agent that is correct 95% of the time sounds impressive.

Until it runs a thousand times a day.

Then you have fifty confident, articulate mistakes every day, each carrying your company's name.

This session is based on evaluation systems we have built for real-world AI applications, including one that caught a legally required disclosure disappearing shortly before release.
You will leave with working code and a practical answer to the only question that matters after changing an AI system:
Did we make it better, or did we just make it different?

Live coding in C#. Code published after the session. A sequel to the 2026 Architecting AI workshop at FutureTech, but it stands on its own. Uses real-world examples from projects we've done in Australia.