The evolution of GenAI applications brings with it the challenge of developing testing methods that can effectively evaluate the complexity and subtlety of responses generated by advanced artificial intelligences.

The proposal to use an LLM as a Validator Agent represents a promising approach, paving the way towards a new era of software development and evaluation in the field of artificial intelligence. Over time, we hope to see more innovations that allow us to overcome the current challenges and maximize the potential of these transformative technologies.

This proposal involves defining detailed validation criteria and using an LLM as an “Evaluator” to determine if the responses meet the specified requirements. This approach can be applied to validate answers to specific questions, drawing on both general knowledge and especialised information. By incorporating detailed instructions and examples, an Evaluator can provide accurate and justified evaluations, offering clarity on why a response is considered correct or incorrect.

In this session we’ll leverage langchaingo to interact with LLMs, and Testcontainers Go to provision the runtime dependencies to use RAG.

Accepted in Gophercon SG 2025