Session

How to Test a Confidential Agent Before Production

Most efforts to secure AI agents focus on infrastructure: encrypted memory, execution environments, attestation. But even inside a confidential runtime the agent can behave unsafely, selecting wrong tools, fabricating parameters, accessing data outside its scope, or following tool sequences that violate policy. These are application layer trust failures that need application layer testing.

This session presents a three layer evaluation framework built in Java with LangChain4j. Layer one is policy compliance: mock the LLM and verify the agent selects only approved tools with valid parameters. Layer two is judge based answer safety: a second model scores whether responses are grounded in results and free of filtered data. Layer three is trajectory validation: assert the exact tool sequence, because in regulated environments the path matters as much as the destination.

We build all three live in a JUnit suite, define golden datasets encoding expected sequences and forbidden tools, and wire it into CI that fails the build when behavior drifts outside policy. Attendees leave with a JUnit template for agent trust evaluation and a CI pattern making agent safety a build time gate.

Sachin Gupta

Technical Leader at eBay

San Jose, California, United States

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top