Beyond the Demo: Testing Strategies for Production AI Agents

AI agents are easy to demo, but much harder to trust in production than traditional software. They can return different results for the same prompt and context, and we cannot prove they will behave correctly every time. A small prompt change, model switch, tool response, timeout or streaming issue can change the behaviour of the whole application. So how do we test an AI agent that is part of a real system?

This session does not replace the unit and integration tests you would already write for a JVM application. Instead, it focuses on the additional testing strategies you need once an AI agent becomes part of the system.

We will demonstrate a practical testing strategy for production AI agents and run the tests live. First, we will test the agent graph itself: verifying flow, decisions and tool calls without paying for model inference on every test run. This gives fast, deterministic feedback about the orchestration of the agent, while making clear what this layer does not prove: the quality of the model-dependent output.

Next, we will use eval-driven development. We will update a prompt live, run evaluation tests, compare the results, and discuss how this helps when improving prompts, changing context, switching models or preventing regressions.

Finally, we will test the application around the agent. We will use mocked AI inference directly in the cloud to simulate slow responses, streaming and provider failures, so we can run repeatable cloud integration tests without depending on real AI services for every scenario.

Attendees will leave with a layered testing model for production AI agents: graph tests for deterministic orchestration, evals for model-dependent behaviour and prompt quality, and mocked cloud integration tests for the full application experience.

7 ± 2 Kotlin Features That Can Work For or Against You

In this live coding session, I’ll walk through Kotlin features that can work for or against you. For each one, we’ll show the problem it solves, the simple usage, side effects or gotchas, and then an advanced use case, using Jupyter notebooks for live coding.

Live coding session includes features like:
* getOrNull and getOrElse for safe collection access
* as? for safe casts
* require, check, checkNotNull, and error for assertions in the code
* runCatching for dealing with exceptions

Clean Architecture for Serverless: Business Logic You Can Take Anywhere

As cloud platforms continue to evolve, serverless functions provide powerful ways to build and scale applications with reduced infrastructure concerns. However, cloud-specific requirements often make it challenging to keep business logic portable across different cloud providers. This session presents a variant of clean architecture designed to separate core business logic from cloud-specific code, leveraging Kotlin’s capabilities along with Spring Cloud Function and headless Gradle modules.

In this talk, I’ll demonstrate how to structure a Kotlin application for serverless using a clean architecture approach for serverless. This structure maintains cloud-agnostic business logic while allowing platform-specific implementations for integration functionality like storage. By isolating business logic in headless modules, developers can deploy the same core code across multiple cloud function providers such as AWS Lambda and Azure Functions.

Through live coding, we’ll explore practical techniques for building and deploying serverless applications with cloud-agnostic business logic.
This session is aimed at developers interested in serverless architecture, cloud flexibility, and clean architecture principles.

For in-person presentation I will need an internet connection for the live deploy and demo