Thomas Sabol
Senior cloud solutions architect
Copenhagen, Denmark
Actions
Building with serverless & AWS. Optimizing, automating and solving challenges @Lenus with AWS.
Links
Area of Expertise
Topics
Evaluating AI Agents in Production
Call add(2, 3) a thousand times and you get 5 every time. Ask an AI agent to process the same refund request three times and you may get three different answers, take three different paths through your tools, and still get an HTTP 200 on each one. Classic QA breaks: assertEquals fails on harmless variation, and one passing run proves almost nothing.
Physics hit the same wall a century ago and answered it with probability models instead of exact predictions. In this session we do the same for agents. We’ll look at where non-determinism really comes from (sampling, path, environment, supply chain), what an evaluation should measure (outcome and trajectory), and how to build an evaluation pyramid from cheap deterministic checks up to LLM-as-a-judge and human review. Then we’ll wire it into a release lifecycle with CI gates, shadow traffic, canaries and online evals, using observability on AWS as the measuring instrument. The examples come from a live refund agent built with Strands Agents and Amazon Bedrock.
Beyond the Warehouse Walls: Scaling logistics with Serverless First Principles
- Describing Serverless First Principles, including event-driven design, scalability, and managed infrastructure. Describing the benefits of modular and event-based systems that enhance resilience and enable rapid innovation.
- Diving into cost-optimization strategies using AWS serverless pay-as-you-go pricing and auto-scaling features.
- Real-world application/case study: how serverless technologies like AWS Lambda, DynamoDB, Aurora, S3, SNS, Event-bus, SQS and Step Functions help streamline warehouse operations and logistics at GymBeam, from inventory management, to real-time order fulfillment with next day delivery.
You may not need a Lambda. From Serverless to Functionless
We all love Lambda. However, developers often overuse it. In this session and true event-driven fashion, we will explore functionless architecture - connecting AWS services directly through native integrations rather than Lambda and SDK. Eliminating the need to bundle and deploy these functions, as well as saving on cost. During the demo, you'll see a functionless application build with AWS CDK, its direct integration to various AWS Services such as API Gateway, EventBridge, S3, DynamoDB, Step Functions, handling business logic and data flow.
Thomas Sabol
Senior cloud solutions architect
Copenhagen, Denmark
Links
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top