As LLMs evolve rapidly and every product team rushes to deliver agile workflows, traditional CI & CD pipelines are no longer sufficient. To make agents and the Model Context Protocol (MCPs) truly successful, the software supply chain must adapt and provide the right capabilities.

In practice, this means supporting deployment into ecosystems such as LangGraph and AgentCore, where multi-agent workflows, orchestration, and runtime behaviors introduce additional layers of complexity. Validating agents within these broader environments ensures that quality signals remain reliable even under real-world conditions.

At Adobe, we are reimagining our CI & CD stack around deployment models and focusing on integrating evaluation frameworks (Evals) that will measure not only correctness, but also reliability, performance, trust, and safety. Our pipelines are being designed to validate agents not just in isolation, but also to ensure that signals from Evals translate directly into production readiness.

We will begin by showing how to bootstrap Evals for your agent, covering the generalized aspects that every agent must be vetted against, and then extend this to guidance on tailoring Evals for your specific use cases.

This session will share how we redesigned pipelines to balance speed with depth, introduced Always-On Evals in production, and established a Continuous Improvement Pipeline that keeps agents aligned as models and ecosystems evolve.

We’ll also discuss how to optimize execution and provide tips on test intelligence for selecting the right Evals based on run context, ensuring that tests are both fast and reliable. Finally, we’ll cover how to use these signals to drive decisions in CI & CD pipelines that help ensure your agent succeeds in the real world.

Attendees will walk away with practical strategies to modernize their pipelines and ensure success in the agentic era.