Soham Dasgupta

Soham Dasgupta

Deployed Engineer @ LangChain

Utrecht, The Netherlands

Actions

I am a technology enthusiast working at LangChain as a Deployed Engineer, with over 20 years of experience in software programming, designing, and architecture which includes on-prem, cloud-native applications, and Agentic application design.

Area of Expertise

  • Information & Communications Technology
  • Region & Country

Topics

  • java
  • Enterprise Java
  • Java & JVM
  • Core Java / Java SE
  • Enterprise Java / Jakarta EE
  • Java and Server-side
  • Java language
  • Java in the cloud
  • Java user Group Leader
  • Java Concurrency
  • Java EE
  • Java Security
  • Spring
  • Spring Boot
  • Spring Framework
  • Java Performance
  • Spring Cloud
  • Spring Data JPA
  • Spring Security
  • Spring Tools
  • Kubernetes
  • Container and Kubernetes security
  • Azure Kubernetes Services (AKS)
  • CI/CD
  • CI/CD Security
  • CI/CD Pipelines
  • kuberentes community day
  • Automation & CI/CD
  • maven
  • Azure
  • Microsoft Azure
  • Agentic AI
  • AI Agents
  • Agentic AI architecture
  • Agentic AI / Autonomous Agents
  • Generative & Agentic AI
  • LLMs & Agentic AI
  • Autonomous Agents
  • Multi-Agents System
  • LangChain
  • LangSmith

Continuous Evaluation & Monitoring for AI Agents

Agentic/GenAI applications aren’t fundamentally different from traditional software, but they fail differently: a prompt tweak, model upgrade or new tool can quietly degrade answers while every test stays green, because a wrong answer looks just like a right one. Continuous Evaluation (CE) scores every change for quality and safety before it ships, blocking regressions like a failing unit test. Continuous Monitoring (CM) watches quality in production and surfaces the failures no test anticipated. Together they form one loop: what monitoring finds becomes tomorrow’s PR and test.

This session walks that loop end to end with live demos. You’ll learn to design and write evaluators as code, from deterministic checks to LLM-as-judge, gate pull requests and deployments on thresholds and baseline regression checks that tolerate noisy judges, treat hard controls differently from averages, and use red teaming as adversarial evaluation. For monitoring: tracing, online evaluators on live traffic, metrics, dashboards and alerts as code, and turning production failures into new test cases and automated PR fixes. You’ll leave with a tool-agnostic blueprint and five practical rules.

Your pipeline can prove the code compiles and the tests pass, but not that your Agents still gives correct answers. This talk shows how to make answer quality both a build gate and a production signal. Continuous Evaluation blocks the regressions you can name. Continuous Monitoring finds the ones you can’t. The loop between them turns every production failure into a new test.

Many teams are now adding agentic/gen-ai features to systems they already ship with mature CI/CD, and they find that none of their existing gates can see quality. This session fills that gap with engineering practice, not tooling:

evaluators written and versioned like unit tests;
thresholds and baselines that cope with non-deterministic output;
hard controls for the checks that must never fail;
red teaming as adversarial testing;
monitoring built on traces, metrics and alerts as code.
Everything is shown live and end to end: a pull request, a blocked deploy, and a production failure that becomes the next test.

Two contrasting stacks, one assembled from Azure building blocks and one on LangSmith, make the point that the loop matters more than the tool. The core agent code and some evals happen to be in Python, but every pattern maps onto pipelines: evaluations run as tests in the same CI gates, and monitoring builds on OpenTelemetry, which almost every team already use. Attendees leave with a tool-agnostic blueprint, five practical rules and two public repositories to fork.

Repos used:

https://github.com/sohamda/azure-ai-redteam-eval

https://github.com/sohamda/kyb-agent

Context Is a Budget — Spend It Wisely

AI coding assistants are now a daily tool, but most teams use them unstructured, i.e., pasting whole repos, leaving giant chats running for days, and reflexively reaching for the most expensive model. The result is slow responses, blown token or premium-request budgets, and counterintuitively, worse code, because models lose accuracy as their context fills up ("context rot").

This session is a practical guide to using AI assistants with discipline. I'll cover the eight levers that actually move the needle: context engineering, prompt caching, tool design, custom instructions and skills, model routing, output discipline, repo hygiene, and workflow patterns including the Ralph Wiggum loop, auto-compact, and agent handover. Live demos and a token-cost calculator will show the impact in real numbers. You'll leave with a checklist your team can apply this week.

J-Fall 2022 Sessionize Event

November 2022 Ede, The Netherlands

Soham Dasgupta

Deployed Engineer @ LangChain

Utrecht, The Netherlands

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top