Session
94% Accuracy, 40% Adoption: Why We Rebuilt Our Agent Around Trust, Not Intelligence
Our CI/CD agent diagnosed flaky tests with 94% accuracy. Engineers used it 40% of the time. The answer wasn't "it's wrong"—it was "I don't trust it enough to act on it."
This talk shares how we rebuilt our agent around trust—and why MCP's design patterns solve problems we spent months figuring out. (github.com/chebroluharika/agentic-ai-cicd-bot)
THE GAP: Our assistant parsed 10,000+ line logs across 500+ daily builds. Engineers still debugged manually because they couldn't see WHY.
TRUST PATTERNS (now in our MCP tools):
• Reasoning as Output: Not "retry" but "Found 3 similar failures, all resolved by retry. Confidence: 89%"
• Confidence Calibration: 90%+ = act. 60-90% = verify. <60% = guessing.
• Graceful Degradation: Uncertain? Say "I don't know" and suggest what to check.
• Correction Loops: One-click feedback. Adoption +34% with engineer ownership.
MCP ENABLES TRUST BY DEFAULT:
Structured responses force transparency. Schema validation prevents hallucinations. These patterns are now MCP-native in our toolkit.
RESULTS:
• Adoption: 40% → 87%
• Resolution: 23 min → 5 min
• NPS: -12 → +67
Harika Chebrolu
Engineering Agentic AI Systems at Enterprise Scale
Bengaluru, India
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top