Many AI and agent products are optimized for a five-minute demo: fluent answers, a clean UI, one happy path. Operators running these in production need reliability under messy inputs, clear ownership of mistakes, and workflows that fit how work gets done.

Drawing on experience shipping GenAI infrastructure (RAG, vector search, orchestration) for enterprise customers and AI-assisted workflows in a high-stakes trading environment, this talk contrasts demo metrics (wow factor, clean-query latency) with operator metrics (task completion, exception rate, time-to-recovery, auditability).

Topics: designing for partial data, making confidence and provenance visible, avoiding review-burden features, and choosing when an agent should draft, recommend, or act. Vendor-agnostic, next-sprint focused.