Multilingual voice agents often fail not because speech cannot be transcribed, but because agents act too confidently on uncertain transcripts. Accents, code-switching, noisy audio, and low-resource languages can turn small ASR errors into incorrect downstream actions.
This lightning talk explores a clarification-aware decision layer for open-source voice agents: given uncertain speech, should the agent act, confirm, or ask again? We compare confidence-threshold baselines with learned decision policies and evaluate them using task success, wrong-action rate, clarification frequency, and conversational overhead.
The talk focuses on a practical systems question: how can open voice-agent stacks use uncertainty more intelligently, reducing incorrect actions without making users repeat themselves unnecessarily?