Full-Duplex Bench v3: Gemini Live 3.1 Is Fastest and Goes Silent 22% of the Time
The voice AI leaderboards measure latency and accuracy on clean, well-formed speech. Full-Duplex-Bench v3 measures something else: what happens when users hesitate, stop mid-sentence, and change their mind.
The paper — arXiv:2604.04847, from Lin, Chen, Chen, and Lee — runs 100 scenarios recorded by 12 speakers in uncontrolled everyday environments across six systems: GPT-Realtime, Gemini Live 2.5, Gemini Live 3.1, Grok, Ultravox v0.7, and a cascaded pipeline (Whisper + GPT-4o + OpenAI TTS). All tool calls route to local mocks at zero latency to eliminate network confounds. The comparison is clean.
The Numbers That Matter
Gemini Live 3.1
- Average latency: 4.25 seconds (fastest in the benchmark)
- Turn-take rate: 78%
- Meaning: 22 of 100 scenarios produced no spoken response
The critical detail is what happened in those 22 silent cases. 86% of them still executed tool calls correctly — the model identified what to do, called the right APIs with the right parameters, and never told the user anything. Failures of expression, not reasoning.
GPT-Realtime
- Average latency: 9.20 seconds (2.3x slower than Gemini Live 3.1)
- Most accurate in the benchmark
- Turn-take rate: effectively 100%
- Still fails on over 40% of mid-sentence correction scenarios
Cascaded pipeline (Whisper + GPT-4o + TTS)
- Highest reliability for response delivery
- Worst latency
- No architectural advantage on disfluency handling
Ultravox v0.7
- 88% filler rate — explicit attempt to mask tool-call latency through conversational fillers
- Strategy backfires when fillers overlap with user speech
The Mid-Sentence Problem
No system in the benchmark handles intent corrections well. The test scenario: a user asks about flights to Rome, then corrects to Milan mid-sentence. Gemini Live 3.1’s architecture committed to the Rome tool call before the correction arrived — tool-call latency was negative 2.27 seconds, meaning the API was invoked over two seconds before the user finished speaking. The destination passed was Rome. Speed broke correctness.
GPT-Realtime got this scenario right. It waited, caught the correction, and searched for Milan. At 9.2 seconds versus Gemini’s 3.92 seconds on clean inputs, that patience cost nearly 2.5x the response time. On production voice agents, where users expect sub-5-second responses, that tradeoff is real.
What This Means for Voice Agent Selection
The practical split: Gemini Live 3.1 is appropriate for use cases where a missed response is recoverable and fast interaction feels premium — casual queries, ambient assistants, contexts where the user will naturally re-ask. It is not appropriate for high-stakes single-turn tasks — healthcare intake, financial transactions, booking flows — where a silent success (tool called correctly, nothing said) represents a failure.
GPT-Realtime’s accuracy advantage comes at a latency cost that is non-trivial in contexts where response speed determines perceived intelligence. The 9.2-second average means building filler strategies is not optional — it is required infrastructure for any production deployment.
Ultravox’s filler approach is mechanically reasonable but poorly calibrated in the benchmark: 88% filler rate with no reliable end-of-turn detection generates overlapping speech rather than masked latency.
The Metric Gap
FDB-v3’s lasting contribution is methodological. Accuracy and latency, the two metrics voice AI leaderboards have optimized for, are too coarse to capture production failure modes. Turn-take rate, interruption handling, silent-success detection, and mid-sentence state rollback are the axes that determine whether a voice agent works in real environments. None of them appear on current public leaderboards.
Teams shipping voice agents into healthcare, finance, or customer service should run their own disfluency eval before selecting a provider. The latency number on the leaderboard is not the latency number users experience.
arXiv:2604.04847 — Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency. Demo: daniellin94144.github.io/FDB-v3-demo/