← Back to feed

OpenAI published a detailed technical post explaining how it rebuilt its voice infrastructure from the ground up for GPT-Live — the always-on, bidirectional voice layer that ships inside ChatGPT. The disclosure covers every layer of the stack: model inference, context management, media transport, and session startup.

Continuous Audio: No Turn Detection

The previous architecture relied on a turn detector to decide when a user had finished speaking before passing input to the model. GPT-Live eliminates that entirely. Audio streams continuously in both directions — the model listens while it speaks. Turn detection latency disappears.

The consequence is that the voice layer must handle two simultaneous streams reliably at scale, which required a full redesign of the inference path.

Parallel Background Reasoning

Tool calls and deeper reasoning tasks — queries that take time — now run in a separate background model. The foreground model keeps speaking while the background model works. When the result arrives, it folds back into the conversation without any perceptible pause.

This separates the real-time audio delivery problem from the latency-tolerant reasoning problem, letting each be optimized independently.

Go Rewrite: p95 Moved to p50

The media frontend and inference logic were rewritten entirely in Go. The result: the new system’s p95 frame delivery (the tail latency, where problems appear) now matches what the previous system hit at its median (p50). The slowest calls under the new system perform as well as average calls did before.

Long-Call Continuity Without Pausing

Long conversations accumulate context until the model’s window fills. The previous approach paused the conversation to compact or reload context. The new approach uses a parallel handoff: a replacement model is prefilled with the current conversation in the background, and traffic switches over once it is ready. The old model keeps speaking during the transition. Context compaction happens the same way — no pause, no audible gap.

WebRTC WARP: Six Round Trips to One

Session startup for WebRTC typically requires six network round trips before audio can begin flowing. OpenAI’s WebRTC Abridged Roundtrip Protocol, WARP, collapses that to one. A further optimization, Instant Connect, can begin a session from a single UDP packet — eliminating the setup overhead entirely for clients with clean network paths.

The combination makes pickup time — the interval between tapping the microphone button and hearing the model speak — meaningfully shorter, especially on mobile and high-latency connections.

What This Represents

The architecture is purpose-built for scale. Every optimization target — continuous audio, parallel background calls, Go inference, parallel handoffs, WARP — addresses a specific bottleneck that appeared as ChatGPT’s voice user base grew. The p95 → p50 framing on the Go rewrite is the most concrete measure of that: the long tail of bad calls was the main user-visible problem, and the rewrite attacks it directly.

OpenAI has not disclosed GPT-Live’s user count separately from the 800 million ChatGPT weekly user figure, but the architectural choices — aggressive parallelism, single-UDP startup — suggest an infrastructure being hardened for a much larger voice-first user base than currently exists.