Inside OpenAI’s GPT-Live Architecture: How Full-Duplex Realtime Voice Works

Ditching Turn Detectors for Full-Duplex Speech

For years, conversational AI relied on an awkward compromise known as turn-taking. In early cascaded architectures, speech-to-text transcription, large language model inference, and text-to-speech synthesis ran strictly in sequence. Even when speech-to-speech models emerged, systems still depended on external "turn detectors"—lightweight helper models designed to guess when a human speaker had finished talking.

This design created an inescapable engineering dilemma. If the turn detector triggered too aggressively, it interrupted users before they could finish their sentence. If it acted too conservatively, it introduced unnatural, multi-second silences that ruined the flow of real-time dialogue. In an engineering post published by OpenAI engineers Justin Uberti and Zahan Malkani, the company detailed how its third-generation voice system, GPT-Live, completely removes turn detectors from the audio loop.

Instead of guessing turn boundaries, GPT-Live uses a full-duplex speech model capable of listening and speaking at the exact same time. The neural network continuously processes incoming audio frames while simultaneously generating speech outputs, deciding several times per second whether to continue listening, speak, pause, or yield the turn if a user speaks over it.

The Decoupled Architecture: Fast Path vs. Slow Path

Maintaining uninterrupted, low-latency audio delivery requires strict separation between media transport and heavy compute tasks. OpenAI accomplished this by decoupling the architecture into two distinct operational layers linked across an asynchronous RPC boundary:

• The Audio Fast Path: Live microphone audio streams continuously between the client application and the voice model over a dedicated, low-latency connection. This path is kept lean, highly predictable, and strictly focused on generating real-time audio tokens.
• The Asynchronous Delegation Path: Heavy tasks—such as executing code, conducting live web searches, or performing multi-step reasoning—are offloaded to background models such as GPT-5.5 without stalling the voice loop.

When a user asks a complex question requiring live information retrieval, GPT-Live can immediately respond with natural conversational fillers or initial thoughts on the fast path, while background workers run web searches asynchronously. Once the background model completes its reasoning, the results are injected back into the voice session without dropping audio frames or causing jitter.

To further enforce stability on the fast path, OpenAI's engineering team completely rewrote the media frontend and inference core, migrating from Python’s asyncio to Go. The performance impact was immediate: the new Go-based system's 95th percentile (p95) frame delivery latency matched the 50th percentile (p50) latency of the previous Python infrastructure.

Hot-Swapping Models and Protocol Hacks (WARP)

Long-running voice calls present significant memory and state management challenges. As a conversation progresses, accumulating tokens fill the model's context window. Periodically compacting this context reduces memory overhead, but compaction invalidates the key-value (KV) cache, requiring an expensive prefill step that would normally freeze speech output.

To eliminate these pauses, OpenAI introduced a parallel model handoff mechanism. When context compaction or model maintenance is required, the system warms a secondary model instance in the background, prefills it with the compacted context, and runs both instances in parallel. Once the secondary instance is synchronized, the media pipeline hot-swaps to the new instance without losing a single frame of audio.

At the network layer, OpenAI built upon WebRTC, the industry-standard transport protocol for real-time video and voice communications. WebRTC excels at handling packet loss, clock drift, and network jitter by dynamically stretching audio packets or briefly accelerating playback. However, standard WebRTC connections require up to six network round trips to establish a secure session.

To slash initialization delay, OpenAI developed the WebRTC Abridged Roundtrip Protocol (WARP). WARP collapses the connection sequence down to a single round trip by piggybacking DTLS handshakes over Interactive Connectivity Establishment (ICE), leveraging DTLS 1.3, and pre-negotiating data channels. Paired with an "Instant Connect" feature, WARP allows ChatGPT Voice sessions to launch almost instantaneously.

Voice AI Ecosystem and Pricing Comparison

As competition intensifies across frontier labs, understanding the pricing and capabilities of different AI voice and language models is crucial for developers and enterprise teams. Below is an overview of key AI models operating across the market in August 2026:

For more detailed latency and model evaluations across cloud and local hardware, check out our benchmark tests in the AI Arena benchmark section.

European Implications: AI Act Enforcement and Data Transit

The rollout of full-duplex voice technology comes at a pivotal moment for European AI governance. On August 2, 2026, legal enforcement under the EU AI Act took full effect across all 27 member states, granting formal auditing and enforcement powers to the EU AI Office and national market surveillance authorities.

For real-time voice streaming architectures like GPT-Live, two regulatory requirements take center stage:

1. Synthetic Content Transparency (Article 50): Under Article 50 of the AI Act, providers offering generative AI systems that output audio or synthetic media must ensure output is marked in a machine-readable format and clearly identified as artificially generated. In full-duplex systems, audio watermarking must be baked directly into the low-latency encoding pipeline without degrading packet transmission times.
2. GDPR and Edge Data Processing: Realtime WebRTC streaming requires edge servers positioned close to end-users to maintain sub-second responsiveness. OpenAI’s deployment across European edge points ensures that voice data transit complies with strict EU cross-border data transfer regulations while shielding users in France, Germany, and neighboring regions from transatlantic network latency.

As OpenAI continues expanding GPT-Live capabilities into desktop computer control and agent coordination, European enterprises will be watching closely to see how low-latency voice interaction balances compliance with cutting-edge performance.