Перейти к основному содержимому

Full-Duplex Voice AI: Why Voice Agents Suddenly Got Real

Начальный

For a decade, talking to a computer meant taking turns with a walkie-talkie that pretended to be a person. On July 8, 2026, OpenAI shipped GPT-Live — a voice model that listens while it talks — and the walkie-talkie era officially started ending. This page explains what actually changed under the hood, why the old voice stack was doomed to feel robotic, and how to judge the whole 2026 voice-agent landscape without the hype.

What you'll learn
  • Understand why the classic STT → LLM → TTS pipeline always felt laggy — it's physics, not polish
  • Know what full-duplex means: one speech-native model that listens and speaks simultaneously
  • Get the verified facts on GPT-Live and the current voice landscape (OpenAI, Google, ElevenLabs, Anthropic, open models)
  • Know when a voice agent is genuinely viable today — and what still breaks

The 200-millisecond problem​

Here's the fact that explains everything else on this page: humans respond to each other in about 0–200 milliseconds. A landmark cross-linguistic study of 10 languages (Stivers et al., PNAS 2009) found response gaps cluster near 0 ms in every culture tested — we routinely start answering before the other person finishes, because our brains predict the end of their turn.

Now compare the classic voice-assistant stack. It was a pipeline of three separate models glued together:

  1. STT (speech-to-text) transcribes your audio into text,
  2. an LLM reads the transcript and writes a reply,
  3. TTS (text-to-speech) turns the reply back into audio.

Each stage must (mostly) finish before the next starts, so their delays stack. Worse, the pipeline has no idea when you've stopped talking — audio has no "send" button — so engineers bolted on a VAD (voice activity detection) silence timer: wait for roughly half a second to a second of silence, then guess the turn is over. That single hack explains both classic failure modes: pause mid-sentence to think and the bot barges in; finish crisply and it still sits there waiting for its silence timer. Add it all up and you get 1–3 seconds of dead air where a human expects ~0–200 ms — an order of magnitude too slow, before the model has even said a word.

And it gets worse: the pipeline is half-duplex, like a walkie-talkie. While the bot is speaking, it is not listening. You can't interrupt ("barge in") without special engineering, the bot can never say "mhmm" while you talk, and any overlap — the most human part of conversation — is simply impossible by construction.

What "full-duplex" actually means​

Full-duplex is a telecom term: both directions transmit at the same time (a phone call), versus half-duplex, where they alternate (a walkie-talkie). Applied to AI voice:

  • The model listens and speaks simultaneously. There is no "your turn / my turn" state machine — input audio streams in continuously while output audio streams out.
  • It's speech-native. One model consumes and produces audio directly, instead of three models passing text between them. No transcription step, no synthesis step, no stacked latency — and no information loss (tone, hesitation, irony, and emotion survive, because they were never flattened into text).
  • Turn-taking becomes a learned behavior, not a timer. Per OpenAI's description of GPT-Live, the model makes interaction decisions "many times per second": whether to speak, keep listening, pause, acknowledge, interrupt, or invoke a tool. The silence-detection hack disappears because the model predicts turn ends the way humans do.
  • Backchannels and barge-in come for free. It can murmur "mhmm" while you talk (a backchannel), stop mid-sentence the instant you cut in (barge-in), or stay silent while you think — all impossible or hacky in a pipeline.

Few people know this: full-duplex wasn't invented by OpenAI in 2026. French lab Kyutai open-sourced Moshi in 2024 — a full-duplex speech model with ~160 ms theoretical / ~200 ms practical latency that models two parallel audio streams (yours and its own) and uses an "Inner Monologue" of time-aligned text tokens to keep its speech linguistically coherent. You can download the weights and run it locally today. What changed this month is that full-duplex went from research demo to the default interface for hundreds of millions of ChatGPT users.

GPT-Live: what OpenAI actually shipped​

Verified against OpenAI's announcement and launch coverage (July 8, 2026):

  • Two models: GPT-Live-1 and GPT-Live-1 mini. The mini replaces Advanced Voice Mode as the ChatGPT Voice default (including free tier); the larger GPT-Live-1 is for paid tiers. TechCrunch reports over 150 million people already use ChatGPT's voice features.
  • True full-duplex architecture. Continuous processing of input while generating output, with speak/listen/pause/interrupt/tool decisions made many times per second. It backchannels ("mhmm", "yeah"), handles rapid back-and-forth, and — notably — can stay quiet and just absorb context until called on.
  • Delegation to a frontier model. For web search, deeper reasoning, or agentic work, GPT-Live hands the task to OpenAI's frontier model (GPT-5.5 at launch) in the background and keeps talking with you while the result comes back. The voice model is the conversational front-end; the heavy thinking happens elsewhere. This "fast talker + slow thinker" split is the architecture pattern to watch.
  • Live translation falls out of the continuous listen-while-speaking design — the model can render your sentence in another language nearly as you say it. (Launch coverage noted accent quality is still uneven in some languages.)
  • No developer API at launch. GPT-Live is a ChatGPT product for now; OpenAI says API access is coming and has a sign-up form. For builders, gpt-realtime on the Realtime API remains the current developer product (see below).
  • Known limits at launch: no video/screen-share in the voice session, uneven quality outside major languages, and OpenAI says it is monitoring emotional-reliance effects.

The voice landscape, verified (July 2026)​

PlayerWhat existsFull-duplex?Notes
OpenAI — GPT-LiveChatGPT Voice (consumer)Yes — speech-nativeDelegates hard tasks to a frontier model mid-conversation; no API yet
OpenAI — Realtime API (gpt-realtime)Developer API, GASpeech-to-speech, single modelProduction voice agents: SIP phone calling, remote MCP servers, image input
Google — Gemini Live APIDeveloper API (AI Studio / Vertex, GA)Native-audio, streamingBarge-in, "proactive audio" (speaks only when relevant), affective dialog, tool use + Google Search
ElevenLabs — AgentsAgent platform (launched Mar 2026)Orchestrated stack with proprietary turn-taking modelTTS/STT + turn-taking + tool calls; 70+ languages; sub-500 ms first-turn claims; phone/web/app channels
Anthropic — ClaudeVoice mode in the Claude apps; push-to-talk /voice in Claude Code (Mar 2026, multilingual out of beta Jun 2026)No — turn-basedSpeak, get spoken replies with a saved transcript. No speech-native full-duplex model announced as of the verify date — don't let anyone tell you otherwise
Kyutai — MoshiOpen weights + code (GitHub, Hugging Face)Yes — the open-source proof~160–200 ms latency, dual-stream audio, "Inner Monologue"; runs locally

Two takeaways from that table most coverage misses: (1) "voice agent" today means two different architectures — genuinely speech-native full-duplex models (GPT-Live, Moshi, Gemini's native audio) versus very fast, well-orchestrated pipelines with a learned turn-taking model on top (ElevenLabs Agents). Both can feel good; only the first can overlap speech. (2) The open-source option is real: Moshi proves you can run full-duplex on your own hardware, which matters if your use case can't ship audio to a cloud (see Choosing a Model for that decision framework).

How a full-duplex conversation actually flows​

Guided walkthrough1 of 5
  1. There is no record-then-send. Your microphone audio is encoded into tokens frame by frame (Moshi's codec uses 80 ms frames) and fed to the model as it arrives — even while the model is mid-sentence.

When voice agents are now viable — and what still breaks​

Now genuinely viable:

  • Customer support and phone workflows. Sub-second turn-taking plus barge-in removes the two biggest complaint drivers. The Realtime API's SIP support and ElevenLabs Agents both target exactly this.
  • Hands-free and eyes-busy use. Driving, cooking, field work, accessibility — the interaction finally keeps up with speech (voice mode on the Claude apps already covers the capture-and-transcribe version of this).
  • Live translation and language practice. Listen-while-speaking makes near-simultaneous interpretation and natural conversation drills possible for the first time.
  • Voice as an agent front-end. The delegation pattern — chat with a fast voice model while a slow frontier model does the work — is OpenAI's stated bet for managing "long-running agentic work" by voice.

Still breaks:

  • Hallucinated audio. Speech-native models can hallucinate in sound, not just facts: wrong-language drift, mangled names and numbers, or off accents (GPT-Live's launch translation demo drew exactly this criticism). Never trust a spoken number you didn't confirm.
  • Noisy environments and crosstalk. Always-open microphones hear everything — side conversations, TV, a second speaker. Full-duplex makes the model more exposed to ambient audio, not less.
  • Safety of real-time actions. A model that acts at conversational speed can act on a misheard sentence at conversational speed. Any voice agent that touches money, messages, or deletions needs explicit spoken confirmation gates and a bias toward read-only defaults — the same rules as any agent (see Foundations), but with a lower-fidelity input channel.
  • Emotional reliance. A system that backchannels, hesitates, and never tires of you is engineered to feel like a friend. OpenAI itself flags monitoring for this. Design (and use) accordingly.

System-prompt skeleton for a voice agent (works on speech-native models and pipelines)

You are a voice assistant for {company}. You are SPEAKING, not writing.

Style:
- Short sentences. One idea per sentence. No lists, no markdown, no URLs read aloud.
- If the user interrupts, stop immediately and address what they said.
- If the user pauses mid-thought, stay silent. Do not fill silence.

Safety:
- Before ANY action that sends, buys, deletes, or changes something:
say back exactly what you will do and wait for a clear spoken "yes".
- Repeat numbers, names, and addresses back for confirmation — always.
- If audio is unclear or noisy, say what you think you heard and ask.
- If asked for something outside {scope}, say so and offer a human handoff.

Check yourself

0/4
  1. Why did the classic STT → LLM → TTS stack always feel laggy?
  2. What can a full-duplex model do that a half-duplex pipeline cannot, by construction?
  3. How does GPT-Live handle a question that needs web search or deep reasoning?
  4. Which of these was true BEFORE GPT-Live launched?
Voice-AI vocabulary
Нажмите Enter или пробел, чтобы перевернуть карточку. Используйте стрелки влево и вправо для перехода между карточками.Показан термин.
1 / 8
Key takeaways
  • The old stack wasn't badly built — it was structurally too slow: stacked STT/LLM/TTS latency plus a silence timer, versus the ~0–200 ms humans expect
  • Full-duplex = one speech-native model that listens while speaking; barge-in, backchannels, and deliberate silence become learned behaviors, not hacks
  • GPT-Live (July 8, 2026) mainstreams full-duplex in ChatGPT and pioneers delegation: fast voice model up front, frontier model reasoning in the background
  • The landscape splits in two: speech-native full-duplex (GPT-Live, Gemini native audio, Moshi — open source since 2024) vs fast orchestrated pipelines with learned turn-taking (ElevenLabs Agents); Claude's voice remains turn-based
  • Voice agents are now viable for support, hands-free, and translation — but audio hallucinations, noisy rooms, and real-time action safety still demand confirmation gates

Next​

Sources & further reading​