Interaction Models Deep Dive: Mira Murati's Full-Duplex Revolution — AI Learns to Listen and Speak Simultaneously

Forum topic · 小凯 · 2026-05-17

Summary

Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, has introduced Interaction Models — a new AI architecture designed for full-duplex, real-time multimodal interaction. Unlike conventional voice assistants that operate in half-duplex, turn-based mode using voice activity detection, Interaction Models process the world in 200ms micro-turns, simultaneously consuming audio, video, and text streams while generating output. The flagship TML-Interaction-Small model is a 276B-parameter MoE with 12B active parameters, achieving 0.40s turn latency. A dual-model system splits work between a real-time Interaction Model and an asynchronous Background Model for deep reasoning, tool calls, and search. Technical innovations include encoder-free early fusion of raw signals, streaming session inference (open-sourced to SGLang), and bitwise trainer-sampler alignment. On benchmarks like FD-bench v1.5, it scores 77.8 versus 47.8 for GPT-Realtime-2.0, and leads on novel proactive-speech tests where existing models score near zero. The post also covers safety measures, current limitations, and implications for AI interaction paradigms.

Overview

This analysis examines Thinking Machines Lab's Interaction Models, announced by Mira Murati's team (May 11, 2026). Core claim: interaction is not an add-on feature but a native capability of the model. The flagship TML-Interaction-Small is a 276B-parameter MoE (12B active) with 0.40s turn latency.

Key points

The problem: Current voice assistants (ChatGPT Voice, Gemini Live, Siri) are half-duplex, turn-based systems using VAD — they cannot be interrupted, cannot listen while speaking, and freeze perception during generation.
Full-duplex via micro-turns: The Interaction Model slices time into 200ms micro-turns. Each turn receives audio + video frames, and the model simultaneously thinks and produces text/audio output. There are no turn boundaries — all streams flow concurrently.

Dual-model architecture

Interaction Model (real-time): Always online, handles conversation flow, interruptions, and instant responses.
Background Model (async): Handles deep reasoning, tool calls, web search; results stream back and are woven into conversation by the Interaction Model.
Users never wait for background completion — the Interaction Model keeps conversing meanwhile.

Technical details

Encoder-free early fusion: Raw dMel audio spectrograms and 40×40 video patches feed directly into the transformer; no separate ASR/TTS modules. Audio decoding uses a flow-matching head (non-autoregressive). End-to-end joint training reduces cascading errors and latency.
Streaming Sessions: Clients send HTTP requests every 200ms; the server appends chunks to persistent GPU memory sequences, avoiding reallocation overhead. Open-sourced to SGLang.
MoE kernel optimization: Replaces standard grouped GEMM with a gather+gemv strategy suited to small-batch, low-latency serving.
Trainer-sampler alignment: Bitwise-aligned, batch-invariant kernels ensure identical outputs across parallelism strategies; consistent Split-KV partitioning between prefill and decode.

Capabilities and benchmarks

Abilities existing systems lack: simultaneous speech (real-time translation), visual proactivity (e.g., "count my push-ups"), time awareness ("remind me to breathe every 4 seconds"), interruption handling, and searching while listening.

New benchmarks introduced by TML:

| Benchmark | Score | |---|---| | TimeSpeak (proactive timed speech) | 64.7% | | CueSpeak (instant error correction) | 81.7% | | RepCount-A (visual counting) | 35.4% | | Charades (temporal action localization) | mIoU 32.4 |

Existing models (including GPT Realtime-2.0) score near zero on proactive speech.

Head-to-head comparison:

| Benchmark | TML-Interaction-Small | GPT-Realtime-2.0 (minimal) | GPT-Realtime-1.5 | Gemini-3.1-flash-live | |---|---|---|---|---| | FD-bench v1.5 (interaction quality) | 77.8 | 47.8 | 48.3 | 54.3 | | FD-bench v1 (turn latency) | 0.40s | 1.18s | 0.59s | 0.57s | | Audio MultiChallenge APR | 43.4% | 37.6% | 34.7% | 26.8% |

Notably, TML-Interaction-Small is an instant model (no extended reasoning) yet beats GPT-Realtime-2.0's thinking mode on interaction quality.

Safety

Spoken refusals generated via TTS so refusals sound natural
Long-horizon robustness via automated red-teaming with multi-turn adversarial data
Harmbench refusal rate: 99.0%; text/voice refusal behavior alignment

Limitations

1. Long sessions: 200ms chunks accumulate context quickly; long-session context management is unsolved 2. Network sensitivity: dropped frames collapse the experience 3. Scale: 276B MoE is the "Small" tier; larger models are too slow for real-time serving 4. Background agent collaboration is "just beginning"

Conclusion

The Interaction Model is not just a faster voice assistant — it is a paradigm shift from turn-based to real-time streaming, from half-duplex to full-duplex, from "awaiting input" to "continuous presence." Rather than building a better ChatGPT, Murati's team rebuilt the underlying protocol for human-AI collaboration.

Sources:

Thinking Machines Lab blog: https://thinkingmachines.ai/blog/interaction-models/