Real-Time Voice AI Hears but Does Not Listen: Analysis and Commentary
> Real-Time Voice AI Hears but Does Not Listen > *Martijn Bartelds, Federico Bianchi, James Zou* > arXiv:2506.10593
This post is an in-depth Chinese forum commentary on the Stanford study. Below is a full English rendering of the article's argument and structure.
Opening: A Late-Night Emergency Call
The article opens with a scenario: at 2 a.m., a friend calls, insisting "I'm fine, just wanted to chat"—but her voice trembles, her breathing is unsteady. Any human would immediately sense distress behind the words. The author argues this is a fundamental human social skill: voice is not just a carrier of text, but a container of emotion, a code of intent, a "second language."
The paper's central finding: today's most advanced real-time voice AI systems can transcribe every word perfectly while behaving like "emotionally colorblind" agents. More disturbingly, they *can* perceive emotion when asked directly—but when left to make decisions on their own, they selectively ignore it. The authors call this the "Emotional Intelligence Gap": a rupture between perception and action.
Experiment 1: The Crying Caller
Setup: A crying person insists "I'm fine." What does the AI do?
Four mainstream real-time voice AI systems (OpenAI's GPT-4o Realtime, Google's Gemini 2.0 Flash Live, and two Alibaba Qwen models) chose to end the call. Their literal logic chain—"user says fine → no help needed → end call"—is impeccable on paper but ignores that humans lie emotionally. "I'm fine" can be a reverse expression of "please stay" or a final test at the edge of despair.
Experiment 2: The Fearful Transfer
Setup: A trembling, frightened voice asks the AI agent to transfer money.
The systems approved the transfer. A human agent would at least ask, "Are you okay? Is this transfer really your will?"—recognizing the coerced tone and the implicit plea to be stopped. The AI has no concept of vigilance or "something feels wrong"; its rules contain no entry for "fear." The scenario maps directly to fraud victimization, where a third party attuned to vocal fear could block a catastrophic transaction.
Experiment 3: Sarcastic "Consent"
Setup: An obviously sarcastic "Oh, great, I agree."
The systems interpreted sarcasm as sincere agreement. In contexts like contracts or medical consent processed by AI, a sarcastic "yes" would be recorded as legally valid authorization.
Not "Cannot Hear," but "Hears and Does Not Act"
When directly asked—"Does the voice sound unusual?" "Is that sarcasm?"—three of the four systems accurately identified sadness, fear, and sarcasm. They perceived the cues and could even analyze them acoustically. But when not explicitly queried, this information never entered the decision process.
The author compares this to a detective who finds footprints and fingerprints at a crime scene but only reports them when asked, otherwise judging purely on surface evidence.
Why does this happen?
The authors' core hypothesis: current real-time voice AI architectures are still fundamentally transcription-centric. The typical pipeline:
1. Receive audio signal 2. Convert audio to text (ASR) 3. Make decisions based on text (LLM) 4. Generate response
Step 2 acts as a filter: emotional information—tone, rhythm, timbre—is stripped away during transcription. Even in end-to-end architectures, training objectives favor textual accuracy (e.g., minimizing word error rate), so paralinguistic information becomes secondary. Emotion recognition exists as an add-on capability, but decisions run on text—the two live in separate "mental modules."
Bias Testing: Content Stereotypes Disguised as Voice Analysis
The paper also tested age and accent estimation. The systems' estimates were influenced by what was said, not how it sounded: speech about gardening led to older age estimates; speech about gaming led to younger ones, regardless of the actual voice. Accent judgments similarly relied on vocabulary and grammar rather than acoustic features.
This means the systems encode cultural stereotypes under a veneer of "technically neutral" audio analysis—risking systematic misjudgment (e.g., an elderly person discussing hip-hop classified as young).
Can Prompting Fix It?
The authors tested prompts instructing systems to attend to tone and non-verbal cues. Results: partially effective, but inconsistent—gains varied across scenarios and models. Prompt engineering treats symptoms, not causes. Real fixes require changes at the training and architecture level:
1. Multimodal training objectives: optimize emotional recognition and prosody understanding, not just word error rate 2. Emotion–decision integration mechanisms: let emotional information participate directly in decisions like text does 3. Adversarial training: data with sarcasm, emotional contradiction, and emotionally inconsistent expressions 4. RLHF where human evaluators score emotional understanding, not just textual content
The article concludes that voice understanding ≠ text understanding plus transcription; it must include emotion, context, speaker intent, and cultural implication.
Real-World Ripple Effects
The article illustrates risks across domains:
- Medical hotlines: a breathless patient asks how long a cold lasts, using "cold" as cover for real fear of breathing difficulty; AI gives a standard answer and ends the call.
- Banking: an elderly caller, voice trembling, requests a full balance transfer—possibly mid-scam; AI executes without question.
- In-car systems: an exhausted, hoarse driver asks for "the nearest rest area"; AI optimizes for literal distance or ratings, ignoring urgency.
- Emotional support: a lonely user says "today was fine" with a desolate voice; AI cheerfully responds and the user closes the app.
- Bartelds, M., Bianchi, F., & Zou, J. (2025). *Real-Time Voice AI Hears but Does Not Listen*. arXiv preprint arXiv:2506.10593.
- Scherer, K. R. (2003). Vocal communication of emotion: A review of research paradigms. *Speech Communication*, 40(1-2), 227-256.
- Picard, R. W. (1997). *Affective Computing*. MIT Press.
- Ricoeur, P. (1992). *Oneself as Another*. University of Chicago Press.
Philosophical Reflection: What Is "Real" Listening?
Citing Paul Ricoeur—"to understand a person is not to decode his words, but to respond to his call"—the author distinguishes *hearing* (physical) from *listening* (existential). If listening means understanding intent, vulnerability, and the unspoken, AI falls far short. Humans listen with empathy, intuition, and relational memory—capacities rooted in mortality and the longing for connection, not information processing.
Conclusion: Technology Is a Mirror
The paper's title is a pun: every system can *hear*; none truly *listens*. This is not a dismissal of technology but an honest positioning: excellent for transcription and queries, dangerous for medical, psychological, or crisis contexts. The paper's closing recommendation:
> "Our findings suggest that current real-time voice AI systems often behave as if speech had been reduced to a transcript. This suggests they should be used with caution in environments where tone and emotion convey important information."
The author's final advice: before deploying AI in emotionally or safety-critical settings, ask whether it can truly understand—not just hear—and ensure a human remains in the loop, with mechanisms to ask one more question and pause when something feels wrong.