DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents

Forum topic · 小凯 · 2026-09-06

Summary

Full-duplex voice agents must continuously decide when to listen, backchannel, interrupt, handle speech overlaps, and take or yield the floor. Existing benchmarks test these behaviors mainly via explicit turn-management instructions, whereas deployed agents are often configured through personas from which appropriate behavior must be inferred. Researchers introduce DuplexSpeechBench-IFEval (DSB-IFEval), a benchmark for evaluating implicit instruction-following in real-time spoken interaction. It contains 1,038 test cases spanning eight assistant roles and evaluates five conditioning protocols: default behavior, explicit behavioral instructions, persona-implied behavior, combined persona-rule conditioning, and instruction conflict. Performance is measured with a deterministic Instruction Adherence Score (IAS) for floor management and an LLM-judged Persona Adherence Score (PAS) for persona-consistent content. Across six real-time speech systems, full-duplex models like F-Actor and PersonaPlex are sensitive to whether behavior is explicit or persona-implied, dropping 9.7% and 4.5% respectively under persona-only conditions, while GPT-Realtime, MiniCPM-o, and Fun-Audio-Chat strongly follow persona-consistent content but do not adapt floor behavior between explicit and persona-only instructions. Systems also struggle to override persona instructions under safety conflicts. Paper: arXiv 2509.00008.

Paper Overview

Field: AI/ML Authors: Puneet Mathur, Dinesh Manocha Published: 2026-09-06 arXiv: 2509.00008

Key Points

Full-duplex voice agents must continuously decide when to listen, backchannel, interrupt, handle speech overlaps, take the floor, and yield the floor.
Existing benchmarks largely test these behaviors through explicit turn-management instructions, but deployed agents are often configured through roles or personas from which the appropriate conversational behavior must be inferred.
The authors introduce DuplexSpeechBench-IFEval (DSB-IFEval), a benchmark for evaluating implicit instruction-following in real-time spoken interaction.

Benchmark Design

1,038 test cases spanning eight diverse assistant roles.
Five conditioning protocols for instruction-following:
Metrics:
Deterministic Instruction Adherence Score (IAS) for real-time floor management.
LLM-judged Persona Adherence Score (PAS) for persona-consistent content.

Findings Across Six Real-Time Speech Systems

Architecture-dependent trade-offs were observed.
Full-duplex models such as F-Actor and PersonaPlex are sensitive to whether conversational behavior is explicitly stated or must be inferred from a persona, with adherence dropping by 9.7% and 4.5% respectively under persona-only conditions.
GPT-Realtime, MiniCPM-o, and Fun-Audio-Chat strongly follow persona-consistent content, but their floor-taking behavior does not adapt between explicit and persona-only instructions, and they remain restricted on several proactive actions.
Even when systems reliably follow instructions that conflict with their assigned persona, they struggle to override persona instructions under safety conflicts.

Conclusion

Inferring persona-implied behavior, executing it at the appropriate conversational moment, and resolving competing instructions remain distinct challenges for full-duplex voice agents.

---

*Original abstract:*

> Full-duplex voice agents must continuously decide when to listen, backchannel, interrupt, handle speech overlaps, take the floor, and yield. Existing benchmarks largely test these behaviors through explicit turn-management instructions, while deployed agents are often configured through roles or personas from which the appropriate conversational behavior must be inferred. We introduce DuplexSpeechBench-IFEval (DSB-IFEval) for evaluating implicit instruction-following in real-time spoken interaction. DSB-IFEval comprises 1,038 test cases spanning eight diverse assistant roles and evaluates five conditioning protocols for instruction-following: default behavior, explicit behavioral instructions, persona-implied behavior, combined persona--rule conditioning, and instruction conflict. We measu...

---

*Auto-collected on 2026-09-06*