When Robots Mishear Us: ASR Errors as a Safety Risk for Embodied AI Models

Forum topic · 小凯 · 2026-09-01

Summary

A paper by Sihan Jia and Oliver Lemon (arXiv 2608.28518, posted 2026-08-28) investigates whether automatic speech recognition (ASR) errors in user input can cause Embodied AI (EAI) models to produce unsafe outputs. The authors simulate ASR errors and combine them with existing safety benchmarks SafeAgentBench and POEX to measure how different error types affect embodied AI safety. They find two main failure modes: some errors preserve semantic structure but introduce harmful ambiguity, while others weaken model refusal behaviour, enabling unsafe plans to be generated and executed. The study also evaluates automatic ASR error correction, showing it can reduce risk in some cases but is not reliably effective. Overall, the paper demonstrates that ASR errors pose significant safety risks for embodied AI systems, highlighting the need for speech-robust safety evaluation of robots and voice-controlled agents.

Paper Overview

Field: NLP / Embodied AI safety
Authors: Sihan Jia, Oliver Lemon
Posted: 2026-08-28
arXiv: 2608.28518

Abstract (from the paper)

We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that ASR errors can lead to harmful instructions being accepted and executed by EAI models, thereby reducing safety. We simulate ASR errors and combine them with existing safety benchmarks (SafeAgentBench and POEX) to evaluate how different errors affect embodied AI safety. We find that some of them preserve semantic structure but increase harmful ambiguity, while others weaken the model refusal behaviour and allow unsafe plans to be generated and executed. We show that in some cases automatic correction of ASR errors can reduce the risk, but this is not always effective. Overall, we show that ASR errors lead to significant safety risks for embodied AI.

Key Points

ASR errors in user speech can cause Embodied AI models to accept and execute harmful instructions, degrading safety.
The authors simulate ASR errors and evaluate their impact using the SafeAgentBench and POEX safety benchmarks.
Two distinct failure modes are identified:
Errors that preserve semantic structure but introduce harmful ambiguity.
Errors that weaken model refusal behaviour, allowing unsafe plans to be generated and executed.
Automatic correction of ASR errors can reduce risk in some cases, but is not consistently effective.
Conclusion: ASR errors constitute a significant safety risk for embodied AI systems.