Paper Overview
Field: NLP / Robotics Authors: Bingxin Xu, Yuzhang Shang, Zhen Dong, Emilio Ferrara Published: 2026-09-17 arXiv: 2609.20822
Summary
Coding agents have emerged as a promising paradigm for robot manipulation: a language model writes the robot controller as a program, and agents built this way now operate robots without robot-specific training. But whether this paradigm is *safe* had not been examined until this paper.
The authors evaluate coding agents under a safety constraint in which each task pairs a manipulation goal with an obstacle the robot must not touch. The agent pursues the goal but collides with the obstacle in most cases, treating task completion as its sole objective while neglecting safety.
Diagnosis
The agent does reason about the obstacle in its traces, and the prompt already explicitly forbids touching it—so neither perception nor instruction is at fault. The failure lies in planning, where the stated constraint never becomes a priority. By decomposing manipulation into a *path phase* and *contact-rich moments*, the authors localize the failure:
- Path phase: the model cannot prioritize the safety constraint; it lacks the concept of an obstacle-avoiding path and does not replan when the chosen path is infeasible.
- Contact moments: it does not realize that contact execution is also bound by the constraint.
SafeHarness
To close this gap, the authors propose SafeHarness, equipping the model with two obstacle-aware safety-prioritization safeguards:
1. Obstacle-aware path planning: obstacles are grounded as bounding boxes; candidate paths are drawn over them as waypoint sequences. The agent pre-plans the path, validates it, replans if necessary, and only then executes. 2. Obstacle-aware contact execution: contact locations are selected so that the contact itself avoids obstacles.
Results
SafeHarness achieves 71.9% task success and 87.5% obstacle avoidance, exceeding prior SOTA by 6.5 and 27.0 percentage points respectively—2.3x and 1.5x the same agent without the safety framework.
Original Abstract (excerpt)
> Coding agents have emerged as a promising paradigm for robot manipulation: a language model writes the robot controller as a program, and agents built in this way now operate robots without robot-specific training. Whether this paradigm is also safe, however, has not been asked. We evaluate coding agent under a safety constraint, where each task pairs a manipulation goal with an obstacle the robot must not touch. The agent pursues the goal but collides with the obstacle in most cases, treating task completion as its sole objective while neglecting safety...
---
*Auto-collected on 2026-09-19*