RobormBench: Same Robot Trajectories Get Contradictory Rewards from VLMs Under Paraphrased Instructions

Forum topic · 小凯 · 2026-09-08

Summary

A new paper (arXiv:2609.05401) reveals a critical fragility in vision-language models (VLMs) used as reward functions for robotic learning. The authors show that current VLM reward models violate paraphrase invariance: the same robot trajectory should receive the same reward under semantically equivalent goal descriptions, but simply rewording the instruction can substantially change predicted progress scores and even flip identical robot behavior between failure and success. To quantify this failure mode, the team introduces RobormBench, a benchmark containing 2,390 real-robot trajectories with ground-truth progress labels and 21,673 verified paraphrases spanning lexical, syntactic, and action-goal rewrites. Experiments across both proprietary and open-source VLMs demonstrate that paraphrase-induced reward instability is widespread and severe, and grows worse under more diverse rewrites. The work highlights a fundamental robustness gap that must be addressed before VLM-based reward models can be reliably deployed for robot policy training and evaluation.

Paper Overview

Field: NLP / Robotics Authors: Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Yoonjun Cho, Dongjae Jeon, Bumjun Kim, Jean Oh, Youngjae Yu, Albert No Published: 2026-09-04 arXiv: 2609.05401

Abstract (translated from source)

Vision-language models are increasingly used as reward functions for robotic learning, but this role requires paraphrase invariance: the same trajectory should receive the same reward under semantically equivalent goal descriptions. The authors show that current VLM reward models often violate this property.

Key Findings

Paraphrase fragility: Rewording the instruction alone can substantially change predicted progress scores, and can even flip identical robot behavior between failure and success.
RobormBench benchmark: A benchmark with 2,390 real-robot trajectories, ground-truth progress labels, and 21,673 verified paraphrases spanning lexical, syntactic, and action-goal rewrites.
Widespread instability: Across proprietary and open-source VLMs, paraphrase-induced instability is widespread and severe, and grows under more diverse paraphrases.