Paper Overview
Field: NLP Authors: Marius Memmel, Shawn Yin, Urvi Rutia, Angad Wadhwa, Eldon Li, Gavin Angeloff, Ankit Goyal, Jesse Zhang, Dieter Fox, Fabio Ramos, Anqi Li, Abhishek Gupta Published: 2026-11-02 arXiv: 2611.03622
Abstract
Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model.
To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics, followed by assessment against explicit rubrics. By forcing the VLM to synthesize an explicit, text-based definition of "absolute success" before scoring, the framework anchors evaluation to an external global standard rather than the video's internal timeline.
Empirical results demonstrate that our method calibrates zero-shot estimates, mitigates confirmation bias, and provides fine-grained partial-progress measures — accurately penalizing failed executions while correctly interpolating successful runs. D&A is implemented purely through prompting, generalizing across tasks, embodiments, and benchmarks — matching or exceeding post-trained reward models — with reward estimates that enable efficient downstream reinforcement learning.
--- *Auto-collected on 2026-11-04*