Overview
Field: CV Authors: David Yan, Le Xue, Forrest Huang, Qiuyi Zhang, Dustin Tran, Andrew M. Dai, Yinfei Yang Published: 2026-11-05 arXiv: 2611.08485
Abstract
Grounded reasoning, where a model outputs text coordinates of relevant locations in chain-of-thought reasoning, is a common strategy for solving complex visual reasoning problems. However, the benefits and limitations of grounded reasoning are understudied. This work studies where and why grounded reasoning is most useful.
The authors introduce a chain-of-thought analysis method to measure how much a model actually uses its outputted grounding in its thinking trace, and perform a detailed ablation study of a grounded reasoning training recipe.
Key findings
- Grounded reasoning improves counting ability.
- Across a wide range of spatial reasoning and perception benchmarks, grounded reasoning shows no significant gain over baselines without coordinate grounding.
- Using several synthetic tasks, the paper shows grounded reasoning is most useful when a task requires undescribable references—visual concepts that are difficult to describe in language.
- However, many existing visual reasoning benchmarks can be solved using only describable textual references.
- To close this gap, the authors introduce BrickVision, a new benchmark that requires undescribable visual references to solve.
- arXiv: https://arxiv.org/abs/2611.08485