[论文] Learning Grounded Reasoning from Human Gaze
研究领域: CV 作者: Gabriel Sarch, Declan Campbell, Nicholas Budny, Kia Ghods, Katerina Fragkiadaki, Jonathan D. Cohen, Brenden M. Lake, Taylor W. Webb 发布时间: 2026-11-…
论文概要
研究领域: CV 作者: Gabriel Sarch, Declan Campbell, Nicholas Budny, Kia Ghods, Katerina Fragkiadaki, Jonathan D. Cohen, Brenden M. Lake, Taylor W. Webb 发布时间: 2026-11-17 arXiv: 2611.23209
中文摘要
人类视觉推理由系统化探索和任务相关视觉证据提取驱动,注视为此过程提供窗口。我们研究如何有效利用人类注视提升 VLM 视觉落地推理。收集 7,109 个多样视觉推理任务的人类注视试验,训练不同形式注视引导的落地推理模型。在图表理解、视觉搜索和计数评估中,使用注视作为空间探索先验始终优于完整注视序列监督。基于发现提出 GGR(注视引导推理器),准确率匹配或超越领先基线,同时实现更高人类对齐和更可靠视觉落地。强化学习进一步提升了任务准确率和对齐质量——尽管未用注视监督。
原文摘要
Human visual reasoning is driven by systematic exploration and extraction of task-relevant visual evidence. Human gaze provides a window into this process, making gaze a useful source of guidance for visually-grounded reasoning in vision-language models (VLMs). However, human gaze also includes noise and redundancies that may transfer poorly to VLMs. In this work, we investigate how human gaze data can be effectively leveraged to improve visual reasoning in VLMs, while maintaining alignment with human behavior. We collect 7,109 human gaze trials from a diverse and challenging set of visual reasoning tasks, and train grounded reasoning models with different forms of gaze guidance. Across evaluations in chart understanding, visual search, and counting, we find that using human gaze as a prio...
*自动采集于 2026-11-19*
#论文 #arXiv #CV #小凯