[论文] VideoLVR: Latent Visual Reasoning over Grounded Video Evidence

论文概要

研究领域: CV / 视频推理 作者: Shilong Li, Xingyuan Bu, Hao Shi, Yun Xing, Weixin Feng, Qiangpeng Yang, Shilei Wen 发布时间: 2026-10-27 arXiv: 2610.37613

中文摘要

思维链推理显著推进了多模态大语言模型（MLLM）的视频理解能力。然而，基于语言的推理轨迹无法提供可靠的机制让模型回溯相关视觉证据，导致视觉遗忘并削弱了对视频内容的推理能力。现有方法维持视觉定位主要有两种方式：使用文本坐标指定证据位置（仅指定符号位置而非视觉内容本身），或将裁剪区域/帧重新插入多模态上下文（保留视觉细节但上下文大幅膨胀）。为此，本文提出VideoLVR——在相关视频时刻从局部区域构建潜在视觉证据，为中间推理步骤提供视觉支撑。为监督潜在视觉证据生成，引入视觉token预测目标：将视觉token重构形式化为基于输入视频派生词表的预测，统一视觉证据重构和文本生成于同一token预测框架。该方法还扩展到强化学习：将采样视觉token视为动作与文本token联合优化。通过在推理中高效回溯任务相关视觉信息，VideoLVR在不显著扩展多模态上下文的情况下改善了视频理解。

原文摘要

Chain-of-thought reasoning has substantially advanced video understanding in multimodal large language models (MLLMs). However, language-based reasoning traces do not provide a reliable mechanism for models to revisit relevant visual evidence, leading to visual forgetting and impairing their ability to reason over video content. Existing approaches maintain visual grounding during reasoning in two main ways: they either refer to evidence using textual coordinates, which specify symbolic locations rather than the visual content itself, or reinsert cropped regions or frames into the multimodal context, preserving visual details at the cost of substantial context expansion. To address these limitations, we introduce VideoLVR, which constructs latent visual evidence from local regions at relevant video moments, thereby providing visual support for intermediate reasoning steps. To supervise latent visual evidence generation, we introduce a visual token prediction objective that formulates visual token reconstruction as prediction over a vocabulary derived from the input video, thereby unifying visual evidence reconstruction and text generation within the same token prediction framework. This shared prediction framework enables MLLMs to reason over interleaved text tokens and latent visual states while integrating evidence from different moments. We further extend VideoLVR to reinforcement learning by treating sampled visual tokens as actions alongside text tokens and jointly optimizing the policy using final-answer rewards. By efficiently revisiting task-relevant visual information during reasoning, VideoLVR improves video understanding without substantially expanding the multimodal context. Experiments on multiple video benchmarks demonstrate its effectiveness, especially on tasks requiring fine-grained visual evidence.

---

*自动采集于 2026-10-29*

#论文 #arXiv #AI #小凯