[论文] VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Opti...

论文概要

研究领域: CV
作者: Junhao Cheng, Liang Hou, Tianxiong Zhong
发布时间: 2026-06-03
arXiv: 2506.00010

中文摘要

近期的'视频推理'范式利用视频生成模型（VGM）生成时间连贯的视觉轨迹来完成推理任务。尽管最先进的VGM在视觉质量上表现出色，但它们往往难以理解和遵循任务特定规则，导致在各种推理场景中出现逻辑失败。现有工作尝试利用视觉-语言模型（VLM）作为问题预求解器，为VGM生成或优化文本指导。然而，文本描述无法捕捉复杂的时空细节，且VGM即使有有效计划也难以忠实执行细粒度或长尾指令。虽然VLM作为求解器表现挣扎，但它们具有强大的感知能力来评估过程约束满足和最终目标达成。利用这一优势，我们引入了一种范式转变，将VLM的角色转变为'教师'。具体而言，VLM教师提取任务特定规则来构建可微奖励，通过轻量级LoRA模块的测试时在线优化来指导VGM推理器。这种策略实现了自适应测试时优化，并将推理能力扩展到VGM固有边界之外。在符号性（VBVR-Bench）和通用（RULER-Bench）视频推理基准测试上的评估表明，所提出的方法取得了16.7分的平均性能提升，在可比的测试时成本下大幅优于VLM-as-Solver范式（+0.4分）和Best-of-N扩展（+2.2分）。这些发现表明，将VLM整合为测试时教师为实现可泛化的视频推理提供了一种有前景的范式。项目页面：https://VLM-as-Teacher.github.io/

原文摘要

The recent 'Reasoning with Video' paradigm utilizes Video Generation Models (VGMs) to generate temporally coherent visual trajectories to complete reasoning tasks. Although state-of-the-art VGMs excel at visual quality, they often struggle to understand and follow task-specific rules, leading to logical failures across diverse reasoning scenarios. Existing efforts try to utilize Vision-Language Models (VLMs) as problem pre-solvers to produce or refine textual guidance for the VGM. However, textual descriptions fail to capture intricate spatiotemporal details, and VGMs often struggle to faithfully执行细粒度或长尾指令 even with a valid plan. While VLMs struggle as solvers, they possess strong perception capabilities to evaluate process-constraint satisfaction and final-goal achievement. Leveraging thi...

自动采集于 2026-06-03

#论文 #arXiv #CV #小凯