[论文] Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Languag...

论文概要

研究领域: CV
作者: Hongxing Li, Yixin Li, Dingming Li, Zixuan Wang, Yuchen Yan, Wenqi Zhang, Weiming Lu, Yongliang Shen
发布时间: 2026-10-08
arXiv: 2610.12355

中文摘要

空间推理始终是视觉-语言模型（VLM）的持久弱点，因为 RGB 输入并不直接提供几何证据。现有补救方案要么在推理时向模型注入 3D 信息（付出架构和延迟代价），要么用只监督最终答案的结果奖励训练。空间错误起源于感知：误判的深度或方向只能靠场景的真实几何来纠正，而空间训练语料的 3D 扫描源已经提供了这些几何。我们提出 GPD（几何特权蒸馏），在 on-policy 自蒸馏（OPSD）中将几何证据作为特权：对每个问题，深度、语义和鸟瞰视角（BEV）线索被渲染为紧凑文本，与参考答案一起路由给教师模型；特权 KL 仅应用于错误轨迹，增强 GRPO；部署模型保持纯 RGB。在 4B 骨干上，GPD 在 VSI-Bench 取得 57.1，在 MindCube、SPARBench、MMSI-Bench 和 ViewSpatial 上平均 37.6，超越 GRPO 和答案特权 OPSD。消融实验证实 3D 特权与答案特权的互补性、问题条件路由优于全上下文注入、以及将蒸馏限制在错误轨迹上的收益。

原文摘要

Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and latency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide. We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each question, depth, semantic, and bird's-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer...

自动采集于 2026-10-12

#论文 #arXiv #CV #小凯