[论文] Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

论文概要

研究领域: CV 作者: Shravan Venkatraman, Ritesh Thawkar, Omkar Thawkar 发布时间: 2026-06-27 arXiv: 2606.27373

中文摘要

最近，自进化大型多模态模型（LMMs）在纯无监督环境下提升视觉推理能力方面受到了关注。然而，现有自进化LMMs中的多角色自博弈和自一致性奖励方案虽然优化了答案一致性，但并未确保解码器关注视觉内容，而是依赖统计语言先验来产生自一致的输出。这导致了一种持续存在的失败模式，我们称之为视觉欠条件化（visual under-conditioning），即解码器依赖语言模式而非视觉信息。我们通过引入视觉token注意力机制来解决这个问题，该机制明确强制解码器在自进化过程中关注视觉内容，确保视觉推理真正基于视觉信息而非语言先验。

原文摘要

Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi-role self-play and self-consistency reward schemes in existing self-evolving LMMs optimize answer agreement without ensuring the decoder attends to visual content, relying instead on statistical language priors to produce self consistent outputs. This leads to a persistent failure mode we term visual under-conditioning, where the decoder r...

--- *自动采集于 2026-06-27*

#论文 #arXiv #计算机视觉 #小凯