[论文] Omni-Woodpecker: Hallucination Correction via Evidence-Prior Rebalanci...

论文概要

研究领域: NLP
作者: Zijian Liu, Chaoyou Fu, Yan Yang, Caifeng Shan, Ran He
发布时间: 2026-10-22
arXiv: 2610.27103

中文摘要

最近的全模态LLM（OLLM）在联合视听推理方面展现出卓越能力，但跨模态幻觉仍然是一个重大挑战。现有的缓解方法主要在输出层面操作，忽略了融合表示中锚定证据与学习先验的纠缠。为解决这一问题，我们提出Omni-Woodpecker（OWP），一个无需训练的训练框架，将幻觉缓解从输出层面操作转向基于冲突条件的表示再平衡。具体而言，OWP首先从互补的模态特定视图中识别跨模态冲突。然后采用正交分解将隐藏状态中的锚定证据与竞争先验分离。最后，OWP在解码前通过抑制误导性先验和恢复被忽略的证据来自适应地再平衡融合表示。大量实验证明了OWP的有效性和强跨架构泛化能力。以Qwen2.5-Omni-7B为骨干网络，OWP在CMM和AVHBench上分别提升12.9和8.6个百分点，同时在四个通用OLLM推理基准上一致获得0.67-2.25个百分点的提升。此外，OWP还能有效泛化到VideoLLaMA2-AV，在相同基准上分别提升4.6和3.9个百分点。代码已公开发布。

原文摘要

Recent Omni LLMs (OLLMs) have demonstrated remarkable capabilities in joint audio-visual reasoning, yet cross-modal hallucinations remain a major challenge. Existing mitigation methods mainly operate at the output level, overlooking the entanglement of grounding evidence and learned priors within fused representations. To address this issue, we propose Omni-Woodpecker (OWP), a training-free framework that shifts hallucination mitigation from output-level manipulation to conflict-conditioned representation rebalancing. Specifically, OWP first identifies cross-modal conflicts from complementary modality-specific views. It then employs orthogonal decomposition to separate grounding evidence from competing priors within the hidden states. Finally, OWP adaptively rebalances fused representation...

自动采集于 2026-10-22

#论文 #arXiv #NLP #小凯