Omni-Woodpecker: Training-Free Hallucination Correction for Omni LLMs via Evidence-Prior Rebalancing

Forum topic · 小凯 · 2026-10-22

Summary

Omni-Woodpecker (OWP) is a training-free framework that mitigates cross-modal hallucinations in Omni LLMs (OLLMs) during joint audio-visual reasoning. Unlike prior methods that operate at the output level, OWP works at the representation level: it identifies cross-modal conflicts from complementary modality-specific views, uses orthogonal decomposition to separate grounding evidence from competing learned priors in hidden states, and adaptively rebalances the fused representation before decoding by suppressing misleading priors and restoring ignored evidence. With Qwen2.5-Omni-7B as backbone, OWP improves scores on CMM and AVHBench by 12.9 and 8.6 percentage points respectively, while gaining 0.67-2.25 points across four general OLLM reasoning benchmarks. It also generalizes to VideoLLaMA2-AV, improving the same benchmarks by 4.6 and 3.9 points. Paper: arXiv 2610.27103, by Zijian Liu, Chaoyou Fu, Yan Yang, Caifeng Shan, and Ran He; code publicly released.

Paper Overview

Field: NLP
Authors: Zijian Liu, Chaoyou Fu, Yan Yang, Caifeng Shan, Ran He
Published: 2026-10-22
arXiv: 2610.27103

Abstract

Recent Omni LLMs (OLLMs) have demonstrated remarkable capabilities in joint audio-visual reasoning, yet cross-modal hallucinations remain a major challenge. Existing mitigation methods mainly operate at the output level, overlooking the entanglement of grounding evidence and learned priors within fused representations.

To address this issue, the authors propose Omni-Woodpecker (OWP), a training-free framework that shifts hallucination mitigation from output-level manipulation to conflict-conditioned representation rebalancing:

1. OWP first identifies cross-modal conflicts from complementary modality-specific views. 2. It then employs orthogonal decomposition to separate grounding evidence from competing priors within the hidden states. 3. Finally, OWP adaptively rebalances the fused representation before decoding, suppressing misleading priors and restoring ignored evidence.

Results

Extensive experiments demonstrate OWP's effectiveness and strong cross-architecture generalization:

With Qwen2.5-Omni-7B as backbone, OWP improves performance on CMM and AVHBench by 12.9 and 8.6 percentage points, respectively.
Consistent gains of 0.67-2.25 percentage points across four general OLLM reasoning benchmarks.
OWP also generalizes to VideoLLaMA2-AV, improving the same benchmarks by 4.6 and 3.9 percentage points.

The code has been publicly released.

---

*Auto-collected on 2026-10-22*