Seeing But Not Believing: Diagnosing Attention-to-Answer Disconnect in VLMs and Amazon's Zero-Cost Fix

Key Points

• Core finding ("Seeing But Not Believing"): VLMs are not "blind"; their deep layers accurately localize evidence regions. Failures occur because reasoning stages underutilize the visual signal already encoded.
• RAPT metric: Tracks per-layer relative attention. Early layers focus on text (question parsing), deeper layers progressively shift to image patches, revealing a sequential text→visual workflow rather than parallel processing.
• Deep-layer attention is precise: Visualization shows shallow layers spread attention broadly, while deeper layers (e.g., layers 16–26 of a 32-layer model) produce sparse, highly concentrated heatmaps that align with ground-truth evidence.
• Evidence is seen but not used: Even on incorrect answers, deep-layer attention to evidence tokens remains higher than to non-evidence tokens, confirming a perception vs. cognition gap.
• Root cause: Textual priors dominate the reasoning/decoding stage. Visual information decays as generation proceeds, producing linguistically fluent but visually ungrounded outputs—analogous to RAG's context under-utilization problem.
• VEA (Visual Evidence Augmentation): A zero-cost, training-free inference-time pipeline:
1. Extract attention from the last ~50% of layers. 2. Denoise and smooth into a highlight mask. 3. Overlay the mask onto the original image. 4. Run a second inference pass on the augmented image.
• Results: Stable accuracy improvements across LLaVA, Qwen, Gemma, and InternVL on VisualCOT; architecture-agnostic; no fine-tuning required.
• Engineering implications: Shifts the locus of VLM hallucination research from "better encoders / more alignment data" toward "better perception–reasoning integration." VEA is immediately deployable, doubling inference cost (two passes) but avoiding retraining, new data, or architectural changes.
• Future directions suggested: Preserve explicit visual participation during decoding (e.g., residual attention to visual tokens, KV-cache prioritization for visual positions), and dynamic inference-time visual enhancement inspired by VEA.

Bullet Findings

• The paper introduces RAPT to quantify attention allocation per token across layers.
• Deep-layer attention concentrates on ground-truth evidence regions even when the final answer is wrong.
• Textual dominance in reasoning, not visual encoder weakness, is identified as the primary driver of multimodal hallucination.
• VEA converts implicit internal attention signals into explicit input-level attentional prompting via image-space highlight masks.
• Experiments use the VisualCOT benchmark with human-annotated evidence regions.
• Improvements are reported across multiple VLM families, supporting the generality of the method.
• Limitations acknowledged: requires two inference passes (latency tradeoff), depends on human-annotated evidence regions for evaluation, and does not directly counteract strong text priors.

Source

• Paper: *Seeing But Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs*
• arXiv: 2510.17771v1 [cs.AI] (20 Oct 2025)
• Authors: Zhining Liu, Ziyi Chen, Hui Liu, Chen Luo, Xianfeng Tang, Suhang Wang, Joy Zeng, Zhenwei Dai, Zhan Shi, Tianxin Wei, Benoit Dumoulin, Hanghang Tong
• Institutions: University of Illinois Urbana-Champaign, Amazon, Penn State University
• Venue: ICLR 2026 (accepted)