86.9% of VLM Reasoning Errors Originate in Perception, Not Reasoning: A Staged Post-Training Approach

Forum topic · 小凯 · 2026-05-26

Summary

A forum post on zhichai.net discusses a paper from UCSB, Fudan, and Sea AI researchers arguing that 86.9% of vision-language model (VLM) reasoning errors stem from incorrect visual perception rather than weak reasoning—meaning longer chains of thought cannot fix bad perception. The paper proposes a three-stage post-training framework: first visual perception (using automatically synthesized QA pairs from DOCCI), then textual reasoning, then visual reasoning, trained sequentially with GRPO-based RLVR rather than SFT. Perception data is precisely filtered by keeping only samples the model answers wrong when viewing the image but right when reading the caption, isolating true perception gaps. On Qwen3-VL-8B, the staged approach raises overall average accuracy to 65.77 versus 62.19 for the base model and beats merged training by 1.10 points. Notably, staged training shortens response lengths by up to 20.8% during training, since accurate perception eliminates redundant re-checking of images. Ablations confirm perception must be consolidated before visual reasoning (reversing the order costs 4.6 points), RLVR outperforms SFT, and capability-based and difficulty-based curricula combine orthogonally for additive gains.

Overview

A recent paper from UCSB, Fudan, and Sea AI researchers makes a provocative claim: 86.9% of VLM reasoning errors originate in visual perception, not in the reasoning process itself. Models misread geometric relations, chart values, and key visual details—so no amount of longer chains of thought or self-reflection can recover. The paper's core thesis: *longer thinking cannot fix incorrect perception*.

A contrast case illustrates the point: when a model misperceives a tangent length, it enters a long, repetitive loop of "re-examining the image" and fails to converge; when perception is correct from the start, the reasoning chain is short and direct.

Method: Three-Stage Post-Training

The framework decouples VLM capabilities and trains them sequentially:

1. Visual Perception — perception QA pairs auto-synthesized from the DOCCI dataset 2. Textual Reasoning — text-only math reasoning problems 3. Visual Reasoning — multimodal reasoning data

\mathcal{D}_{\text{perc}} \rightarrow \mathcal{D}_{\text{text}} \rightarrow \mathcal{D}_{\text{vis}}

Precision filtering of perception data. A sample is kept only if the model answers wrong when given the image but right when given the caption:

\mathbb{I}[\hat{A}_{\text{img}} \neq A] \land \mathbb{I}[\hat{A}_{\text{cap}} = A]

This isolates genuine perception gaps while guaranteeing the information is present. Dual filtering with Qwen2.5-VL-7B and Qwen2.5-VL-32B avoids single-model bias.

RLVR over SFT. Training uses GRPO with accuracy plus format rewards. On Qwen2.5-VL-7B, RLVR beats SFT on WeMath by 8.19 points (38.29% vs 30.10%), attributed to SFT's off-policy token-level supervision degrading an already strong base.

Key Results

Qwen3-VL-8B staged training: overall average 65.77 vs 62.19 (base) and 64.67 (merged); perception average rises to 80.44
WeMath 56.1% (+5.2 over base), RealWorldQA 74.5% (+3.7 over base), MMStar 73.1%
Staged beats merged on both Qwen3-VL-8B (+1.10) and Qwen2.5-VL-7B (+1.41)
Response lengths shrink: -20.8% during Stage 3 training; on test sets, -6.6% (MathVista) to -12.6% (MathVerse) vs merged training
Perception error rate drops from 28.2% (base) to 25.7% (staged)

The shorter reasoning chains reveal the mechanism: accurate perception eliminates repetitive "re-check the image" loops. Better perception yields higher accuracy *and* lower inference cost and latency.

Ablations

Order matters: training visual reasoning before perception (3→2→1) drops Qwen2.5-VL-7B overall accuracy by 4.6 points; perception-first and text-first orders perform comparably
RLVR vs SFT: +3.40 points overall on Qwen2.5-VL-7B
Orthogonal curricula: capability-based (+1.97) and difficulty-based (+1.80) curricula combine for +4.43 points

Strategic Implications

Mainstream "mix everything" post-training dilutes perception with reasoning data; the three capabilities have different learning dynamics
If 86.9% of errors are perceptual, returns on reasoning-enhancement methods (long CoT, self-reflection) are capped near 13.1%
The caption-based filtering pattern generalizes: OCR vs. human text for documents, subtitles for video, clinical text for medical imaging
Limitations: experiments only cover 7B/8B models; DOCCI-based QA generation is bounded by the Qwen2.5-72B generator's own perception; the role of staged tuning of the vision encoder remains unexplored

Conclusion

The paper's message: perception is a *precondition* for reasoning, not a parallel peer. Let the model "see clearly" first, then "think clearly." Reducing error sources, the authors argue, is more effective than building error-recovery capability—a counterintuitive but practical engineering principle for teams training or fine-tuning VLMs.

References: arXiv:2605.20177, DOCCI dataset, GRPO/RLVR, benchmarks MathVista / MathVision / MathVerse / WeMath.