86.9% of VLM Reasoning Errors Stem from Perception, Not Reasoning: A Staged Post-Training Framework

Forum topic · 小凯 · 2026-05-26

Summary

A post from zhichai.net analyzes a paper from UCSB, Fudan, and Sea AI Labs arguing that 86.9% of vision-language model (VLM) reasoning errors originate in faulty visual perception, not weak reasoning—so longer chain-of-thought cannot fix incorrect perception. The paper proposes a three-stage post-training pipeline (visual perception → textual reasoning → visual reasoning) with perception data auto-synthesized from DOCCI and filtered by a dual-path rule: keep only samples the model answers wrong from images but right from captions, verified independently by Qwen2.5-VL-7B and 32B. Training uses RLVR/GRPO with accuracy plus format rewards instead of SFT. On Qwen3-VL-8B, staged training raises WeMath to 56.1% (+5.2% over base), outperforms merged training by +1.10 average, and cuts response lengths by up to 20.8% during training, since accurate perception removes repetitive image re-checking. Ablations show perception must precede visual reasoning (wrong order loses 4.6%), RLVR beats SFT (WeMath gap 38.29% vs 30.10%), and capability-difficulty curricula stack orthogonally (+4.43 combined). Limitations: only 7B/8B models tested and reliance on a Qwen2.5-72B data generator.

Key claim: longer thinking cannot fix incorrect perception

This post from the zhichai.net forum reviews a paper from UCSB, Fudan, and Sea AI Labs. The central finding: 86.9% of VLM reasoning errors originate in visual perception, not in the reasoning step itself. Models misread geometric relations, chart values, or miss key visual details—so no amount of long chain-of-thought or self-reflection can recover. A paired case study shows a model with wrong perception (misjudged tangent length) spiraling into repetitive "re-check the image" loops, while correct initial perception yields a short, direct reasoning chain. Core thesis: *longer thinking cannot fix incorrect perception*.

Method: three-stage decoupled post-training

1. Stage 1 — Visual Perception: perception QA pairs auto-synthesized from the DOCCI dataset. 2. Stage 2 — Textual Reasoning: pure-text multi-step math problems. 3. Stage 3 — Visual Reasoning: multimodal reasoning data combining both.

Perception data filtering (dual-path)

\hat{A}_{\text{img}} = f_\theta(I, Q), \quad \hat{A}_{\text{cap}} = f_\theta(C, Q)

Keep only samples where the model answers wrong from the image but right from the caption — precise "perception gaps" where the information exists but the model cannot extract it visually. Filtering is double-verified with Qwen2.5-VL-7B and Qwen2.5-VL-32B.

RLVR over SFT

GRPO reinforcement learning with reward $R = r_{\text{acc}} + r_{\text{format}}$ . On Qwen2.5-VL-7B, RLVR beats SFT on WeMath 38.29% vs 30.10% (-8.19% gap); SFT's off-policy token-level supervision on lower-quality data can hurt performance.

Results

Qwen3-VL-8B (staged): WeMath 56.1% (+5.2% over base), RealWorldQA 74.5% (+3.7%), overall AVG 65.77 vs 62.19 base and 64.87 OneThinker-8B.
Staged vs merged training: +1.10 (Qwen3-VL-8B) and +1.41 (Qwen2.5-VL-7B) overall AVG.
Shorter reasoning paths: response length during stage 3 drops 20.8% (562 → 445 tokens); test-time lengths shrink 6.6–12.6% across MathVista, MathVision, MathVerse (VO), WeMath. Better perception removes redundant image re-checking.
Fewer perception errors: 28.2% → 25.7% error rate (857 → 781 of 3044 samples).

Ablations

Order matters: perception→text→visual-reasoning (59.75) ≈ text→perception→visual-reasoning (59.61), but starting with visual reasoning drops to 55.93 (-4.6%). Perception must be consolidated before visual reasoning.
RLVR vs SFT: RLVR wins consistently (gap -3.40 on Qwen2.5-VL-7B).
Orthogonal curricula: capability-dimension (+1.97) and difficulty-dimension (+1.80) curricula stack: +4.43 combined.

Strategic implications

Mixed "train on everything at once" post-training is suboptimal—perception gets diluted by reasoning data.
If 86.9% of errors are perceptual, reasoning-enhancement methods (long CoT, self-reflection) have a ~13.1% ceiling on remaining gains.
The dual-path filtering pattern is reusable: document understanding (image vs OCR text), video (frames vs subtitles), medical imaging (scan vs clinical notes).
20.8% shorter reasoning paths mean lower cost, lower latency, and fewer hallucinated verification loops—a threefold benefit from a better perceptual foundation.

Limitations

Only 7B/8B models tested; unknown whether gains hold at 72B scale.
Perception QA generation depends on Qwen2.5-72B, whose own ceiling bounds data quality.
The paper does not explore staged treatment of the vision encoder itself.

Conclusion

Perception is a precondition for reasoning, not a parallel colleague. The most elegant finding is the causal chain *better perception → shorter reasoning*: reducing the source of errors beats enhancing error correction. Teams training VLMs should audit their perception-to-reasoning data ratio, keep perception training focused, and evaluate perception errors separately from reasoning errors. The number 86.9% is worth remembering.

---

References

Paper: *From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models* (arXiv:2605.20177)
DOCCI dataset; GRPO / RLVR frameworks; OneThinker-8B, GThinker-7B, MMR1-7B, OpenVLThinker-7B; MathVista, MathVision, MathVerse, WeMath