Paper Overview
Field: Computer Vision (CV) Authors: Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar Published: 2026-08-11 arXiv: 2508.03807
Summary
Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models.
We introduce CVPD (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots — regions where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information the model can encode but fails to consistently exploit under full-image conditions.
We propose a three-stage counterfactual criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD surpasses six self-evolution baselines (including methods that rely on external GPT-4o supervision) across 12 benchmarks, with no performance regressions. It achieves +3.60 on OCRBench, +3.38 on MMStar fine-grained perception, and +3.08 on MMStar logical reasoning, while maintaining or improving performance on a broader range of multimodal benchmarks.
Key Points
- Problem: Existing MLLM self-improvement uses coarse reward signals; visual distillation depends on external annotations, tools, or stronger teacher models.
- Method — CVPD: A fully self-contained, on-policy, token-level visual self-distillation framework that needs no external supervision.
- Core Mechanism — Visual Blind Spots: Regions where zoomed-in views sharpen the answer distribution, while removing those regions leaves full-image behavior nearly unchanged — exposing latent perceptual capability.
- Three-Stage Counterfactual Criterion: Derives dense contrastive supervision purely from the model's own responses.
- Results: On Qwen3-VL-8B-Instruct, outperforms six self-evolution baselines (including GPT-4o-supervised methods) across 12 benchmarks with no regressions.
- Notable Gains: +3.60 on OCRBench, +3.38 on MMStar fine-grained perception, +3.08 on MMStar logical reasoning.
- Significance: Demonstrates that counterfactual, self-derived dense signals can match or exceed distillation pipelines built on stronger external models.