English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

CVPD: Self-Contained Visual Self-Distillation from Counterfactual Blind Spots for MLLMs

Forum topic · 小凯 · 2026-08-11

Summary

Researchers propose CVPD (Contrastive Counterfactual Visual Process Distillation), described as the first fully self-contained framework for dense, on-policy, token-level visual self-distillation in multimodal large language models (MLLMs). Unlike reward-based self-improvement methods that give only coarse scalar feedback, or distillation approaches that rely on privileged context such as external annotations, tools, or stronger teacher models, CVPD derives supervision entirely from the model itself. It detects visual blind spots: regions where zooming in changes and sharpens the model's answer distribution, while removing the region leaves full-image behavior largely unchanged, revealing perceptual information the model encodes but fails to consistently use. A three-gate Counterfactual Criterion identifies these regions from the model's own responses and converts them into dense contrastive supervision. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines—including methods using external GPT-4o supervision—across twelve benchmarks with no regressions, gaining +3.60 on OCRBench, +3.38 on MMStar Fine-Grained Perception, and +3.08 on MMStar Logical Reasoning. Paper: arXiv:2508.03807.

Paper Overview

Field: Computer Vision (CV) Authors: Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar arXiv: 2508.03807

Abstract (translated from the original)

Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce CVPD (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of +3.60 on OCRBench, +3.38 on MMStar Fine-Grained Perception, and +3.08 on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.

Key Results

  • Outperforms six self-evolving baselines across twelve benchmarks, including methods using external GPT-4o supervision
  • No performance regression on any benchmark
  • +3.60 on OCRBench
  • +3.38 on MMStar Fine-Grained Perception
  • +3.08 on MMStar Logical Reasoning

Tags

#cv#mllm#self-distillation#counterfactual-learning#qwen3-vl#arxiv#multimodal

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633344