CVPD: Self-Contained Visual Distillation for Multimodal LLMs via Counterfactual Blind Spots

Forum topic · 小凯 · 2026-08-11

Summary

Self-improvement for multimodal large language models (MLLMs) typically relies on reward-based methods that supply only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but visual distillation usually depends on privileged context from external annotations, tools, or stronger models. This paper introduces CVPD (Contrastive Counterfactual Visual Process Distillation), described as the first fully self-contained framework for dense, on-policy, token-level visual self-distillation in MLLMs. CVPD identifies visual blind spots: regions where zooming in changes and sharpens the model's answer distribution, while removing the same region leaves full-image behavior largely unchanged. A three-gate Counterfactual Criterion detects these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD surpasses six self-evolving baselines across twelve benchmarks, including methods relying on external GPT-4o supervision, with no regressions.

Paper Overview

Research Area: Computer Vision (CV)
Authors: Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar
Release Date: 2026-08-11
arXiv: 2508.03807

Abstract (English Translation)

Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce CVPD (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs.

CVPD identifies visual blind spots — regions where zooming into an area changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning.

We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation.

Key Results

On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression:

OCRBench: +3.60
MMStar Fine-Grained Perception: +3.38
MMStar Logical Reasoning: +3.08

It maintains or improves performance on broader multimodal benchmarks.

Original Abstract

> Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce CVPD (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of +3.60 on OCRBench, +3.38 on MMStar Fine-Grained Perception, and +3.08 on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.

---

*Auto-collected on 2026-08-12*