OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks

Paper Overview

• Field: AI
• Authors: Wenbo Hu, Xin Chen, Yan Gao-Tian
• Released: 2025-04-10
• arXiv: 2504.07072

Abstract

Group Relative Policy Optimization (GRPO) has emerged as the de facto Reinforcement Learning (RL) objective driving recent advancements in Multimodal Large Language Models. However, extending this success to open-source multimodal generalist models remains heavily constrained by two primary challenges: the extreme variance in reward topologies across diverse visual tasks, and the inherent difficulty of balancing fine-grained perception with multi-step reasoning capabilities.

To address these issues, the authors introduce Gaussian GRPO (G^2RPO), a novel RL training objective that replaces standard linear scaling with non-linear distributional matching. By mathematically forcing the advantage distribution of any given task to strictly converge to a standard normal distribution N(0,1), G^2RPO theoretically ensures:

• Inter-task gradient equity
• Reduced vulnerability to heavy-tail outliers
• Symmetric updates for positive and negative rewards

Leveraging the enhanced training stability of G^2RPO, two task-level shaping mechanisms are introduced to seamlessly balance perception and reasoning:

1. Response length shaping — dynamically elicits extended reasoning chains for complex queries while enforcing direct outputs to bolster visual grounding. 2. Entropy shaping — tightly bounds the model's exploration zone, effectively preventing both entropy collapse and entropy explosion.

Integrating these methodologies, the authors present OpenVLThinkerV2, a highly robust, general-purpose multimodal model. Extensive evaluations across 18 diverse benchmarks demonstrate superior performance over strong open-source and leading proprietary frontier models.

--- *Auto-collected on 2025-04-11*