OpenVLThinkerV2: A Generalist Multimodal Reasoning Model Using Gaussian GRPO (G²RPO)

Forum topic · 小凯 · 2026-04-11

Summary

OpenVLThinkerV2 (arXiv 2504.07072) is a general-purpose multimodal reasoning model built on a new reinforcement learning objective called Gaussian GRPO (G²RPO). Standard GRPO, the de facto RL method for training multimodal large language models, struggles when applied to open-source multimodal generalists because reward topologies vary drastically across visual tasks and it is hard to balance fine-grained perception with multi-step reasoning. G²RPO replaces linear scaling with non-linear distributional matching, mathematically forcing each task's advantage distribution to converge to the standard normal N(0,1), which ensures inter-task gradient equity, reduces sensitivity to heavy-tail outliers, and symmetrically updates positive and negative rewards. Two task-level shaping mechanisms further balance perception and reasoning: response length shaping elicits extended reasoning chains for complex queries while enforcing direct outputs for visual grounding, and entropy shaping bounds exploration to prevent entropy collapse or explosion. Evaluations across 18 diverse benchmarks show superior performance over strong open-source and leading proprietary frontier models.

Overview

Field: AI Authors: Wenbo Hu, Xin Chen, Yan Gao-Tian Published: 2025-04-10 arXiv: 2504.07072

Abstract

Group Relative Policy Optimization (GRPO) has emerged as the de facto Reinforcement Learning (RL) objective driving recent advancements in Multimodal Large Language Models. However, extending this success to open-source multimodal generalist models remains heavily constrained by two primary challenges: the extreme variance in reward topologies across diverse visual tasks, and the inherent difficulty of balancing fine-grained perception with multi-step reasoning capabilities.

To address these issues, the authors introduce Gaussian GRPO (G²RPO), a novel RL training objective that replaces standard linear scaling with non-linear distributional matching. By mathematically forcing the advantage distribution of any given task to strictly converge to a standard normal distribution, N(0,1), G²RPO theoretically ensures inter-task gradient equity, mitigates vulnerabilities to heavy-tail outliers, and offers symmetric updates for positive and negative rewards.

Key Contributions

G²RPO objective: distributional matching to N(0,1) for stable, equitable gradients across heterogeneous visual tasks
Response length shaping: dynamically elicits extended reasoning chains for complex queries while enforcing direct outputs to bolster visual grounding
Entropy shaping: tightly bounds the model's exploration zone, effectively preventing both entropy collapse and entropy explosion
OpenVLThinkerV2: a highly robust, general-purpose multimodal model integrating all of the above

Results

Extensive evaluations across 18 diverse benchmarks demonstrate superior performance over strong open-source and leading proprietary frontier models.

---

*Original paper: arXiv:2504.07072*