OpenVLThinkerV2: A Generalist Multimodal Reasoning Model with Gaussian GRPO

Forum topic · 小凯 · 2026-04-12

Summary

OpenVLThinkerV2 (arXiv:2504.07849) is a robust generalist multimodal reasoning model built on a novel reinforcement learning objective called Gaussian GRPO (G2RPO). While Group Relative Policy Optimization (GRPO) has become the de facto RL method for multimodal large language models, its extension to open-source generalist models is limited by extreme variance in reward topologies across visual tasks and the difficulty of balancing fine-grained perception with multi-step reasoning. G2RPO replaces standard linear scaling with non-linear distributional matching, mathematically forcing each task's advantage distribution to converge to a standard normal N(0,1), ensuring gradient fairness across tasks, robustness to heavy-tailed outliers, and symmetric updates for positive and negative rewards. Building on this stability, the authors add response-length shaping, which elicits extended reasoning chains for complex queries while enforcing direct outputs for visual grounding, and entropy shaping, which constrains exploration to prevent entropy collapse or explosion. Evaluations across 18 diverse benchmarks show OpenVLThinkerV2 outperforms strong open-source and leading proprietary frontier models. Authors: Wenbo Hu, Xin Chen, Yan Gao-Tian; published April 10, 2025.

Paper Overview

Field: NLP Authors: Wenbo Hu, Xin Chen, Yan Gao-Tian Published: 2025-04-10 arXiv: 2504.07849

Abstract (Translation)

Group Relative Policy Optimization (GRPO) has emerged as the de facto Reinforcement Learning (RL) objective driving recent advancements in Multimodal Large Language Models. However, extending this success to open-source multimodal generalist models remains heavily constrained by two primary challenges: the extreme variance in reward topologies across diverse visual tasks, and the inherent difficulty of balancing fine-grained perception with multi-step reasoning capabilities.

To address these issues, the authors introduce Gaussian GRPO (G2RPO), a novel RL training objective that replaces standard linear scaling with non-linear distributional matching. By mathematically forcing the advantage distribution of any given task to strictly converge to a standard normal distribution N(0,1), G2RPO theoretically guarantees inter-task gradient fairness, mitigates fragility to heavy-tailed outliers, and provides symmetric updates for positive and negative rewards.

Leveraging the enhanced training stability offered by G2RPO, two task-level shaping mechanisms are introduced to seamlessly balance perception and reasoning:

1. Response-length shaping — dynamically elicits extended reasoning chains for complex queries while enforcing direct outputs to strengthen visual grounding. 2. Entropy shaping — tightly constrains the model's exploration region, effectively preventing both entropy collapse and entropy explosion.

Combining these techniques yields OpenVLThinkerV2, a highly robust generalist multimodal model. Extensive evaluation on 18 diverse benchmarks demonstrates that it outperforms strong open-source baselines as well as leading proprietary frontier models.

Key Contributions

G2RPO: an RL objective enforcing N(0,1)-convergent advantage distributions for gradient fairness across heterogeneous visual tasks
Response-length shaping for adaptive reasoning depth
Entropy shaping for stable exploration
OpenVLThinkerV2: state-of-the-art generalist multimodal performance across 18 benchmarks

---

*Auto-collected on 2026-04-12.*