AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in Unified Multimodal Models via GRPO

Summary

AlphaGRPO is a novel framework that applies Group Relative Policy Optimization (GRPO) to AR-Diffusion Unified Multimodal Models (UMMs), enhancing multimodal generation capabilities without requiring an additional cold-start stage. The approach enables two advanced reasoning capabilities: Reasoning Text-to-Image Generation, where the model actively infers implicit user intent, and Self-Reflective Refinement, where it autonomously diagnoses and corrects misalignments in generated outputs. To provide stable supervision for real-world multimodal generation, the authors introduce Decompositional Verifiable Reward (DVReward). Instead of a holistic scalar reward, DVReward uses an LLM to decompose complex user requests into atomic, verifiable semantic and quality questions, which are assessed by a general-purpose MLLM to deliver reliable and interpretable feedback. Extensive experiments show that AlphaGRPO achieves robust improvements on multimodal generation benchmarks including GenEval, TIIF-Bench, DPG-Bench, and WISE, and delivers significant gains on GEdit editing tasks even without any editing-task training. Paper: arXiv 2605.12495, by Runhui Huang, Jie Wu, Rui Yang, Zhe Liu, and Hengshuang Zhao.