Paper Overview
- Field: Computer Vision (CV)
- Authors: Yuan Wang, Yongchao Du, Mengting Chen, et al.
- arXiv: 2507.17086
- Single-image editing
- Multi-image composition
- Text-to-image generation
- Bootstrapped Reward Refinement (BRR)
- Conflict-aware Pareto Advantage Fusion (CPAF)
- Paper: https://arxiv.org/abs/2507.17086
Abstract (Translated Summary)
Recent advances in multimodal generative models have enabled instruction-based image generation to move beyond semantic manipulation toward knowledge-driven visual reasoning. However, existing methods focus on explicit commonsense reasoning, shallow causal understanding, and direct knowledge recall, and perform poorly on knowledge-intensive generation tasks.
The authors introduce ExpertVerse, a capability-centric benchmark that evaluates generative models through a knowledge-intensive lens. ExpertVerse stratifies reasoning-driven generation across an orthogonal taxonomy of 9 cognitive capabilities and 8 expert disciplines, yielding 58 sub-disciplines. It comprises 1,611 expert-annotated instances covering:
The team further develops an automated pipeline to generate ExpertVerse-100K, a large-scale dataset annotated with reasoning trajectories and knowledge-grounded rationales. Building on this, they train KnowThinker, a VLM reasoning engine with world knowledge, via RL fine-tuning to jointly produce thinking processes and refined instructions.
To address cross-modal credit assignment misalignment and multi-objective gradient conflicts in multi-reward optimization, they propose Bootstrapped Pareto Policy Optimization (BPPO), which combines:
Extensive experiments across open-source and proprietary models expose critical reasoning deficiencies, highlighting the need for knowledge-intensive benchmarks for next-generation visual generation.