Generative World Renderer: A Large-Scale AAA Game Dataset for Inverse and Forward Rendering

Forum topic · 小凯 · 2026-04-05

Summary

Generative World Renderer (arXiv:2604.02329) addresses the domain gap limiting generative inverse and forward rendering in real-world scenarios. The authors introduce a large-scale dynamic dataset curated from visually complex AAA games, captured via a novel dual-screen stitched method yielding 4 million continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, weather, and motion-blur variants. The dataset supports bidirectional rendering: robust in-the-wild geometry and material decomposition for inverse rendering, and high-fidelity G-buffer-guided video generation for forward rendering. A novel VLM-based evaluation protocol measures semantic, spatial, and temporal consistency of inverse rendering without ground truth, correlating strongly with human judgment. Inverse renderers fine-tuned on this data show superior cross-dataset generalization and controllable generation, and the accompanying toolkit enables text-prompt style editing of AAA games from G-buffers.

Paper Overview

Research Area: Computer Vision Authors: Zheng-Hui Huang, Zhixiang Wang, Jiaming Tan Release Date: 2026-04-02 arXiv: 2604.02329

Abstract

Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.

Key Contributions

Large-scale dynamic dataset: 4M continuous frames (720p/30 FPS) extracted from AAA games with synchronized RGB and five G-buffer channels.
Dual-screen stitched capture: a novel method enabling wide, temporally coherent captures across diverse scenes, visual effects, adverse weather, and motion blur.
Bidirectional rendering: supports robust in-the-wild geometry and material decomposition (inverse rendering) and high-fidelity G-buffer-guided video generation (forward rendering).
VLM-based evaluation protocol: assesses inverse rendering without ground truth via semantic, spatial, and temporal consistency, correlating strongly with human judgment.
Text-driven style editing: the toolkit lets users edit AAA game styles from G-buffers using text prompts.