[论文] Isekai3D SDG: A Platform for Diverse Synthetic Gameplay Visual Data

论文概要

研究领域: CV 作者: Rafal Karp, Gene Liu, Shih-Hsin Li, Eric Courtemanche, Shengze Wang, Josef Spjut, David Pankratz, Mischa Kolbe, Michael Stengel, Sven Elflein, Adam Grygielski, Liwen Wu, Anjul Patney 发布时间: 2026-11-02 arXiv: 2611.03626

中文摘要

我们提出 Isekai3D 合成数据生成（SDG）平台，用于模拟和捕获合成 3D 游戏画面，服务于视觉与世界模型、神经图形学、游戏智能体等研究。平台将可复用的环境与角色资产与 Unreal Engine 插件结合，采用两阶段工作流。第一阶段「世界模拟」生成游戏过程，同时记录世界状态、角色动作、行为与相机轨迹，供后续回放。第二阶段将记录的模拟进行回放，按指定分辨率、帧率、渲染与相机配置捕获选定的输出——如渲染视频、图像、深度图等。将模拟与捕获分离意味着同一段记录的游戏过程可产生多个输出变体，而无需重复原始模拟。视觉输出附带帧对齐的相机、角色与动作元数据。平台将生成的数据以训练友好的格式直接上传，减少中间存储、准备时间与人工处理。本报告描述系统的整体设计，包括平台的资产、插件及其功能，解释它们如何协同生成多样的合成游戏数据。

原文摘要

We present Isekai3D Synthetic Data Generation (SDG), a platform for simulating and capturing synthetic 3D gameplay for research in vision and world models, neural graphics, gameplay agents, and other applications. The platform combines reusable environment and character assets with Unreal Engine plugins in a two-stage workflow. First, world simulation generates gameplay while recording the world state, character movements, actions, and camera tracks needed for replay. Second, the recorded simulation is replayed to capture selected outputs, such as rendered video, images, and depth maps, at a specified resolution and frame rate under selected rendering and camera configurations. Separating simulation from capture allows the same recorded gameplay to produce multiple output variants without ...

#论文 #arXiv #CV #小凯