[论文] Programmable World Model

论文概要

研究领域: CV 作者: Zheng-Hui Huang, Guixu Lin, Jiacheng Lin, Yi-Chuan Huang, Ruihan Yu, Muyao Niu, Siqi Yang, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, Zhixiang Wang 发布时间: 2026-09-09 arXiv: 2609.10540

中文摘要

近来的视频世界模型能生成越来越逼真且可交互的视觉体验，但缺乏在长时间交互中维持持久世界状态和执行可编程规则的可靠机制。本文提出「可编程世界模型」(Programmable World Model)，一个将世界状态演化与视觉观测生成分离的框架。智能体将自然语言指令翻译成可执行程序，定义实体状态和状态转移规则，实现对单个实体及其交互的直接控制。轻量级引擎执行这些程序以更新并维护显式的持久全局世界状态，包括屏幕外实体和非视觉属性。为了连接世界状态与视觉生成，本文引入「状态增强3D定向包围盒」(OBBs) 作为中间表征，与目标相机轨迹一起被确定性地编译为像素对齐的时空条件信号，供预训练视频模型作为生成渲染器使用。该设计允许用户创建具有预定义机制、对单个实体的直接控制以及贯穿游戏过程的持久世界状态的可玩游戏。本文还提出 CombatStateBench 基准用于评估可编程世界模型，在该基准上达到94%计数准确率和98%状态准确率，大幅超越现有交互式视频世界模型。

原文摘要

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

---

*自动采集于 2026-09-11*

#论文 #arXiv #CV #小凯