Programmable World Model: Decoupling World State from Generative Rendering

Forum topic · 小凯 · 2026-09-11

Summary

A new framework called Programmable World Model addresses a key weakness of current video world models: the inability to maintain persistent world state and enforce programmable rules during long interactions. The system decouples world-state evolution from visual generation. A language agent translates natural-language instructions into executable programs defining entity states and transition rules, while a lightweight engine maintains an explicit, persistent global state including off-screen entities and non-visual attributes. State-augmented 3D oriented bounding boxes serve as an intermediate representation, deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model acting as a generative renderer. This enables playable games with predefined mechanics, direct entity-level control, and persistent state across gameplay. The authors also introduce CombatStateBench, where the method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation.

论文概要

Research area: Computer Vision

Authors: Zheng-Hui Huang, Guixu Lin, Jiacheng Lin, Yi-Chuan Huang, Ruihan Yu, Muyao Niu, Siqi Yang, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, Zhixiang Wang

Published: 2026-09-09

arXiv: 2609.10540

Summary

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. This paper introduces Programmable World Model, a framework that decouples world-state evolution from visual observation generation.

An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes.

To connect world state with visual generation, the authors introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer.

This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay.

Benchmark Results

The paper also introduces CombatStateBench, a benchmark for evaluating programmable world models:

94% Count Accuracy
98% State Accuracy

The method substantially outperforms existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

--- *Auto-collected on 2026-09-11*

Programmable World Model: Decoupling World State from Generative Rendering

Summary

论文概要

Summary

Benchmark Results

Tags