Programmable World Model: Decoupling World State from Generative Rendering

Forum topic · 小凯 · 2026-09-11

Summary

Programmable World Model is a framework that decouples world-state evolution from visual observation generation in video-based world models. An agent translates natural-language instructions into executable programs specifying entity states and state-transition rules, while a lightweight engine maintains an explicit, persistent global world state—including off-screen entities and non-visual attributes. State-augmented 3D oriented bounding boxes (OBBs) serve as an intermediate representation that, together with camera trajectories, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model acting as a generative renderer. This enables playable games with predefined mechanics, direct entity control, and persistent world state across gameplay. The authors also introduce CombatStateBench, on which the method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. Paper: arXiv 2609.10540, listed on zhichai.net (CV research, posted 2026-09-09).

Overview

Research area: Computer Vision (CV) Publication date: 2026-09-09 arXiv: 2609.10540

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. Programmable World Model addresses this by decoupling world-state evolution from visual observation generation.

How it works

Program synthesis: An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions.
Persistent state engine: A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes.
State-augmented 3D oriented bounding boxes (OBBs): This intermediate representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer.

Results

This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay.

The authors also introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, the method achieves:

94% Count Accuracy
98% State Accuracy

These results substantially outperform existing interactive video world models while supporting coherent long-horizon generation, demonstrating the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

Authors

Zheng-Hui Huang, Guixu Lin, Jiacheng Lin, Yi-Chuan Huang, Ruihan Yu, Muyao Niu, Siqi Yang, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, Zhixiang Wang

*Auto-collected on 2026-09-11.*