Programmable World Model: Making AI Worlds Rule-Based, Persistent, and Playable

*English translation of a Chinese forum post from zhichai.net analyzing the paper "Programmable World Model".*

Paper Info

• Title: Programmable World Model
• Authors: Zheng-Hui Huang, Guixu Lin, Jiacheng Lin et al.
• Field: Computer Vision (cs.CV)
• arXiv: To be published

The Core Paradox

We have AI that can write sonnets, code, and cinematic video—yet fails at a seemingly simple task: "generate a Breakout game where the ball destroys bricks on contact." The root cause is that current generative models are stateless. They model statistical associations between pixels, not the evolution of entities, states, and rules. Ask a video model for a Pac-Man game and you get hallucinations: walls shifting, ghosts passing through the player, power-pellets having no effect. The failures are not about drawing—they are about *playing correctly*.

The paper's key move is to separate rules from generation rather than try to teach the generator rules.

Architecture: Decoupling the World

The system splits into two halves connected by an explicit bridge:

1. State side: An Agent translates natural-language instructions into executable state-transition rules (e.g., IF ball collides with brick THEN brick.state = destroyed). A lightweight state engine maintains a global, explicit, persistent world state table—entities remain tracked even when off-screen. Updates are deterministic and repeatable.

2. Pixel side: A pretrained video generation model whose only job is rendering—making things look good, with no responsibility for logic.

3. The bridge — state-augmented 3D oriented bounding boxes (OBBs): Each entity is described by its 3D position, dimensions, rotation, and state label. A deterministic compiler converts OBBs plus camera trajectories into pixel-aligned spatiotemporal conditioning signals that guide the video model like an underdrawing. Deterministic compilation ensures geometric correctness; generative rendering ensures visual realism.

Why OBBs beat pixels as an intermediate representation

• Low-level: pixel output would force the state engine to know what things *look* like.
• Occlusion: OBBs are 3D, so occlusion is handled at render time, not by the state engine.
• Multi-view: the same OBB set renders consistently from any camera angle—nearly impossible with 2D video generation.
• Compact: one box needs a handful of numbers, not hundreds of pixel values.

Results: Logic, Not Just Looks

On CombatStateBench, a benchmark testing rule compliance over long sequences (not visual quality):

• Count Accuracy: 94% — the model does not spontaneously create or destroy entities.
• State Accuracy: 98% — entities enter and hold the correct states.

Existing interactive video world models perform poorly on both, because they have no notion of state at all.

Comparison with game engines

| Component | Game Engine | Programmable World Model | |---|---|---| | State management | Game logic scripts | Agent-translated state engine | | Intermediate representation | Scene graph / ECS | 3D OBBs | | Rendering | Rasterization / ray tracing | Pretrained video generation model |

The logic layer matches traditional engines; the renderer is learned and generative, producing richer visuals while preserving logical correctness.

Applications and Open Questions

Beyond playable game generation, the post sketches uses in physics simulation, robotics environment generation, film previsualization, and interactive education. Caveats remain:

1. Visual quality depends on the underlying video model's training coverage. 2. Rule expressiveness: continuous dynamics, soft bodies, and probabilistic events may strain the Agent's natural-language-to-rule translation. 3. Open worlds: thousands of concurrent entities and exploding state spaces may bottleneck the engine or OBB compilation. 4. Learning rules from observation (causal discovery) remains an open problem—rules are currently hand-specified.

Closing Thought

The deeper claim: intelligence is not prediction but intervention. By making the world's causal structure directly manipulable—persistent state, enforceable rules, controllable entities—the Programmable World Model restores something current generative AI lacks: a world that can be understood, and therefore controlled.

---

Reference: Huang, Z.-H., Lin, G., Lin, J., et al. (2026). *Programmable World Model*. arXiv preprint.

*Translated from a Chinese forum post on zhichai.net; daily paper recommendation series.*