PROWBench: Benchmarking Whether Video Models Render What the Program Specifies

Forum topic · 小凯 · 2026-10-03

Summary

PROWBench is a benchmark for evaluating whether programmable world models—systems that separate executable dynamics from visual generation—visually adhere to explicit, program-specified rules and events. Existing benchmarks focus on visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained program-specified world events. PROWBench comprises 170 programmatically constructed episodes and 600 proxy videos spanning diverse scenes and interactions. It logs entity states and timestamped events, including off-screen events, as replayable world records, from which synchronized views and proxy representations (e.g., coarse 3D renders and bounding boxes) are rendered. This allows generated videos to be checked against the observable outcomes of program execution. The benchmark covers first- and third-person camera views, with some episodes offering synchronized multi-view observations. Evaluation covers entity control, long-horizon memory, and visual realization of specified timelines via two VLM-based metrics: logic-rendering alignment and interaction success rate. The framework is scalable, supporting automated scene construction, behavior control, and multi-representation rendering. Paper: arXiv 2610.02205.

Overview

Research area: Computer Vision (CV)
Authors: Zheng-Hui Huang, Guixu Lin, Yu-Ju Tsai, Jian-Kai Zhu, Fengbo Lan, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, Zhixiang Wang
arXiv: 2610.02205

Abstract

Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events.

The authors introduce PROWBench, comprising 170 programmatically constructed episodes and 600 proxy videos covering diverse scenes and interactions. PROWBench logs entity states and timestamped events, including those outside the camera's field of view, as replayable world records, from which it renders synchronized views and proxy representations. This enables generated videos to be checked against the observable outcomes of program execution.

Key Features

Scalable framework that builds scenes, controls agent behaviors, and renders each camera view into different representations (e.g., coarse 3D and bounding boxes)
Coverage of both first- and third-person camera perspectives
Some episodes provide synchronized multi-view observations

Evaluation

Based on the world records, PROWBench evaluates:

Entity control
Long-horizon memory
Visual realization of specified timelines and timestamped engine-recorded events, measured through two VLM-based metrics:
Logic-rendering alignment
Interaction success rate

---

*Auto-collected on 2026-10-03.*