English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

Forum topic · 小凯 · 2026-08-15

Summary

PlayWorld is a benchmark for evaluating interactive video world models through multi-modal agent players that pursue long-horizon objectives, such as turning 360 degrees to check environmental consistency or walking into water to verify realistic ripple generation. Because the action sequences needed to reach the same goal vary across models, fixed action-conditioned evaluation is unsuitable for cross-model comparison; PlayWorld instead lets agents interact with world models to achieve 171 specified scenario objectives. Models are assessed along four core dimensions—geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution—plus basic metrics for video quality and controllability. Experiments across nine state-of-the-art world models show current models remain unreliable on long-horizon interactive objectives, especially in maintaining spatial consistency and persistent state evolution. Code and data: https://github.com/kxding/PlayWorld (arXiv 2608.13552).

Paper Overview

Research area: Computer Vision Authors: Kaixin Ding, Xi Chen, Minghong Cai, Zhiyuan Xu, Yiyang Wang, Yuxiang Lu, Junyi Li, Shuyang Chen, Yuan Gao, Xin Tao, Pengfei Wan, Hengshuang Zhao Published: 2026-08-13 arXiv: 2608.13552

Abstract

Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives.

Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability.

Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution.

Key Contributions

  • Agent-based evaluation paradigm: replaces fixed action-conditioned testing with multi-modal agent players that pursue specified long-horizon goals, enabling fair cross-model comparison
  • 171 scenarios: each with a clearly defined interactive objective
  • Four core evaluation dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution, plus video quality and controllability metrics
  • Findings: nine state-of-the-art world models still struggle with spatial consistency and persistent state evolution over long-horizon interactions
  • Resources

  • Paper: https://arxiv.org/abs/2608.13552
  • Code and data: https://github.com/kxding/PlayWorld

Tags

#world-models#benchmark#computer-vision#video-generation#agents#evaluation#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633498