English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

From Pixels to States: A Survey on Interactive World Models as Game Engines (arXiv 2607.14076)

Forum topic · 小凯 · 2026-07-17

Summary

This post introduces a CV research survey, 'From Pixels to States: Rethinking Interactive World Models as Game Engines,' authored by Zhen Li, Zian Meng, Shuwei Shi, and colleagues, posted on arXiv (2607.14076). The paper examines whether video generative models can serve as next-generation game engines by predicting future observations conditioned on user actions. Using the classic action-state-observation loop of conventional game engines as an organizing lens, the survey analyzes interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation. For each dimension, existing methods are grouped into representative families, with their strengths and trade-offs discussed. The authors also contribute a scalable data engine built on the game Black Myth: Wukong, collecting over 90 hours of gameplay with frame-aligned player actions, real game states, visual observations, and structural/semantic annotations, released as a resource for state-aware world modeling. The post includes both a Chinese summary and the original English abstract, aiming to provide a clear picture of the field's current state.

Paper Overview

Research Area: Computer Vision (CV) Authors: Zhen Li, Zian Meng, Shuwei Shi, Mingliang Zhai, Jiaming Tan, Chuanhao Li, Kaipeng Zhang Posted: 2026-07-15 arXiv: 2607.14076

Abstract (English)

Building interactive worlds that respond coherently to player actions has long been a shared goal of computer graphics, games, and artificial intelligence. Recent video generative models provide a data-driven route toward this goal by predicting future observations conditioned on user actions, and are increasingly regarded as potential next-generation game engines. Realizing a genuinely interactive game world, however, requires interaction outcomes that follow rules over evolving game conditions, consequences that persist over long horizons, and a generation loop that operates in real time. Conventional game engines realize these properties through a recurrent action-state-observation loop, in which player actions update an explicit game state according to predefined rules and observations are rendered from the resulting state.

Chinese Summary (translated)

Using this recurrent loop as an organizing perspective, the paper surveys interactive game world modeling across four dimensions:

1. Player action control 2. Game state dynamics 3. State-observation persistence 4. Real-time interactive generation

For each dimension, existing methods are grouped into representative families based on the capabilities required for interactive game worlds, and the advantages and trade-offs of each family are discussed.

Data Contribution

As a complementary analysis, the authors present a scalable data engine built on *Black Myth: Wukong*, collecting over 90 hours of gameplay containing:

  • Frame-aligned player actions
  • Real game states and visual observations
  • Structural and semantic annotations
This dataset serves as a resource for state-aware game world modeling. The authors hope the survey offers a clear picture of the field's current state and catalyzes progress toward truly interactive game worlds.

---

*Auto-collected on 2026-07-17*

Tags

#world-models#game-engines#video-generation#survey#computer-vision#interactive-worlds#black-myth-wukong#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178395198