WorldWeaver (W²): Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers

Forum topic · 小凯 · 2026-07-27

Summary

This paper introduces WorldWeaver (W²), a streaming multi-agent video diffusion model designed for interactive world modeling, addressing the challenge of maintaining shared world states across multiple agents and viewpoints. Existing autoregressive video diffusion pipelines carry observation history as conditioning context, which struggles to sustain consistent shared state in multi-agent, multi-view settings. WorldWeaver augments rollout with cross-agent world state registers—learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. These registers are grounded by supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text. The authors also improve the architecture with a hybrid Transformer design that uses separate weights for world state modeling and visual frame modeling. Experiments on two-agent Minecraft video generation show that explicit world state modeling improves logical consistency and generation quality. Authored by Sicheng Mo, Yuheng Li, and Ziyang Leng, the paper was released on July 27, 2025 (arXiv: 2507.21746) in the computer vision domain.

Paper Overview

Field: CV Authors: Sicheng Mo, Yuheng Li, Ziyang Leng Published: 2025-07-27 arXiv: 2507.21746

Abstract

Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings.

The authors present WorldWeaver (W²), a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk.

Key Contributions

Cross-agent world state registers — learnable tokens that capture and persist shared world information across agents and generation chunks.
Supervised grounding of registers — supervision signals span individual agent status, global state views (including bird's-eye views), and scene text.
Hybrid Transformer architecture — separate weights are used for world state modeling and visual frame modeling.

Results

Extensive experiments on two-agent Minecraft video generation demonstrate that explicit world state modeling improves both logical consistency and generation quality.