WorldWeaver (W²): Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers

Forum topic · 小凯 · 2026-07-26

Summary

WorldWeaver (W²) is a streaming multi-agent video diffusion model proposed by Sicheng Mo, Yuheng Li, and Ziyang Leng (arXiv:2507.20487) for interactive world modeling. The paper argues that multi-agent interactive world models must maintain world states that persist across agents and evolve across viewpoints, rather than merely generating consistent observations. Existing autoregressive video diffusion pipelines condition on observation history, which makes shared state hard to maintain in multi-agent, multi-view settings. WorldWeaver addresses this by augmenting the rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. These registers are grounded by supervision signals covering individual agent status, global state views (including bird's-eye views), and scene text. A hybrid Transformer design with separate weights for world state modeling and visual frame modeling further improves the architecture. Experiments on two-agent Minecraft video generation show that explicit world state modeling improves both logical consistency and generation quality.

Paper Overview

Research Area: Computer Vision (CV) Authors: Sicheng Mo, Yuheng Li, Ziyang Leng Published: 2026-07-25 arXiv: 2507.20487

Abstract

Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings.

The authors present WorldWeaver (W²), a streaming multi-agent video diffusion model that augments the rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. These registers are grounded with supervision signals spanning individual agent status, global state views (including bird's-eye views), and scene text. The architecture is further improved through a hybrid Transformer design that uses independent weights for world state modeling and visual frame modeling.

Extensive experiments on two-agent Minecraft video generation demonstrate that explicit world state modeling improves logical consistency and generation quality.

Key Contributions

Cross-agent world state registers: learnable tokens storing shared world information, updated dynamically after each generated chunk
Multi-signal grounding: supervision from individual agent status, global state views (bird's-eye view), and scene text
Hybrid Transformer design: separate weights for world state modeling vs. visual frame modeling
Results: improved logical consistency and generation quality in two-agent Minecraft video generation