WorldWeaver (W²): Streaming Multi-Agent Autoregressive Diffusion with World State Registers

Forum topic · 小凯 · 2026-07-27

Summary

WorldWeaver (W²) is a streaming multi-agent video diffusion model that addresses a key weakness of autoregressive video diffusion pipelines: their reliance on observation history as conditioning context makes shared world state hard to maintain in multi-agent and multi-view settings. W² introduces cross-agent world state registers — learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. These registers are grounded via supervision signals covering individual agent status, global state views (including bird's-eye views), and scene text. The architecture uses a hybrid Transformer design with separate weights for world state modeling and visual frame modeling. Experiments on two-agent Minecraft video generation show that explicit world state modeling improves logical consistency and generation quality. Paper by Sicheng Mo, Yuheng Li, and Ziyang Leng, arXiv:2507.21746 (July 2025), in computer vision.

Paper Overview

Field: Computer Vision (CV) Authors: Sicheng Mo, Yuheng Li, Ziyang Leng Released: 2025-07-27 arXiv: 2507.21746

Abstract

Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings.

The authors present WorldWeaver (W²), a streaming multi-agent video diffusion model that augments the rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk.

Key Ideas

World state registers: learnable tokens persisting across the streaming rollout that capture shared world state and per-agent status, updated after every generated chunk.
Supervision grounding: the registers are anchored with supervision signals spanning:
individual agent status,
global state views, including bird's-eye views,
scene text.
Hybrid Transformer design: the architecture uses separate weights for world state modeling versus visual frame modeling.

Results

Extensive experiments on two-agent Minecraft video generation show that explicit world state modeling improves both logical consistency and generation quality compared to pipelines that merely condition on observation history.