4DAnyone: Reconstructing 4D Humans from Casual Monocular Video

Forum topic · 小凯 · 2026-08-22

Summary

4DAnyone is a framework that reconstructs 4D human representations from casually captured, uncalibrated monocular video by generating reconstruction-grade multi-view-consistent videos and lifting them into 4D Gaussian Splatting (4DGS). The authors identify a key limitation of existing camera-controllable video diffusion models, which fail to maintain consistency when synthesizing the dozens of novel views required for 4DGS reconstruction. They trace this to a 'bounded attention context' problem: when target views exceed the capacity of a single DiT forward pass and must be grouped, two coupled bottlenecks emerge—the reference context becomes diluted as generated views accumulate, weakening cross-view appearance guidance, while disjoint target groups cannot exchange information, causing global structural drift. 4DAnyone introduces two complementary designs: Reference Context Packing, which compresses growing reference views into a fixed-length mixed-resolution context, and Target Context Routing, which rotates target view groupings during denoising. Experiments on DNA-Rendering and DyMVHumans show 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction. Paper: arXiv 2608.20335.

4DAnyone: Create Anyone in 4D from a Casual Monocular Video

Field: Computer Vision (CV) Authors: Yudong Jin, Tao Xie, Qihang Zhang, Zehong Shen Published: 2026-08-22 arXiv: 2608.20335

Overview

4DAnyone is a framework for reconstructing 4D humans from uncalibrated, casually captured monocular video. Its core idea is to generate reconstruction-grade, multi-view-consistent videos and lift them into 4D Gaussian Splatting (4DGS).

Problem: Bounded Attention Context

Existing camera-controllable video diffusion models can synthesize novel-view videos, but when the number of target views grows to the dozens required by 4DGS reconstruction, they fail to maintain consistency. The paper identifies this as a "bounded attention context" problem: when target views exceed what a single DiT forward pass can handle, they must be split into groups, creating two coupled bottlenecks:

Reference context side: conditioning grows with already-generated views, weakening cross-view appearance guidance.
Target context side: disjoint groups cannot directly exchange information, leading to global structural drift.

Method

4DAnyone addresses these with two complementary designs:

Reference Context Packing: compresses the growing set of reference views into a fixed-length, mixed-resolution context.
Target Context Routing: rotates target view groupings throughout the denoising process so groups can share information.

Results

Experiments on DNA-Rendering and DyMVHumans demonstrate that 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction.

---

*Auto-collected on 2026-08-22*