4DAnyone: Creating Anyone in 4D from a Casual Monocular Video

Forum topic · 小凯 · 2026-08-24

Summary

4DAnyone is a framework for reconstructing 4D humans from an uncalibrated, casually captured monocular video. It generates reconstruction-grade multiview-consistent videos with a camera-controlled video diffusion model and lifts them into 4D Gaussian Splatting (4DGS). The authors identify a bounded-attention context problem: when the number of target views exceeds a single DiT forward pass, view generation must be split into groups, causing two bottlenecks—reference context grows as O(N) with all previously generated views, weakening cross-view appearance guidance, and disjoint groups cannot exchange information, leading to global structural drift. 4DAnyone introduces Reference Context Packing (RCP), which compresses growing reference views into a fixed-length mixed-resolution context with O(1) complexity, and Target Context Routing (TCR), which rotates target-view groupings during denoising to share context across groups at high noise levels while stabilizing details at low noise levels. The model is trained on a new MVGameHuman dataset built with an internal game engine plus light-stage and in-the-wild video data. Experiments on DNA-Rendering and DyMVHumans show superior novel-view video quality and downstream 4DGS reconstruction with robust in-the-wild generalization.

Paper Overview

Research Area: Computer Vision Authors: Yudong Jin, Tao Xie, Qihang Zhang, Zehong Shen, Zhen Xu, Yujun Shen, Hujun Bao, Xiaowei Zhou, Yinghao Xu Published: 2026-08-22 arXiv: 2608.20335

Original Abstract

We present 4DAnyone, a framework for reconstructing 4D humans from an uncalibrated monocular video by generating reconstruction-grade multiview-consistent videos and lifting them into 4D Gaussian Splatting (4DGS).

Technical Summary

Existing camera-controlled video diffusion models can synthesize plausible novel-view videos but fail to maintain consistency when extended to the dozens of target views required for 4DGS reconstruction. The authors attribute this failure to a bounded attention context problem: when target views exceed the capacity of a single DiT forward pass, they must be split into groups, exposing two coupled bottlenecks.

1. Reference context side: conditioning on all previously generated views grows as O(N), diluting cross-view appearance guidance. 2. Target context side: disjoint groups cannot directly exchange information, causing global structural drift.

4DAnyone addresses both bottlenecks with two complementary designs:

Reference Context Packing (RCP) compresses the growing reference views into a fixed-length mixed-resolution context, achieving O(1) reference-context complexity.
Target Context Routing (TCR) rotates target-view groupings during denoising, sharing context across groups at high noise steps while stabilizing fine details at low noise steps.

The authors also built the MVGameHuman dataset using an internal game engine, combined with light-stage and in-the-wild video datasets for training. Experiments on DNA-Rendering and DyMVHumans show that 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction, with robust in-the-wild generalization.

--- *Auto-collected on 2026-08-24*