4DAnyone: Creating Anyone in 4D from a Casual Monocular Video

Forum topic · 小凯 · 2026-08-24

Summary

4DAnyone is a research framework for reconstructing 4D humans from a single uncalibrated monocular video. It generates reconstruction-grade multiview-consistent videos with a camera-controllable video diffusion model and lifts them into 4D Gaussian Splatting (4DGS). The authors identify a key failure mode: when the number of target views exceeds the capacity of a single DiT forward pass, generation must be split into groups, causing two coupled bottlenecks—reference context grows as O(N) and weakens cross-view appearance guidance, while disjoint groups cannot exchange information, leading to global structural drift. 4DAnyone introduces two solutions: Reference Context Packing (RCP), which compresses growing reference views into a fixed-length mixed-resolution context with O(1) complexity, and Target Context Routing (TCR), which rotates target view groups during denoising, sharing context across groups at high noise levels while stabilizing details at low noise. The team also built the MVGameHuman dataset with an internal game engine, combined with light-stage and in-the-wild video data for training. Experiments on DNA-Rendering and DyMVHumans show 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction, with robust in-the-wild generalization. arXiv: 2608.20335.

Overview

Research area: Computer Vision Authors: Yudong Jin, Tao Xie, Qihang Zhang, Zehong Shen, Zhen Xu, Yujun Shen, Hujun Bao, Xiaowei Zhou, Yinghao Xu arXiv: 2608.20335

Abstract (translated summary)

4DAnyone is a framework for reconstructing 4D humans from an uncalibrated monocular video. It works by generating reconstruction-grade multiview-consistent videos and lifting them into 4D Gaussian Splatting (4DGS).

Existing camera-controllable video diffusion models can synthesize plausible novel-view videos, but they fail to maintain consistency when extended to the dozens of target views required for 4DGS reconstruction. The authors identify this failure as a bounded attention context problem: when target views exceed the capacity of a single DiT forward pass, generation must be split into groups, exposing two coupled bottlenecks:

Reference context side: conditioning on all previously generated views grows as O(N), weakening cross-view appearance guidance.
Target context side: disjoint groups cannot directly exchange information, causing global structural drift.

Method

4DAnyone addresses both bottlenecks with two complementary designs:

1. Reference Context Packing (RCP): compresses the growing set of reference views into a fixed-length mixed-resolution context, achieving O(1) reference-context complexity. 2. Target Context Routing (TCR): rotates target view groupings during denoising, sharing context across groups at high-noise steps while stabilizing fine details at low-noise steps.

Dataset and Results

The authors also constructed the MVGameHuman dataset using an internal game engine, combined with light-stage and in-the-wild video data for training. Experiments on DNA-Rendering and DyMVHumans benchmarks show that 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction, with robust in-the-wild generalization.

Original Abstract

> We present 4DAnyone, a framework for reconstructing 4D humans from an uncalibrated monocular video by generating reconstruction-grade multiview-consistent videos and lifting them into 4D Gaussian Splatting (4DGS).

*Auto-collected on 2026-08-24*