4DAnyone: Reconstructing 4D Humans from Casual Monocular Video via Consistent Novel View Synthesis

Forum topic · 小凯 · 2026-08-22

Summary

4DAnyone is a framework that reconstructs animatable 4D human representations from uncalibrated, casually captured monocular videos. It works by generating reconstruction-grade multi-view-consistent videos with a camera-controlled video diffusion model and lifting them into 4D Gaussian Splatting (4DGS). The authors identify a fundamental limitation of existing camera-control video diffusion models: when the number of target views required for 4DGS reconstruction (dozens of views) exceeds the capacity of a single DiT forward pass, views must be processed in groups. This causes two coupled bottlenecks termed the bounded attention context problem: reference-side conditioning degrades as generated views accumulate, and disjoint target-view groups cannot exchange information, leading to global structural drift. 4DAnyone addresses these with two complementary designs: Reference Context Packing, which compresses the growing set of reference views into a fixed-length mixed-resolution context, and Target Context Routing, which rotates target view groupings during the denoising process. Experiments on DNA-Rendering and DyMVHumans show that 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction. Paper: arXiv 2608.20335.

Paper Overview

Field: Computer Vision Authors: Yudong Jin, Tao Xie, Qihang Zhang, Zehong Shen Published: 2026-08-22 arXiv: 2608.20335

Abstract

4DAnyone is a framework for reconstructing 4D humans from uncalibrated monocular video. It generates reconstruction-grade multi-view-consistent videos and lifts them into 4D Gaussian Splatting (4DGS).

Motivation: The Bounded Attention Context Problem

Existing camera-controlled video diffusion models struggle to maintain consistency when synthesizing the dozens of novel views required for 4DGS reconstruction. Once the number of target views exceeds the capacity of a single DiT forward pass, views must be split into groups, creating two coupled bottlenecks:

Reference context side: the conditioning set grows as more views are generated, weakening cross-view appearance guidance.
Target context side: disjoint groups cannot directly exchange information, causing global structural drift.

Method

4DAnyone resolves these issues with two complementary designs:

1. Reference Context Packing — compresses the growing set of reference views into a fixed-length, mixed-resolution context. 2. Target Context Routing — rotates the grouping of target views throughout the denoising process so all views can exchange information.

Results

Experiments on DNA-Rendering and DyMVHumans demonstrate that 4DAnyone outperforms previous methods in both novel-view video quality and downstream 4DGS reconstruction quality.

---

*Auto-collected on 2026-08-22.*