AnyRecon: Arbitrary-View 3D Reconstruction with a Video Diffusion Model

Forum topic · 小凯 · 2026-04-23

Summary

AnyRecon is a scalable framework for sparse-view 3D reconstruction that works from arbitrary, unordered sparse image inputs while preserving explicit geometric control. Unlike prior diffusion-based approaches that condition on only one or two capture frames, AnyRecon supports flexible conditioning cardinality by building a persistent global scene memory through a prepended capture view cache and removing temporal compression to maintain frame-level correspondence under large viewpoint changes. The method couples generation with reconstruction via a geometry-aware conditioning strategy, combining explicit 3D geometric memory with geometry-driven capture view retrieval. For efficiency, it integrates 4-step diffusion distillation with context-window sparse attention to reduce quadratic complexity. Experiments demonstrate robust, scalable reconstruction on unconventional inputs, large viewpoint gaps, and long trajectories. The paper is authored by Yutian Chen, Shi Guo, and colleagues and is available on arXiv as 2604.19747.

Paper Overview

Field: Computer Vision Authors: Yutian Chen, Shi Guo, Renbiao Jin, Tianshuo Yang, Xin Cai, Yawen Luo, Mingxin Yang, Mulin Yu, Linning Xu, Tianfan Xue arXiv: 2604.19747

Abstract

Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remains challenging for non-generative reconstruction. Existing diffusion-based approaches mitigate this issue by synthesizing novel views, but they often condition on only one or two capture frames, which restricts geometric consistency and limits scalability to large or diverse scenes. We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs that preserves explicit geometric control while supporting flexible conditioning cardinality.

To support long-range conditioning, our method constructs a persistent global scene memory via a prepended capture view cache, and removes temporal compression to maintain frame-level correspondence under large viewpoint changes.

Beyond a stronger generative model, we observe that the interplay between generation and reconstruction is critical for large-scale 3D scenes. We therefore introduce a geometry-aware conditioning strategy that couples generation and reconstruction through explicit 3D geometric memory and geometry-driven capture view retrieval.

For efficiency, we combine 4-step diffusion distillation with context-window sparse attention to reduce quadratic complexity. Extensive experiments show that the proposed method achieves robust and scalable reconstruction on unconventional inputs, large viewpoint gaps, and long trajectories.

Key Ideas

Flexible conditioning: accepts arbitrary, unordered sparse views rather than fixed one- or two-frame conditioning.
Persistent global scene memory: prepended capture view cache plus removal of temporal compression preserves frame-level correspondence across large viewpoint changes.
Geometry-aware conditioning: explicit 3D geometric memory and geometry-driven capture view retrieval couple generation with reconstruction.
Efficiency: 4-step diffusion distillation combined with context-window sparse attention reduces quadratic attention complexity.

---

*Auto-collected on 2026-04-23.*