Paper Overview
Research area: CV / Graphics Authors: Junxuan Li, Rawal Khirodkar, Chengan He arXiv: 2604.02320
Abstract
High-quality 3D avatar modeling faces a critical trade-off between fidelity and generalization. On the one hand, multi-view studio data enables high-fidelity modeling of humans with precise control over expressions and poses, but it struggles to generalize to real-world data due to limited scale and the domain gap between the studio environment and the real world. On the other hand, recent large-scale avatar models trained on millions of in-the-wild samples show promise for generalization across a wide range of identities, yet the resulting avatars are often of low-quality due to inherent 3D ambiguities.
To address this, the authors present Large-Scale Codec Avatars (LCA), a high-fidelity, full-body 3D avatar model that generalizes to world-scale populations in a feedforward manner, enabling efficient inference.
Key Ideas
- Pre/post-training paradigm for 3D avatars: Inspired by the success of large language models and vision foundation models, the paper presents, for the first time, a pre/post-training paradigm for 3D avatar modeling at scale.
- Two-stage training:
- Pre-training on 1M in-the-wild videos to learn broad priors over appearance and geometry.
- Post-training on high-quality curated data to enhance expressivity and fidelity.
- LCA generalizes across hair styles, clothing, and demographics while providing precise, fine-grained facial expressions and finger-level articulation control, with strong identity preservation.
- Emergent capabilities observed despite the absence of direct supervision:
- Generalization to relightability
- Support for loose garments on unconstrained inputs
- Zero-shot robustness to stylized imagery