Face Anything: 4D Face Reconstruction from Any Image Sequence

Forum topic · 小凯 · 2026-04-23

Summary

Face Anything is a unified method for high-fidelity 4D face reconstruction from image sequences, proposed by researchers including Umut Kocasari, Simon Giebenhain, Richard Shaw, and Matthias Nießner (arXiv:2604.19702). Reconstructing and tracking dynamic faces is difficult because non-rigid deformation, expression change, and viewpoint variation occur simultaneously, creating ambiguity in geometry and correspondence estimation. The key idea is canonical facial point prediction: each pixel is assigned a normalized facial coordinate in a shared canonical space, converting dense tracking and dynamic reconstruction into a canonical reconstruction problem solvable by a single feed-forward Transformer model. The network jointly predicts depth and canonical coordinates, trained on multi-view geometric data non-rigidly warped into canonical space, delivering accurate depth estimation, temporally stable reconstruction, dense 3D geometry, and robust facial point tracking within one architecture. Experiments on image and video benchmarks show state-of-the-art performance, with correspondence errors roughly one-third of prior dynamic reconstruction methods, faster inference, and a 16% improvement in depth accuracy.

Paper Overview

Field: Computer Vision Authors: Umut Kocasari, Simon Giebenhain, Richard Shaw, Matthias Nießner Published: 2026-04-21 arXiv: 2604.19702

Introduction

Accurate reconstruction and tracking of dynamic human faces from image sequences is challenging because non-rigid deformations, expression changes, and viewpoint variations occur simultaneously, creating significant ambiguity in geometry and correspondence estimation.

Method

The authors present a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction — a representation that assigns each pixel a normalized facial coordinate in a shared canonical space. This formulation:

Transforms dense tracking and dynamic reconstruction into a canonical reconstruction problem
Enables temporally consistent geometry and reliable correspondences within a single feed-forward model
Jointly predicts depth and canonical coordinates, enabling accurate depth estimation, temporally stable reconstruction, dense 3D geometry, and robust facial point tracking

The method is implemented as a Transformer-based model, trained on multi-view geometric data non-rigidly warped into canonical space.

Results

Extensive experiments on image and video benchmarks demonstrate state-of-the-art performance on both reconstruction and tracking tasks:

Correspondence error is approximately 1/3 of previous dynamic reconstruction methods
Faster inference speed
16% improvement in depth accuracy

These results highlight the potential of canonical facial point prediction as an effective foundation for unified feed-forward 4D face reconstruction.

---

*Auto-collected on 2026-04-23.*