Face Anything: 4D Face Reconstruction from Any Image Sequence (arXiv 2604.19702)

Forum topic · 小凯 · 2026-04-23

Summary

"Face Anything" is a computer vision paper (arXiv 2604.19702) by Umut Kocasari, Simon Giebenhain, Richard Shaw, and Matthias Nießner that introduces a unified method for high-fidelity 4D face reconstruction from image sequences. The core idea is canonical facial point prediction: each pixel is assigned a normalized facial coordinate in a shared canonical space, converting dense tracking and dynamic reconstruction into a canonical reconstruction problem. A Transformer-based model jointly predicts depth and canonical facial coordinates, trained on multi-view geometry data non-rigidly warped into canonical space, enabling temporally consistent geometry, accurate depth estimation, dense 3D geometry, and robust face tracking in a single feed-forward pass. Experiments show state-of-the-art results on image and video benchmarks, with correspondence errors about one-third of prior dynamic reconstruction methods, faster inference, and 16% improved depth accuracy.

Paper Overview

Field: Computer Vision Authors: Umut Kocasari, Simon Giebenhain, Richard Shaw, Matthias Nießner Released: 2026-04-21 arXiv: 2604.19702

Abstract

Accurate reconstruction and tracking of dynamic human faces from image sequences is challenging because non-rigid deformations, expression changes, and viewpoint variations occur simultaneously, creating significant ambiguity in geometry and correspondence estimation.

The authors propose a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction — a representation that assigns each pixel a normalized facial coordinate in a shared canonical space. This formulation transforms dense tracking and dynamic reconstruction into a canonical reconstruction problem, enabling temporally consistent geometry and reliable correspondences within a single feed-forward model.

Key Contributions

By jointly predicting depth and canonical coordinates, the method achieves accurate depth estimation, temporally stable reconstruction, dense 3D geometry, and robust facial point tracking within a single architecture.
The approach is implemented with a Transformer-based model that jointly predicts depth and canonical facial coordinates.
Training uses multi-view geometry data that is non-rigidly warped into the canonical space.

Results

Extensive experiments on image and video benchmarks demonstrate state-of-the-art performance on both reconstruction and tracking tasks:

Correspondence error is roughly 1/3 of previous dynamic reconstruction methods.
Faster inference than prior approaches.
Depth accuracy improved by 16%.

These results highlight the potential of canonical facial point prediction as an effective foundation for unified, feed-forward 4D facial reconstruction.

Original Abstract (Excerpt)

> Accurate reconstruction and tracking of dynamic human faces from image sequences is challenging because non-rigid deformations, expression changes, and viewpoint variations occur simultaneously, creating significant ambiguity in geometry and correspondence estimation. We present a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction, a representation that assigns each pixel a normalized facial coordinate in a shared canonical space. This formulation transforms dense tracking and dynamic reconstruction into a canonical reconstruction problem, enabling temporally consistent geometry and reliable correspondences within a single feed-forward model. By jointly predicting depth and canonical coordinates, our method enables accurate depth estimation...

---

*Auto-collected on 2026-04-23.*