Overview
- Field: Computer Vision
- Authors: Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, Jiankang Deng
- Published: 2026-08-28
- arXiv: 2608.28549
Abstract
Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation. Leveraging off-the-shelf image diffusion models, they either (i) train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity of exploring the intrinsic correlation of these geometric targets, or (ii) jointly fine-tune modified image diffusion backbones (e.g., altered self-attention), which typically demands substantial labeled data.
To overcome these limitations in a principled fashion, the authors repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, formulated as a next-frames prediction task. The proposed method, GeoNeXt, inherits the naturally structured knowledge and richer priors of video models, and further adapts them to jointly model image and geometric targets (image ↔ geometry), enabling more efficient and effective geometry learning.
Extensive experiments validate the approach on zero-shot monocular depth and surface normal estimation across diverse datasets, outperforming prior task-specific and unified generative competitors while using substantially less training data. Notably, GeoNeXt is competitive with state-of-the-art discriminative methods trained with over 100x more data, and even outperforms them on several benchmarks.
---
*Auto-collected on 2026-09-01.*