Video Generative Models as Geometry Learner: GeoNeXt for Depth and Surface Normal Estimation

Forum topic · 小凯 · 2026-09-01

Summary

GeoNeXt is a new computer vision paper (arXiv:2608.28549) by Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, and Jiankang Deng that repurposes pretrained video generative models as a unified, data-efficient framework for geometry estimation. Existing generative geometry approaches adapt image diffusion models either by training task-specific models for depth and surface normal estimation independently—missing the intrinsic correlation between geometric targets—or by jointly fine-tuning modified diffusion backbones, which requires large labeled datasets. GeoNeXt instead formulates geometry estimation as a next-frames prediction task, inheriting the naturally structured knowledge and richer priors of video models while jointly modeling image-to-geometry and geometry-to-image targets. Extensive experiments show state-of-the-art zero-shot monomial depth and surface normal estimation across diverse datasets, outperforming task-specific and unified generative baselines with substantially less training data, and matching or exceeding discriminative methods trained with over 100x more data on several benchmarks.

Video Generative Models as Geometry Learner

Field: Computer Vision Authors: Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, Jiankang Deng arXiv: 2608.28549

Overview

Recent generative approaches to geometry estimation adapt pretrained image diffusion models and treat the task as image-conditioned generation. Using off-the-shelf image diffusion models, they either:

1. Train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity to exploit the intrinsic correlation between these geometric targets, or 2. Jointly fine-tune modified image diffusion backbones (e.g., with altered self-attention), which typically demands substantial labeled data.

To overcome these limitations in a principled way, the authors repurpose pretrained video generative models as a unified and data-efficient framework for geometry estimation, innovatively formulated as a next-frames prediction task.

GeoNeXt

The proposed method, GeoNeXt, inherits the naturally structured knowledge and richer priors of video generative models, and further adapts them to jointly model image and geometry targets (image ↔ geometry), enabling more efficient and effective geometric learning.

Results

Evaluated on zero-shot monocular depth estimation and surface normal estimation across diverse datasets.
Outperforms prior task-specific and unified generative competitors while using substantially less training data.
Comparable to state-of-the-art discriminative methods trained with over 100x more data, and even outperforms them on several benchmarks.