Summary
This paper examines a reliability problem in multiview 3D consistency evaluation: standard metrics assume that scored images come from one static 3D scene, which often fails in novel view synthesis (NVS) and sparse-view reconstruction. Inputs or generated outputs may contain artifacts, outlier frames, repeated views, or random noise yet still receive high consistency scores. Reference-based metrics require ground truth, while ground-truth-free metrics like MEt3R rely on learned reconstruction backbones whose failure modes are poorly understood. The authors compare neural reconstruction priors with classical geometric verification, introduce a controlled robustness benchmark, and propose a parametric framework that decomposes neural metrics into backbone, residual, and aggregation components. This framework reproduces MEt3R and yields variants with up to 3x improved robustness. The analysis shows VGGT, MASt3R, DUSt3R, and Fast3R hallucinate dense geometry and cross-view support for irrelevant scenes, repeated images, and random noise. A COLMAP-based metric leveraging matching, registration, dense support, and reconstruction failures correlates with human judgments up to 4x better than MEt3R on real NVS outputs and a structured human study. Paper: arXiv 2505.14311.
Paper Overview
Field: Computer Vision
Authors: Soumava Paul, Prakhar Kaushik, Alan Yuille
arXiv: 2505.14311
Abstract (Translated)
Multiview 3D evaluation assumes that the images being scored are observations of one static 3D scene. This assumption can fail in novel view synthesis (NVS) and sparse-view reconstruction: inputs or generated outputs may contain artifacts, outlier frames, repeated views, or noise, yet still receive high 3D consistency scores. Existing reference-based metrics require ground truth, while ground-truth-free metrics such as MEt3R depend on learned reconstruction backbones whose failure modes are poorly characterized.
The authors study this reliability problem by comparing neural reconstruction priors with classical geometric verification. Contributions include:
- A controlled robustness benchmark for multiview 3D consistency evaluation.
- A parametric family of metrics that decomposes neural metrics into backbone, residual, and aggregation components. This framework reproduces MEt3R and derives variants with up to 3x improved robustness.
- An analysis showing that VGGT, MASt3R, DUSt3R, and Fast3R hallucinate dense geometry and cross-view support for irrelevant scenes, repeated images, and random noise.
- A COLMAP-based metric that uses matching, registration, dense support, and reconstruction failures as failure-aware consistency signals. On real NVS outputs and a structured human study, these metrics correlate with human judgments up to 4x better than MEt3R.
Links
- Paper: https://arxiv.org/abs/2505.14311
*Auto-collected 2026-05-20*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177620479