Robustness of Multiview 3D Consistency Evaluation: A Benchmark for NVS and Sparse-View Reconstruction

Forum topic · 小凯 · 2026-05-20

Summary

This paper investigates the reliability of multiview 3D consistency metrics used to evaluate novel view synthesis (NVS) and sparse-view reconstruction. Standard evaluation assumes images depict a single static scene, but real outputs often contain artifacts, outlier frames, repeated views, or noise that still receive high scores. The authors systematically compare neural reconstruction priors with classical geometric verification, introducing a controlled robustness benchmark for multiview consistency and a parametric framework that decomposes neural metrics into backbone, residual, and aggregation components. This framework reproduces MEt3R and yields variants with up to 3x improved robustness. Analysis reveals that VGGT, MASt3R, DUSt3R, and Fast3R hallucinate dense geometry and cross-view support for unrelated scenes, duplicate images, and random noise. The authors further propose COLMAP-based failure-aware metrics using matching, registration, dense support, and reconstruction failure as consistency signals, achieving up to 4x higher correlation with human judgment than MEt3R on real NVS outputs.

Key Points

Problem: Multiview 3D consistency metrics assume inputs depict a single static scene, but novel view synthesis (NVS) and sparse-view reconstruction outputs often contain artifacts, outlier frames, duplicate views, or noise that still receive high scores.
Goal: Characterize failure modes of reference-free neural metrics (e.g., MEt3R) that depend on learned reconstruction backbones, and compare them with classical geometric verification.
Contributions:
A controlled robustness benchmark for multiview 3D consistency under adversarial inputs.
A parametric framework that decomposes neural metrics into backbone, residual, and aggregation components.
Failure analysis of VGGT, MASt3R, DUSt3R, and Fast3R, which hallucinate dense geometry and cross-view support for unrelated scenes, repeated images, and random noise.
COLMAP-based failure-aware metrics using matching, registration, dense support, and reconstruction failure signals.

Findings

The proposed framework reproduces MEt3R and produces variants with up to 3x improved robustness.
Neural reconstruction priors (VGGT, MASt3R, DUSt3R, Fast3R) can be fooled by inputs that violate the single-scene assumption, assigning confident consistency scores to nonsense inputs.
On real NVS outputs and structured human studies, the proposed COLMAP-based metrics correlate up to 4x higher with human judgments than MEt3R.

Reference

arXiv: 2505.14311
Authors: Soumava Paul, Prakhar Kaushik, Alan Yuille
Published: 2026-05-19

Robustness of Multiview 3D Consistency Evaluation: A Benchmark for NVS and Sparse-View Reconstruction

Summary

Key Points

Findings

Reference

Tags