Less Context, Better Geometry: Masked Geometric Encoder for Robust 3D Vision

Forum topic · 小凯 · 2026-10-07

Summary

A new paper on arXiv (2610.06813) introduces the Masked Geometric Encoder (MGE), a method for learning robust geometry representations in 3D foundation models. While current 3D foundation models enable fast 3D reconstruction and camera calibration via spatial priors, their all-to-all global attention incurs quadratic complexity and allows unconstrained cross-view interactions to propagate unreliable evidence from occluded or geometrically distant views. MGE strategically drops frame tokens from global attention during training and distills knowledge from a pretrained full-context teacher, producing intrinsically richer per-frame representations without performance degradation. Experiments show significant gains under occlusion and duplicated-view scenarios while maintaining strong results on standard benchmarks. The authors further propose an Anchor-Guided Adaptive token merging technique that preserves representative anchor frames and merges redundant tokens from other views, achieving inference speedups with higher reconstruction quality compared to other efficient inference methods, especially in limited-view settings.

Paper Overview

Field: 3D Vision Authors: Zhimin Shao, Xijun Liu, Zhaoliang Zhang, Yutao Tang, Abhay Yadav, Rama Chellappa, Cheng Peng Published: 2026-10-05 arXiv: 2610.06813

Summary

Recent advances in 3D foundation models enable fast 3D reconstruction and camera calibration by leveraging 3D priors learned from massive spatial data. However, their all-to-all global attention design leads to quadratic complexity, limiting long-sequence inference. Moreover, unconstrained cross-view interaction can propagate unreliable evidence from occluded views or views that are visually similar but geometrically distant.

This paper introduces the Masked Geometric Encoder (MGE), which promotes learning of robust geometric representations under incomplete cross-view context. During training, MGE strategically drops frame tokens from global attention and distills from a pretrained full-context teacher. This enables the model to learn intrinsically richer per-frame representations while providing sufficient intermediate supervision to avoid performance degradation.

Key Findings

Extensive experiments show MGE delivers significantly stronger performance under occluded and "duplicated" views, while maintaining high performance on standard benchmarks.
Richer frame representations also enable more effective token reduction at inference time.
The authors develop an Anchor-Guided Adaptive token merging technique that preserves representative anchor frames while jointly merging redundant tokens from the remaining views.
Compared with other efficient inference methods, the approach achieves inference acceleration while consistently maintaining higher reconstruction quality, particularly in limited-view settings.

Original Abstract

MGE strategically drops frame tokens from global attention during training and distills from a pretrained full-context teacher, learning richer per-frame representations. An Anchor-Guided Adaptive token merging technique preserves anchor frames while merging redundant tokens for efficient inference.

Links: arXiv:2610.06813

--- *Auto-collected on 2026-10-07*

Less Context, Better Geometry: Masked Geometric Encoder for Robust 3D Vision

Summary

Paper Overview

Summary

Key Findings

Original Abstract

Tags