ModMap: Crossmodal Feature Mapping with Cross-View Modulation for 3D Anomaly Detection

Forum topic · 小凯 · 2026-04-04

Summary

ModMap (arXiv:2504.01262) is a natively multiview and multimodal framework for 3D anomaly detection and segmentation, proposed by Alex Costanzino, Pierluigi Zama Ramirez, and Giuseppe Lisanti (April 2025). Unlike prior methods that process camera views independently, ModMap adapts the crossmodal feature mapping paradigm to learn feature mappings across both modalities and views, explicitly modeling view-dependent relationships through feature-wise modulation. The authors introduce a cross-view training strategy that exploits all possible view combinations, enabling effective anomaly scoring via multiview ensembling and aggregation. To handle high-resolution 3D data, they train and publicly release a foundational depth encoder tailored to industrial datasets. Experiments on SiM3D—the first benchmark offering a multiview, multimodal setup for 3D anomaly detection and segmentation—show that ModMap outperforms previous methods by a large margin, achieving state-of-the-art results. The framework is relevant to industrial visual inspection and computer vision research.

Paper Overview

Field: Computer Vision (CV)
Authors: Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti
Published: 2025-04-01
arXiv: 2504.01262

Abstract

We present ModMap, a natively multiview and multimodal framework for 3D anomaly detection and segmentation. Unlike existing methods that process views independently, our method draws inspiration from the crossmodal feature mapping paradigm to learn to map features across both modalities and views, while explicitly modelling view-dependent relationships through feature-wise modulation. We introduce a cross-view training strategy that leverages all possible view combinations, enabling effective anomaly scoring through multiview ensembling and aggregation. To process high-resolution 3D data, we train and publicly release a foundational depth encoder tailored to industrial datasets. Experiments on SiM3D, a recent benchmark that introduces the first multiview and multimodal setup for 3D anomaly detection and segmentation, show that ModMap outperforms previous methods by a large margin, achieving state-of-the-art performance.

Key Contributions

1. Natively multiview, multimodal design: Instead of treating each camera view independently, ModMap learns to map features across modalities (RGB and depth) and across views. 2. Feature-wise modulation: View-dependent relationships are explicitly modeled through feature-level modulation. 3. Cross-view training strategy: Training leverages all possible view combinations, and anomaly scoring benefits from multiview ensembling and aggregation. 4. Public foundational depth encoder: A depth encoder tailored to industrial datasets is released to handle high-resolution 3D data. 5. State-of-the-art on SiM3D: The method significantly outperforms prior approaches on the first multiview, multimodal 3D anomaly detection benchmark.

*Auto-collected on 2026-04-04.*