Overview
Field: Computer Vision Authors: Ninghui Xu, Fabio Tosi, Lihui Wang Published: 2025-04-17 arXiv: 2504.13101
Abstract (Translated)
Conventional frame-based cameras capture rich contextual information but suffer from limited temporal resolution and motion blur in dynamic scenes. Event cameras offer an alternative visual representation with higher dynamic range free from such limitations. The complementary characteristics of the two modalities make event-frame asymmetric stereo promising for reliable 3D perception under fast motion and challenging illumination. However, the modality gap often leads to marginalization of domain-specific cues essential for cross-modal stereo matching.
The paper introduces Bi-CMPStereo, a novel bidirectional cross-modal prompting framework that fully exploits semantic and structural features from both domains for robust matching. The approach learns finely aligned stereo representations in a target canonical space and integrates complementary representations by projecting each modality into both the event and frame domains. Extensive experiments demonstrate that the method significantly outperforms existing state-of-the-art approaches in accuracy and generalization.
Key Contributions
- A bidirectional cross-modal prompting framework that leverages cues from both event and frame domains, avoiding the marginalization of domain-specific information.
- Learning of finely aligned stereo representations within a target canonical space.
- Integration of complementary modal representations by projecting each modality into both the event domain and the frame domain.
- Strong empirical improvements in accuracy and generalization over prior state-of-the-art cross-modal stereo matching methods.
*Auto-collected on 2026-04-18.*