EI: Early Intervention Framework for Multimodal Medical Imaging-Based Disease Recognition (arXiv 2503.13833)

Forum topic · 小凯 · 2026-03-19

Summary

A 2025 arXiv paper (2503.13833) by Qijie Wei, Hailan Lin, and Xirong Li proposes an Early Intervention (EI) framework for multimodal medical imaging-based disease recognition. Current methods face two challenges: the prevailing 'fusion after unimodal image embedding' paradigm fails to fully exploit complementary and correlated information across modalities, and the scarcity of labeled multimodal medical images—combined with domain shift from natural images—limits the use of Vision Foundation Models (VFMs). EI treats one modality as the target and the rest as references, using high-level semantic tokens from the reference as intervention tokens to steer the target modality's embedding at an early stage. The authors also introduce Mixture of Rank-varying Low-rank Adaptations (MoR), a parameter-efficient fine-tuning method with rank-varying adapters and weight-relaxed routing for VFM adaptation. Experiments on three public datasets—retinal disease, skin lesion, and knee abnormality classification—show effectiveness over competing baselines.

Paper Overview

Research Area: Computer Vision Authors: Qijie Wei, Hailan Lin, Xirong Li Published: 2025-03-18 arXiv: 2503.13833

Abstract (English)

Current methods for multimodal medical imaging based disease recognition face two major challenges. First, the prevailing "fusion after unimodal image embedding" paradigm cannot fully leverage the complementary and correlated information in the multimodal data. Second, the scarcity of labeled multimodal medical images, coupled with their significant domain shift from natural images, hinders the use of cutting-edge Vision Foundation Models (VFMs) for medical image embedding.

To jointly address these challenges, the authors propose a novel Early Intervention (EI) framework:

One modality is treated as the target, and the remaining modalities serve as references.
High-level semantic tokens from the reference modality act as intervention tokens, steering the target modality's embedding process at an early stage.

Additionally, the paper introduces MoR (Mixture of Rank-varying low-rank Adaptations), a parameter-efficient fine-tuning method that adapts VFMs using a set of low-rank adapters with varying ranks combined with a weight-relaxed router.

Experimental Results

Extensive experiments on three public datasets validate the proposed method against a variety of competing baselines:

Retinal disease classification
Skin lesion classification
Knee abnormality classification

Key Takeaways

EI shifts multimodal fusion from late-stage (post-embedding) to early-stage intervention, better exploiting cross-modal complementary information.
MoR enables efficient VFM adaptation despite limited labeled medical data and domain shift.
Consistent improvements over baselines across three distinct medical imaging tasks.

---

*Auto-collected on 2026-03-19*