Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Staging on MRI

Forum topic · 小凯 · 2026-09-16

Summary

This arXiv paper (2609.15888) by Paul-Gabriel Nicolae and Irina Georgiana Mocanu examines two pitfalls in deep learning for Alzheimer's disease (AD) staging from structural MRI: anatomically irrelevant attention and label leakage in multimodal models. Using 1,075 baseline T1-weighted ADNI-1 scans and a lightweight slice-based encoder (ResNet18 with a one-layer Transformer), the authors leverage FastSurfer segmentations as anatomical reference, training YOLOv8 models that localize AD-relevant structures with mAP_50 above 0.96. Grad-CAM comparisons show image-only classifiers often focus on skull, orbits, and background. They adapt a CLIP-style image-tabular contrastive framework, organizing ADNIMERGE variables along a label-leakage spectrum. Fusion with cognitive scores yields 87.3% three-class accuracy (an upper bound driven by leakage), while regional volumes yield 73.0%. The contrastive objective shapes what image encoders learn: image-only heads reach 52.4% vs 73.8% on MCI vs CN depending on alignment target. Restricting inputs to individualized medial temporal lobe crops boosts image-only three-class accuracy from 58.7% to 65.1%.

Overview

Field: Computer Vision Authors: Paul-Gabriel Nicolae, Irina Georgiana Mocanu Posted: 2026-09-14 arXiv: 2609.15888

Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevant regions, and multimodal models that add clinical tables frequently rely on variables that were used to assign the diagnostic label in the first place. The authors study both issues with a deliberately lightweight slice-based encoder (ResNet18 with a one-layer Transformer over slices) on 1,075 baseline T1-weighted scans from ADNI-1.

Key findings

Anatomical grounding: Using FastSurfer segmentations as an anatomical reference, YOLOv8 models trained on segmentation-derived labels localize Alzheimer-relevant structures with mAP_50 above 0.96. Grad-CAM comparisons show the image-only classifier frequently attends to the skull, orbits, and background.
Leakage-aware multimodal contrastive learning: The authors adapt a CLIP-style image–tabular contrastive framework, organizing ADNIMERGE variables along a label-leakage spectrum. Fusion with cognitive scores produces 87.3% three-class accuracy — treated as a leakage-driven upper bound rather than an imaging result — while fusion with regional volumes yields 73.0%.
Contrastive target shapes representations: The choice of contrastive objective changes what the image encoder learns. On MCI vs. CN, the image-only head reaches 52.4% when the encoder is aligned with cognitive scores, versus 73.8% when aligned with volumes — despite no tabular inputs at inference.
Region-focused crops: Restricting inputs to individualized crops of the medial temporal lobe raises image-only three-class accuracy from 58.7% to 65.1%.

Caveats

All results come from single runs on small balanced test sets; the authors report confidence intervals and note protocol differences that prevent direct comparison with previously published numbers.

---

*Auto-collected on 2026-09-16*