Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Staging on MRI

Forum topic · 小凯 · 2026-09-16

Summary

This paper investigates two common pitfalls in deep learning for Alzheimer's disease (AD) staging on structural MRI: anatomically irrelevant model attention and label leakage in multimodal fusion. Using a lightweight slice-based encoder (ResNet18 plus a one-layer Transformer) on 1,075 baseline T1-weighted ADNI-1 scans, the authors use FastSurfer segmentations as an anatomical reference, training YOLOv8 models that localize Alzheimer-relevant structures with mAP_50 above 0.96, while Grad-CAM reveals that image-only classifiers often focus on skull, orbits, and background. They also adapt a CLIP-style image-tabular contrastive framework over ADNIMERGE variables organized along a label-leakage spectrum: fusion with cognitive scores yields 87.3% three-class accuracy (treated as a leakage-driven upper bound), while fusion with regional volumes yields 73.0%. Notably, the contrastive target shapes what image encoders learn: on MCI vs. CN, image-only heads reach 52.4% or 73.8% depending on alignment with cognitive scores vs. volumes, with no tabular input at inference. Restricting input to individualized medial temporal lobe crops improves image-only three-class accuracy from 58.7% to 65.1%. arXiv: 2609.15888.

Paper Overview

Field: Computer Vision Authors: Paul-Gabriel Nicolae, Irina Georgiana Mocanu Released: 2026-09-14 arXiv: 2609.15888

Introduction

Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevant regions, and multimodal models that add clinical tables frequently rely on variables that were used to assign the diagnostic label in the first place. This paper studies both issues with a deliberately lightweight slice-based encoder (ResNet18 with a one-layer Transformer over slices) on 1,075 baseline T1-weighted scans from ADNI-1.

Key Findings

1. Anatomical grounding: FastSurfer segmentations are used as an anatomical reference. YOLOv8 models trained on segmentation-derived labels localize Alzheimer-relevant structures with mAP_50 above 0.96, while a Grad-CAM comparison shows that the image-only classifier frequently attends to the skull, orbits, and background.

2. Leakage-aware multimodal contrastive learning: A CLIP-style image-tabular contrastive framework is adapted, organizing ADNIMERGE variables along a label-leakage spectrum. Fusion with cognitive scores produces 87.3% three-class accuracy — treated as a leakage-driven upper bound rather than an imaging result — while fusion with regional volumes produces 73.0%.

3. Contrastive target shapes representation learning: On MCI vs. CN, the pure image head reaches 52.4% when the encoder is aligned with cognitive scores, versus 73.8% when aligned with volumes — despite using no tabular input at inference.

4. Region-restricted input: Restricting input to individualized crops of the medial temporal lobe improves image-only three-class accuracy from 58.7% to 65.1%.

Caveats

All results come from single runs on small balanced test sets. The authors report confidence intervals and protocol differences that prevent direct comparison with previously published numbers.