LAMAE: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning

Forum topic · 小凯 · 2026-09-15

Summary

This post introduces LAMAE (Latent-Attention Masked Autoencoders), a multimodal, structure-aware masked autoencoder for cardiovascular diagnosis presented in arXiv paper 2609.12035. Unlike most medical foundation models that remain modality-specific and only combine ECG, echocardiography, chest X-rays, and clinical variables during fine-tuning, LAMAE jointly learns patient-level representations during self-supervised pretraining. A shared latent-attention module operates over an exam-view-organism hierarchy, exchanging information directly in latent space, which enables aggregation of variable observations and graceful handling of missing modalities. Pretrained on over 1.2 million MIMIC-IV hospital admissions, LAMAE outperforms modality-specific pretraining as well as strong contrastive learning and vision-language baselines on multimodal inpatient tasks including in-hospital mortality prediction, ICD-10 and DRG coding, and length-of-stay estimation, while remaining competitive on unimodal tasks. Notably, the gains persist even when only a single modality is available at test time, showing that jointly modeling intra- and inter-modality structure yields more robust and transferable representations.

Paper Overview

Field: Machine Learning Authors: Andrea Agostini, Simon Böhi, Moritz Vandenhirtz, Samuel Ruiperez-Campillo, Max Krähenmann, Silke Mühlstedt, Irene Cannistraci, Ece Özkan Elsen, Julia E. Vogt, Thomas M. Sutter Published: 2026-09-15 arXiv: 2609.12035

Abstract

Cardiovascular diagnosis rests on integrating complementary modalities, like ECG, echocardiography, chest radiographs, and clinical variables, each capturing distinct but correlated aspects of cardiac physiology. Yet most medical foundation models remain modality-specific, combining modalities only for finetuning or post-training. This discards the cross-modal evidence clinicians naturally integrate and ignores the structure within each modality. The authors introduce Latent-Attention Masked Autoencoders (LAMAE), a multimodal, structure-aware masked autoencoder that jointly learns patient-level representations during self-supervised pretraining.

Rather than fusing modalities post hoc, LAMAE exchanges information directly in the latent space through a shared latent-attention module operating over an exam–view–entity hierarchy. This design enables aggregation of variable observations and graceful handling of missing modalities.

Key Results

Pretrained on over 1.2 million MIMIC-IV hospital admissions.
Outperforms modality-specific pretraining as well as strong contrastive learning and vision-language baselines on multimodal inpatient tasks.
Evaluated tasks include in-hospital mortality prediction, ICD-10 and DRG coding, and length-of-stay estimation.
Remains competitive on unimodal tasks.
Gains persist even when only a single modality is available at test time, indicating that jointly modeling intra- and inter-modality structure produces more robust and transferable representations.

---

*Auto-collected on 2026-09-15*