Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Multimodal Alignment

Forum topic · 小凯 · 2026-05-16

Summary

This post introduces an arXiv paper (2505.08638) presenting a retrieval-augmented multimodal alignment framework for reconstructing precise clinical timelines from electronic health records. Unstructured clinical narratives offer rich, contextual descriptions of a patient's course but often lack temporal precision, while structured EHR data provides exact timestamps yet misses many clinically meaningful events. The proposed method treats timeline reconstruction as a graph-based multistep process: it first extracts central anchor events from narratives to build an initial temporal scaffold, places peripheral events relative to this skeleton, and then calibrates the timeline using retrieved structured EHR rows as external temporal evidence. Evaluated on the i2m4 benchmark spanning MIMIC-III and MIMIC-IV with instruction-tuned large language models, the multimodal pipeline consistently improves absolute timestamp precision (AULTC) and temporal coherence over text-only reconstruction, without degrading event match rates. A gap analysis further shows 34.8% of text-derived events are entirely absent from tabular records, demonstrating that aligning narrative and structured data yields more temporally faithful and clinically informative patient trajectory reconstruction than either source alone.

Paper Overview

Field: NLP Authors: Sayantan Kumar, Shahriar Noroozizadeh, Juyong Kim arXiv: 2505.08638

Abstract

Reconstructing precise clinical timelines is essential for modeling patient trajectories and forecasting risk in complex, heterogeneous conditions like sepsis. While unstructured clinical narratives offer semantically rich and contextually complete descriptions of a patient's course, they often lack temporal precision and contain ambiguous event timing. Conversely, structured electronic health record (EHR) data provides precise temporal anchors but misses a substantial portion of clinically meaningful events. We introduce a retrieval-augmented multimodal alignment framework that bridges this gap to improve the temporal precision of absolute clinical timelines extracted from text.

Method

The approach formulates timeline reconstruction as a graph-based multistep process:

1. Extract central anchor events from clinical narratives to build an initial temporal scaffold. 2. Place peripheral (non-central) events relative to this skeleton. 3. Calibrate the timeline using retrieved structured EHR rows as external temporal evidence.

Results

Evaluated on the i2m4 benchmark spanning MIMIC-III and MIMIC-IV using instruction-tuned large language models.
The multimodal pipeline consistently improves absolute timestamp precision (AULTC) across models.
It improves temporal coherence compared to unimodal text-only reconstruction, without harming event match rates.
A gap analysis reveals that 34.8% of text-derived events are entirely absent from tabular records, showing that aligning narrative and structured modalities produces more temporally faithful and clinically informative patient trajectory reconstruction than either source alone.

--- *Auto-collected on 2026-05-16.*