BERT-LER: Explainable Transformer Models for Clinical Prediction on Structured EHR Data

Forum topic · 小凯 · 2026-08-22

Summary

A forum post introduces BERT-LER, a BERT-style model for encoding electronic health record (EHR) timelines, pre-trained and fine-tuned on a de-identified EHR dataset covering 75 million patients. The model encodes laboratory test results as discrete tokens while preserving graded information through percentile-based binning, and applies Integrated Gradients to produce token-level attributions relative to the input medical event sequence. The authors evaluate BERT-LER on the public EHRShot benchmark and a real-world asthma severity progression study. Results show competitive predictive performance, generally exceeding publicly available baseline models on lab-related tasks, with attributions that align with clinically known risk factors. The architecture and explainability approach are broadly applicable across therapeutic areas and clinical prediction tasks. The paper is available on arXiv as 2608.20315, authored by Jun Ni Du, Lukas Adamek, Maxim Kryukov, and Flavio Dormont.

Paper Overview

Field: ML Authors: Jun Ni Du, Lukas Adamek, Maxim Kryukov, Flavio Dormont Published: 2026-08-22 arXiv: 2608.20315

Abstract (translated)

Prediction models on structured electronic health records (EHR) remain central to medical machine learning, yet few approaches jointly emphasize quantitative laboratory information and interpretability relative to the input medical events. This paper proposes BERT-LER, a BERT-style model for encoding EHR timelines, pre-trained and fine-tuned on a de-identified EHR dataset spanning 75 million patients.

BERT-LER encodes laboratory test results as discrete tokens while preserving graded information via percentile-based binning, and leverages Integrated Gradients to deliver token-level attributions grounded in the input EHR sequence.

The model is evaluated on the public EHRShot benchmark and on an asthma severity progression study based on real-world data. BERT-LER achieves competitive predictive performance, generally exceeding publicly available benchmark models on lab-related tasks, and its attributions align with clinically established risk factors. The architecture and interpretability method can be applied to many therapeutic areas and prediction tasks.

--- *Auto-collected on 2026-08-22*

BERT-LER: Explainable Transformer Models for Clinical Prediction on Structured EHR Data

Summary

Paper Overview

Abstract (translated)

Tags