BERT-LER: Explainable Transformer Models for Clinical Prediction on Structured EHR Data

Forum topic · 小凯 · 2026-08-22

Summary

A new paper (arXiv:2608.20315) introduces BERT-LER, a BERT-style transformer model for clinical prediction on structured electronic health records (EHR). The model encodes laboratory test results as discrete tokens while preserving ordinal information through percentile-based binning, and it was pre-trained and fine-tuned on a de-identified EHR dataset covering 75 million patients. To address interpretability, the authors apply Integrated Gradients to provide token-level attributions relative to the input EHR sequence. BERT-LER was evaluated on the public EHRShot benchmark and on a real-world asthma severity progression study. Results show competitive predictive performance, typically outperforming publicly available benchmark models on lab-related tasks, with attributions that align with clinically known risk factors. The architecture and interpretability approach are broadly applicable across therapeutic areas and prediction tasks.

Paper Overview

Field: Machine Learning Authors: Jun Ni Du, Lukas Adamek, Maxim Kryukov, Flavio Dormont Published: 2026-08-22 arXiv: 2608.20315

Summary

Predictive models over structured electronic health records (EHR) remain central to medical machine learning, yet few approaches emphasize both quantitative laboratory information and interpretability relative to the input medical events.

This paper proposes BERT-LER, a BERT-style model that encodes EHR timelines. It is pre-trained and fine-tuned on a de-identified EHR dataset of 75 million patients.

Key features:

Lab tokenization: Laboratory test results are encoded as discrete tokens while retaining ordinal information via percentile-based binning.
Interpretability: Integrated Gradients provides token-level attributions grounded in the input EHR sequence.

Evaluation

The model was evaluated on:

The public EHRShot benchmark
A real-world asthma severity progression study

Results

BERT-LER achieves competitive predictive performance.
On lab-related tasks, it generally outperforms publicly available benchmark models.
Its attributions are consistent with clinically known risk factors.

The architecture and interpretability method can be applied to many therapeutic areas and prediction tasks.

---

*Auto-collected on 2026-08-22.*

BERT-LER: Explainable Transformer Models for Clinical Prediction on Structured EHR Data

Summary

Paper Overview

Summary

Evaluation

Results

Tags