Summary
A paper by Jin Mu and Guanhua Chen (arXiv:2608.27397) introduces CAST (Concept-guided Artifact Suppression Tuning), a framework for making clinical language models more robust and auditable. Clinical models often achieve high in-hospital accuracy but fail under deployment shifts because they rely on note-specific artifacts such as templates, separators, and boilerplate rather than genuine patient state. CAST uses Sparse Autoencoders (SAEs) on intermediate Transformer activations to expose sparse, human-auditable features, labels SAE latents via an LLM-assisted interpretation pipeline with ICD-10 retrieval constraints, suppresses verified artifact latents through residual subtraction during fine-tuning, and generates post-hoc per-concept attributions for auditing model decisions. On MIMIC-IV discharge-note mortality prediction, CAST outperforms fine-tuned encoder baselines and remains competitive with strong LLM baselines, while producing feature-level audit trails showing the clinical concepts supporting each prediction and the artifact concepts suppressed during training.
Paper Overview
- Field: NLP
- Authors: Jin Mu, Guanhua Chen
- arXiv: 2608.27397
Summary
Clinical language models can achieve strong in-hospital accuracy yet fail under deployment shifts because they exploit note-specific artifacts (e.g., templates, separators, boilerplate) that do not reflect patient state.
The authors propose CAST (Concept-guided Artifact Suppression Tuning), an SAE-based framework for auditable clinical text classification:
- Uses Sparse Autoencoders to expose sparse, human-auditable features from intermediate Transformer activations.
- Labels SAE latents with an LLM-assisted interpretation pipeline and ICD-10 retrieval constraints.
- Suppresses verified artifact latents via residual subtraction during fine-tuning.
- Provides post-hoc per-concept attributions for auditing model decisions.
Results
On MIMIC-IV discharge-note mortality prediction, CAST improves over its corresponding fine-tuned encoder baselines and remains competitive with strong LLM baselines, while producing feature-level audit trails that document the clinical concepts supporting each prediction and the artifact concepts suppressed during training.
---
Paper link: https://arxiv.org/abs/2608.27397
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178634245