Can "AI" Be a Doctor? A Study of Empathy, Readability, and Alignment in Medical LLM Communication

Forum topic · 小凯 · 2026-04-24

Summary

This arXiv paper (2604.20791) by Mariano Barone, Francesco Di Serio, and Roberto Moio evaluates how well general-purpose and domain-specialized large language models align with clinical communication standards in healthcare. The study measures semantic fidelity, readability, and affective resonance across structured medical explanations and real physician-patient interactions. Baseline LLMs amplify negative affect compared to physicians (Very Negative: 43.14-45.10% vs. 37.25%), and larger models like GPT-5 and Claude produce markedly higher linguistic complexity (FKGL up to 16.91-17.60 vs. 11.47-12.50 for physician responses). Empathy-oriented prompting reduces extreme negativity and readability complexity (up to -6.87 FKGL points for GPT-5) but does not significantly improve semantic fidelity. Collaborative rewriting achieves the strongest alignment: rewritten outputs reach the highest semantic similarity to physician answers (up to 0.93 on average) while improving readability and reducing emotional extremes. Dual-stakeholder evaluation shows no model surpasses physicians on cognitive criteria, yet patients consistently prefer rewritten variants for clarity and emotional tone. The authors conclude LLMs work best as collaborative communication enhancers rather than replacements for clinical expertise.

Paper Overview

Field: NLP
Authors: Mariano Barone, Francesco Di Serio, Roberto Moio
Published: 2026-04-22
arXiv: 2604.20791

Abstract

Large Language Models (LLMs) are increasingly deployed in healthcare, yet their communicative alignment with clinical standards remains insufficiently quantified. The authors conduct a multidimensional evaluation of general-purpose and domain-specialized LLMs across structured medical explanations and real-world physician-patient interactions, analyzing semantic fidelity, readability, and affective resonance.

Key Findings

Affective amplification: Baseline models amplify affective polarity relative to physicians (Very Negative: 43.14-45.10% vs. 37.25%).
Linguistic complexity: Larger architectures such as GPT-5 and Claude produce substantially higher complexity (FKGL up to 16.91-17.60 vs. 11.47-12.50 in physician-authored responses).
Empathy-oriented prompting reduces extreme negativity and lowers grade-level complexity (up to -6.87 FKGL points for GPT-5), but does not significantly increase semantic fidelity.
Collaborative rewriting yields the strongest overall alignment. The rewriting configuration achieves the highest semantic similarity to physician answers (up to 0.93 on average) while consistently improving readability and reducing affective extremity.
Dual-stakeholder evaluation: No model surpasses physicians on cognitive criteria, while patients consistently prefer rewritten variants for clarity and emotional tone.

Conclusion

The findings suggest LLMs function most effectively as collaborative communication enhancers rather than substitutes for clinical expertise.

---

*Auto-collected on 2026-04-24*