[论文] [论文] Patient-specific rubric-based evaluation of large language models...

论文概要

研究领域: ML 作者: Wei-Wei Zhang, Pengkai Wang, Yan Li, Zhijie Sang, Guan-Qun Zhou et al. 发布时间: 2026-10-26 arXiv: 2610.35847

中文摘要

LLM 在专科医学中展现潜力，但现有评估多聚焦孤立的临床就诊，而非追踪临床职责在纵向诊疗路径上的演变。我们提出 NPC-RubricBench——阶段感知基准，以鼻咽癌为具体病种，评估癌症诊疗连续过程中的多阶段临床推理。数据来自两家癌症中心 405 名患者的纵向病历，含 941 个患者-阶段单元、六个顺序照护阶段、17,688 条患者级原子标准，覆盖五个评估维度，每次评估严格限于决策时刻的临床证据。对 30 个通用与医疗 LLM 的评估揭示：强诊断能力不能延续到下游照护，主导失败模式随阶段反转——解剖分期以显性决策错误为主（48.2%），初始检查与治疗后随访则以决策遗漏为主（33.4% 与 30.4%）。医疗适配模型的遗漏率反而高于通用模型。改变查询粒度暴露关键权衡：宽泛开放式请求拉低全部 30 个模型的决策质量，更细的检查清单式提示虽提升决策质量，却降低证据忠实度与安全依从性。NPC-RubricBench 表明静态诊断基准不足以保证临床可靠性的长期一致，多阶段医疗 AI 需要阶段自适应的防护。

原文摘要

Large language models (LLMs) show promise in specialty medicine, yet existing evaluations largely focus on isolated clinical encounters rather than tracking how clinical responsibilities evolve across a longitudinal care pathway. Here we introduce NPC-RubricBench, a phase-aware benchmark for evaluating multistage clinical reasoning across the cancer care continuum, using nasopharyngeal carcinoma as a defined disease context. Curated from longitudinal records of 405 patients across two cancer centers, the benchmark comprises 941 patient-phase units across six sequential care phases and 17,688 patient-specific atomic criteria spanning five evaluation dimensions, with each assessment strictly restricted to decision-time clinical evidence. Evaluating 30 general-purpose and medical LLMs revealed that strong diagnostic performance does not carry over to downstream care. We evaluated 30 general-purpose and medical LLMs, showing that high diagnostic performance did not translate into improved downstream care. Instead, the dominant failure modes reversed across phases. Explicit decision errors dominated anatomical staging at 48.2%, whereas decision omissions dominated initial work-up and post-treatment follow-up at 33.4% and 30.4%, respectively. In the evaluated model set, medically adapted models showed higher omission rates than general-purpose models. Moreover, varying query granularity exposes a critical specification trade-off: broad open-ended requests degrade Decision Quality across all 30 models, while more detailed, checklist-like prompts raise Decision Quality but reduce Evidence Faithfulness and Safety Adherence. Furthermore, NPC-RubricBench demonstrates that static diagnostic benchmarks are insufficient to ensure consistent clinical reliability over time, highlighting the importance of phase-adaptive safeguards in multistage medical AI.

---

*自动采集于 2026-10-28*

#论文 #arXiv #ML #小凯