NPC-RubricBench: Patient-Specific Rubric-Based Evaluation of LLMs Across the Cancer Care Continuum

Forum topic · 小凯 · 2026-10-27

Summary

NPC-RubricBench is a phase-aware benchmark for evaluating large language model (LLM) clinical reasoning across longitudinal cancer care, using nasopharyngeal carcinoma as the disease context. Built from longitudinal records of 405 patients at two cancer centers, it contains 941 patient-phase units across six sequential care phases and 17,688 patient-specific atomic criteria spanning five evaluation dimensions, with each assessment restricted to decision-time clinical evidence. Evaluation of 30 general-purpose and medical LLMs shows that strong diagnostic performance does not carry over to downstream care, and dominant failure modes reverse across phases: explicit decision errors dominate anatomical staging (48.2%), while decision omissions dominate initial work-up (33.4%) and post-treatment follow-up (30.4%). Medically adapted models show higher omission rates than general-purpose models. Query granularity reveals a trade-off: broad open-ended prompts degrade decision quality across all 30 models, while checklist-style prompts improve decision quality but reduce evidence faithfulness and safety adherence. The benchmark argues static diagnostic benchmarks are insufficient and multistage medical AI needs phase-adaptive safeguards. arXiv: 2610.35847.

论文概要

领域: Machine Learning / Medical AI 作者: Wei-Wei Zhang, Pengkai Wang, Yan Li, Zhijie Sang, Guan-Qun Zhou et al. 发布时间: 2026-10-26 arXiv: 2610.35847

Abstract

Large language models (LLMs) show promise in specialty medicine, yet existing evaluations largely focus on isolated clinical encounters rather than tracking how clinical responsibilities evolve across a longitudinal care pathway. Here we introduce NPC-RubricBench, a phase-aware benchmark for evaluating multistage clinical reasoning across the cancer care continuum, using nasopharyngeal carcinoma as a defined disease context.

Benchmark Construction

Curated from longitudinal records of 405 patients across two cancer centers:

941 patient-phase units
6 sequential care phases
17,688 patient-specific atomic criteria across 5 evaluation dimensions
Each assessment is strictly restricted to decision-time clinical evidence

Key Findings

Evaluation of 30 general-purpose and medical LLMs revealed:

Strong diagnostic performance does not carry over to downstream care
Dominant failure modes reverse across phases:
Anatomical staging: explicit decision errors dominate at 48.2%
Initial work-up: decision omissions dominate at 33.4%
Post-treatment follow-up: decision omissions dominate at 30.4%
Medically adapted models showed higher omission rates than general-purpose models in the evaluated set

Query Granularity Trade-off

Broad open-ended requests degrade Decision Quality across all 30 models
Detailed, checklist-like prompts raise Decision Quality but reduce Evidence Faithfulness and Safety Adherence

Conclusion

NPC-RubricBench demonstrates that static diagnostic benchmarks are insufficient to ensure consistent clinical reliability over time, highlighting the importance of phase-adaptive safeguards in multistage medical AI.

---

*Collected automatically on 2026-10-28.*