[论文] Would an LLM Match into a Residency? Uncertainty, Reasoning and Image ...

论文概要

研究领域: NLP 作者: Miguel Fernandez-de-Retana, Unai Zulaika, Ruben Sanchez-Corcuera, Aritz Bilbao-Jayo, Aitor Almeida 发布时间: 2026-10-22 arXiv: 2610.31586

中文摘要

医学考试准确性是LLM临床使用的常见证据，但医疗系统需要了解更多：模型能否识别自己可能出错、医学微调是否改善这种能力、正确答案是否基于可靠临床推理、是否真的使用了图像。在西班牙MIR住院医师考试上评估四个8-27B开放模型：三个较大模型可排入约前600-3000名，仍有41-43个专业可选。弃权永远不划算——最不确定的答案正确率也超过25%盈亏平衡点。医学微调以降低准确性为代价减少过度自信。13-38%的正确答案以临床上不可靠的理由拒绝干扰项，移除图像后准确性不变。

原文摘要

Accuracy on medical examinations is common evidence for clinical use of language models, yet a health system running an open model needs to know more: whether the model recognizes when it is likely wrong, whether medical fine-tuning improves this, whether correct answers rest on sound clinical reasoning, or whether it actually uses provided images. We examine four open models of 8 to 27B parameters on the Spanish MIR residency examination. The three larger models would rank among roughly the first 600 to 3,000 candidates. Abstention never pays because even their least confident answers are right more often than the 25% break-even. Medical fine-tuning reduced overconfidence at the cost of accuracy. 13-38% of correct answers rejected a distractor for a clinically unsound reason, and removing...

#论文 #arXiv #NLP #小凯