Towards a Deterministic Math Solver for Clinical Language Models

论文概要

研究领域: ML
作者: Felipe Ocampo Osorio, Sebastián Andrés Cajas Ordoñez, Maximin Lange
发布时间: 2026-09-11
arXiv: 2509.05821

中文摘要

大语言模型在算术上不可靠，这对临床计算器是个问题，因为单个数值错误就会改变建议。标准做法是将每个计算器逐个硬编码为经验证函数。我们测试一种替代方案：模型不进行计算，而是编写特定案例的Python代码，由受限本地执行器作为确定性求解器运行，模型的任务简化为决定如何使用它。我们在MedCalc-Bench Verified（1,100案例，55个计算器）上评估这种Program-Solve接口，与直接模型算术和手写22个计算器库对比，使用Qwen2.5-7B和Qwen2.5-32B-AWQ，在审计基准公式与当前临床指南后标记55个中的16个存在版本、使用或系数问题。在提供公式和黄金变量且两种路径都读取完整记录的情况下，移交给求解器在7B上不是可靠优势(75.31%对72.02%，配对+3.29分，95%计算器聚类区间[-3.49, 10.38])，但在32B上是(90.53%对83.47%，+7.05 [0.47, 14.60]，明显大于零)。手写库在其440个支持案例上完全准确，但在其他地方弃权(总体40.0%)。因此添加执行器对某些开放权重模型帮助更大，即使在匹配的公式、变量和记录访问下，且无论如何都不能替代经验证公式或可靠变量提取。

原文摘要

Large language models are unreliable at arithmetic, which is a problem for clinical calculators where a single numerical error changes the recommendation. The standard response is to hardcode each calculator as a validated function, one at a time. We test an alternative: the model does not calculate. Instead, it writes case-specific Python that a restricted local executor runs as a deterministic solver, and the model's task reduces to deciding how to use it. We evaluate this Program-Solve interface on MedCalc-Bench Verified (1,100 cases, 55 calculators) against direct model arithmetic and a hand-written 22-calculator library, using Qwen2.5-7B and Qwen2.5-32B-AWQ, after auditing the benchmark's formulas against current clinical guidelines and flagging 16 of 55 with version, use or coefficie...

自动采集于 2026-09-12

#论文 #arXiv #ML #小凯