[论文] Towards a Deterministic Math Solver for Clinical Language Models

论文概要

研究领域: cs.AI, cs.SE 作者: Felipe Ocampo Osorio, Sebastián Andrés Cajas Ordoñez, Maximin Lange, Rafi Al Attrach, Sahil Kapadia, Zakaria Laouabdia Sellami, Angelo Antonio Talio, Leo Anthony Celi 发布时间: 2026-09-13 arXiv: 2609.10728

中文摘要

大型语言模型在算术方面不可靠，这对临床计算器来说是个问题，因为单个数值错误就会改变建议。标准做法是将每个计算器逐个硬编码为验证函数。我们测试了一种替代方案：模型不计算，而是编写案例特定的 Python 代码，由受限的本地执行器作为确定性求解器运行，模型的任务简化为决定如何使用它。我们在 MedCalc-Bench Verified（1,100 个案例，55 个计算器）上评估这种 Program-Solve 接口，与直接模型算术和手写 22 个计算器库进行比较，使用 Qwen2.5-7B 和 Qwen2.5-32B-AWQ。在提供公式和黄金变量且两种路径都读取完整笔记的情况下，交给求解器在 7B 上不是可靠优势（75.31% 对 72.02%），但在 32B 上是（90.53% 对 83.47%）。手写库在其支持的 440 个案例上精确但其他方面弃权（总体 40.0%）。

原文摘要

Large language models are unreliable at arithmetic, which is a problem for clinical calculators where a single numerical error changes the recommendation. The standard response is to hardcode each calculator as a validated function, one at a time. We test an alternative: the model does not calculate. Instead, it writes case-specific Python that a restricted local executor runs as a deterministic solver, and the model's task reduces to deciding how to use it. We evaluate this Program-Solve interface on MedCalc-Bench Verified (1,100 cases, 55 calculators) against direct model arithmetic and a hand-written 22-calculator library, using Qwen2.5-7B and Qwen2.5-32B-AWQ, after auditing the benchmark's formulas against current clinical guidelines and flagging 16 of 55 with version, use or coefficient concerns. With formulas and gold variables supplied and both routes reading the whole note, handing off to the solver is not a reliable advantage at 7B (75.31% against 72.02%, a paired +3.29 points with a 95% calculator-cluster interval of [-3.49, 10.38]) but is one at 32B (90.53% against 83.47%, +7.05 [0.47, 14.60], clear of zero). The hand-written library is exact on its 440 supported cases but abstains elsewhere (40.0% overall). Adding an executor thus helps some open-weight models more than others even under matched formula, variable and note access, and is not a substitute for verified formulas or reliable variable extraction either way.

---

*自动采集于 2026-09-13*

#论文 #arXiv #AI #小凯