[论文] Weight-space programming for auditing clinical decision networks

论文概要

研究领域: ML 作者: Humza Nusrat, Omar Nusrat 发布时间: 2026-11-03 arXiv: 2611.05341

中文摘要

放疗中的机器学习模型通过在已知答案的病例上测试来验收，但这可能无法揭示训练数据与测试病例共享的捷径。近期有报道称前沿语言模型可以读写小型神经网络的原始权重，这种能力被称为「权重空间编程」。本研究旨在确定：仅给一份临床协议和网络的权重，这类模型能否审计并修复该网络。我们训练了参数从 433 到 4801 的网络，用于按四条规则筛查头颈部放疗计划申请。缺陷通过训练数据植入：掩盖每条规则的混杂因素、脊髓阈值的 3 Gy 偏移，以及一个隐藏触发器。Claude Opus 5.5 在盲审下审计每个网络（无工具、无数据、无代码执行），每个声明和修复都在约 42 万份申请上测试。语言模型在 18 个缺陷网络中命名出其中 16 个的破损规则（95% CI 67%~97%），包括全部 15 个含混杂或阈值偏移的网络；它通过修改中位数仅 3 个参数修复了全部 16 个。在协议标注的请求上微调也能修复每个网络，但几乎改动了每个参数。它发现的每个混杂与偏移都可在单个单元中看见，每类缺陷选一个简单统计量也能找到同样的缺陷。它未被强迫就通过了 5 个健康网络中的 2 个，找到 3 个触发器中的 1 个。基于协议的测试能发现每个缺陷，黑盒测试更快找到混杂与偏移。结果表明：在小型网络中，权重空间编程可以定位并修复让输入沉默或移动阈值的缺陷，但还无法证明一个网络是正确的。

原文摘要

Machine learning models in radiotherapy are commissioned by testing them on cases with known answers, which may not reveal a shortcut that the test cases share with the training data. Frontier language models have recently been reported to read and write the raw weights of small neural networks, an ability called weight-space programming. The purpose of this study was to determine whether such a model, given only a clinical protocol and a network's weights, can audit and repair the network. Networks with 433 to 4,801 parameters were trained to screen head-and-neck planning requests against four rules. Flaws were planted through the training data: confounds that hid each rule, a 3 Gy shift in the cord threshold, and a hidden trigger. Claude Opus 5.5 audited each network blind, without tools, data or code execution, and every claim and repair was tested on about 420,000 requests. The language model named the broken rule in 16 of 18 flawed networks (95% CI 67% to 97%), including all 15 with a confound or a threshold shift. It repaired each of the 16 by changing a median of 3 parameters. Fine-tuning on protocol-labeled requests also repaired every network, but changed nearly every parameter. Each confound and shift it found was visible in single units, and a simple statistic chosen for each flaw type found the same flaws. It cleared 2 of 5 healthy networks without being forced and found 1 of 3 triggers. Testing against the protocol detected every flaw, and black-box testing found the confounds and the shift faster. The results indicate that, in small networks, weight-space programming can locate and repair flaws that silence an input or shift a threshold, but cannot yet certify that a network is correct.

---

*自动采集于 2026-11-05*

#论文 #arXiv #AI #小凯