Weight-Space Programming for Auditing Clinical Decision Networks

Forum topic · 小凯 · 2026-11-04

Summary

A new arXiv paper (2611.05341) by Humza Nusrat and Omar Nusrat explores whether frontier language models can audit and repair small neural networks using only a clinical protocol and raw weights, an ability called weight-space programming. Networks with 433 to 4,801 parameters were trained to screen head-and-neck radiotherapy planning requests against four rules, with flaws planted via training data: confounds hiding each rule, a 3 Gy shift in the cord threshold, and a hidden trigger. Claude Opus 5.5 audited each network blind—no tools, data, or code execution—and every claim and repair was validated on about 420,000 requests. The model identified the broken rule in 16 of 18 flawed networks (95% CI 67–97%), including all 15 with confounds or threshold shifts, repairing each by changing a median of only 3 parameters. Fine-tuning on protocol-labeled requests also fixed every network but altered nearly every parameter. The model cleared 2 of 5 healthy networks and found 1 of 3 hidden triggers. The authors conclude weight-space programming can locate and repair flaws that silence inputs or shift thresholds, but cannot yet certify network correctness.

Paper Overview

Field: Machine Learning
Authors: Humza Nusrat, Omar Nusrat
Published: 2026-11-03
arXiv: 2611.05341

Abstract

Machine learning models in radiotherapy are commissioned by testing them on cases with known answers, which may not reveal a shortcut that the test cases share with the training data. Frontier language models have recently been reported to read and write the raw weights of small neural networks, an ability called weight-space programming. The purpose of this study was to determine whether such a model, given only a clinical protocol and a network's weights, can audit and repair the network.

Methodology

Networks with 433 to 4,801 parameters were trained to screen head-and-neck planning requests against four rules.
Flaws were planted through the training data: confounds that hid each rule, a 3 Gy shift in the cord threshold, and a hidden trigger.
Claude Opus 5.5 audited each network blind, without tools, data, or code execution.
Every claim and repair was tested on about 420,000 requests.

Key Results

The language model named the broken rule in 16 of 18 flawed networks (95% CI 67% to 97%), including all 15 with a confound or a threshold shift.
It repaired each of the 16 networks by changing a median of 3 parameters.
Fine-tuning on protocol-labeled requests also repaired every network, but changed nearly every parameter.
Each confound and shift it found was visible in single units, and a simple statistic chosen for each flaw type found the same flaws.
It cleared 2 of 5 healthy networks without being forced and found 1 of 3 triggers.
Testing against the protocol detected every flaw, and black-box testing found the confounds and the shift faster.

Conclusion

The results indicate that, in small networks, weight-space programming can locate and repair flaws that silence an input or shift a threshold, but cannot yet certify that a network is correct.

---

*Auto-collected on 2026-11-05*