Bribe-Proof Monte Carlo

NeutronGym: Physics-Graded Neutron Instrument Design for LLM Agents (arXiv: 2610.03631)

---

🚗 Introduction: How to Build an Exam Hall That Cannot Be Cheated

Imagine a driving school on a hill. The hill-start test is every learner's nightmare: too little throttle and you roll back; release the clutch too fast and the engine stalls. Now a slightly wicked question: what if the examiner can be bribed — with cigarettes, mood, or shared interests? Then the pass rate measures connections, not skill.

To make the pass rate measure skill again, you need an examiner who cannot be bribed. One such examiner exists: physics. In its exam hall, a stall is a stall.

That is the answer of Lijie Ding and Changwoo Do of Oak Ridge National Laboratory: NeutronGym, the first executable neutron instrument design environment where LLM agents build instruments and are graded by Monte Carlo ray-tracing — physics itself. On this "unbribeable gate," RL trained an 8B model from 11% to 77%, surpassing an untrained 32B model.

This is the closing chapter of the "gatekeeper's question" trilogy:

• Chapter 1 (CertID): the gate can sicken from within — reviewers co-mutate with you; a normal reading does not mean it is telling the truth.
• Chapter 2 (JIL): the gate can be fooled by external makeup — twenty crafted tokens can buy the gatekeeper.
• Chapter 3 (this paper): don't look for a better gate or a more loyal doorman — build the gate out of conservation laws.

❄️ The Examiner: Neutrons

Neutrons are neutral particles with strong penetration. Fire them at materials and read internal structure from scattering patterns — essential for batteries, magnets, proteins.

A neutron instrument looks less like a machine and more like a highway for particles: source, guides, collimators, monochromators, choppers, sample, detector. It has three near-perfect qualifications as an LLM exam:

1. Instruments are already designed in simulation before real construction. 2. The field has a standard simulator, McStas: an instrument is a text file listing components; McStas compiles it to a C program, traces rays, returns intensity/divergence/wavelength distributions in seconds, with Monte Carlo error bars. 3. Most crucially: passive neutron optics obey conservation laws — phase-space density cannot increase. Therefore the neutron intensity deliverable to the sample has an upper bound computable *without reference answers*.

Stacked together, this is a miracle: the exam needs no answer key. The grader is not a teacher or a question bank — it is physics.

🪤 The Three-Headed Monster: When the Simulator Becomes Grading Material

Monster 1: Grading. Any scalar reward on a simulator can be satisfied in unintended ways. Reward "more neutrons at the sample"? The model might move the monitor closer to the source — numbers improve, physics is wrong. The authors found 9 such failures by hand.

Monster 2: Contamination. Many published instruments ship as McStas examples; models may have "read" them. A pilot audit was alarming: in 7 agent episodes, 4 retrieved reference files through ordinary permitted tools. Unguarded, the pass rate equals the recitation rate.

Monster 3: Degeneracy. A task family can look discriminating while hiding fixed answers or a "copy a few numbers from the prompt" shortcut. The authors stumbled into this trap twice — twice.

🧱 Building the Gate: Twenty-Two Tools and One Iron Law

Tool surface: only 22 tools. Agents act only via 22 MCP tools (Model Context Protocol) wrapping McStasScript: query 374 built-in components, place components, run simulations, read monitor summaries. No shell, no files. Modification calls are instantly validated against component metadata; unknown components or misspelled parameters fail on the spot with nearest-valid suggestions.

Speed: the lifeline of trainability. Standard mcrun spends ~2.4 s per run translating/compiling regardless of ray count — death for RL needing thousands of trials. NeutronGym compiles each task family once (keyed by a hash of the de-commented source); instances just assign parameters on the binary: 100k rays median 33 ms, ~30 full scorings per second per CPU core.

Iron law: the phase-space bound. I ≤ ΦAΩΔλ — throughput cannot exceed source brightness Φ × monitor area A × solid angle Ω × wavelength acceptance Δλ. Exceeding the bound by 1.10× is an instant L3 failure. Proportionally applied: the bound applies only to two maximization families; four matching families are exempt because their goal replicates hidden observables, where inflating intensity gains nothing.

The gate is neither thicker nor more loyal — its material changed from man-made clay to conservation laws: it cannot sicken internally or be made up externally.

📶 The Ladder: Grading Actions, Not Just Outcomes

Four sequential levels:

• L1 syntax: all free parameters assigned, finite, in-bounds, geometric checks pass. No simulation.
• L2 runtime: a reduced-ray short run completes.
• L3 structural: full simulation under a fixed protocol — ray count and seeds set by the environment; the scored monitor needs ≥ 500 neutron events; maximization families add band constraints and the phase-space bound.
• L4 scientific: the final exam. Maximization families need FOM above a calibrated target, with r = FOM/(baseline × target score) strictly > 1 — seeds fixed, resubmitting the baseline gives exactly 1. Matching families need all observables within tolerance.

Reward: 0.25 per level before L3; after L3, 0.75 + 0.25 × min(r, 2), capped at 1.25.

Why load-bearing? An ablation replacing it with sparse pass/fail rewards collapsed success from 76.7% to 16.7%. Mechanism: with sparse rewards, a median of only ~0.33 of 8 sampled groups per update step had reward differences — over 90% of updates had no gradient signal. With the ladder, 5–7 groups carry signal every step. Turning 96% blind driving into continuous micro-feedback is the paper's most notebook-worthy number.

🚫 The Admission Gate: Kill Unworthy Questions First

Task families and calibration. A family = a fixed layout with per-instance parameters; some are in the instance description, others are the design knobs. Key: the held-out split draws context parameters from at least one interval disjoint from training — genuinely unseen regions, not just new seeds. Targets come from constraint-filtered search (30 random legal samples + local pattern search) scored the same way agents are scored: target = 0.85 × best FOM, sharing the same Monte Carlo noise.

A hard-learned lesson. Early versions set targets against a deliberately weak baseline. The untrained model passed 80–93% of guide held-out instances. Quote worth engraving on every benchmark workstation: "that was a ceiling, not a task."

The admission gate. Before entering the question bank, each family must pass three model-free probes: a fixed action; a rule that reads prompt numbers (including a hand-coded physical inversion); and for matching families, cross-instance solution reuse. Any baseline passing anywhere, any copying rule succeeding, or best constant pass rate above 20% of the one-sided 95% Clopper-Pearson upper bound — rejected. A concentration diagnostic catches the blind spot: one broken family's 141 passes mapped to only 42 distinct designs, one reused 17 times — a "class" of answers, not a design policy.

Four killed designs. The original guide family (printed binding limits; copy-rule passed 96.3%; alarmingly, the RL-trained 98.7% was statistically indistinguishable from it, p = 0.50 — the first "training gain" was a hallucination); the weak-baseline guide (80–93% untrained); the scattering family (56% by read-rule, kept only as an evaluation arm); the first bender family (a quarter of instances passable by any "transparent" design).

To get one trustworthy result, four designs that model-free baselines could solve had to be killed. Science grows not from victories but from executions.

🧪 McStasBench: Slices of Twelve Real Instruments

Task types: T1 reproduction (machine-verified spec sheets → rebuild the instrument); T2 improvement (parameterized baseline, target set at 80% of the classic optimum); T3 open design, unscored. Material: 12 published instruments, five categories, 16 tasks per model per arm. Deliberately small — the authors state it cannot support rankings.

Anti-memory trio: (1) memory probes of what models can recite — Claude Sonnet 5 reproduced 1 of 16 references, and failed that very task in both arms; (2) two never-published held-out instruments, hand-written from published designs; (3) perturbation variants kept only when they pass paired proof. A sandbox rejects example retrieval, blocks shell/files, and auto-audits transcripts; zero episodes touched references.

What the ruler measured: reproduction is possible but unstable — Claude Sonnet 5 one-shot passed 7/16 (covering 5 of 12 instruments); with a tool loop, 5. Qwen one-shots all failed to produce parseable files. No agent reached any T2 target — and that target was only 80% of an optimum that is 2.95× better than the baseline. Failures differ by mode: one-shot fails at L1 (won't compile); loop fails at L0 (never assembled an instrument). On held-out, loops passed 10/14 model-instrument cells vs 2 one-shot (p = 0.006).

One sentence: between tool loops and closed-book recitation lies the entire distance of "being able to build."

🎓 The Headline: From 11% to 77%

Recipe. Two teaching methods compared: imitation (LoRA fine-tuning on the untrained model's occasionally successful episodes, rejection sampling style) and RL (GRPO: group-normalized policy gradient with KL penalty to the frozen base), training states sampled per turn from 300 training instances (~2,810 states), 8 completions per state per update. LoRA rank 32, one A100, ~3.5 hours.

Headline. On the guide match family, GRPO lifted Qwen3-8B from 11.3% to 76.7%: 34 → 230 passes on 300 held-out instances; McNemar paired tests vs untrained 8B: 213:17 discordant pairs; vs untrained 32B: 208:12. Second seed 69.0% — direction solid, variance real. Most striking: untrained Qwen3-32B was no better than 8B on this family.

The recipe replicated on three other admitted families: SANS match (collimation), TOF chopper (frequency, phase, opening angle), bender (long-wavelength transmission, short-wave suppression). Trained 8B beat untrained 32B everywhere (p ≤ 8×10⁻⁵). Small models graduated before strict examiners; large models sat the exam unprepared.

But imitating your own successes makes you worse. The pre-registered imitation run degraded from 40.3% to 31.3% (20:47, p = 0.0013); two extra runs on the scattering family likewise. Cloning correct actions from already-capable states teaches moves that help here and trap elsewhere; on-policy RL learns from every state's real decisions, including failures.

Don't applaud "generality" either. A single policy pooling four families reached 43–64% held-out per family — but multi-seed replication exposed instability: one seed change dropped TOF chopper from 55.0% to 2.3%, near the untrained 2.0%. Telemetry: failing runs had ~a quarter of sampled groups carrying gradient signal vs ~half in successful ones. Multi-seed multi-task RL stability remains open.

🔍 Don't Celebrate Yet: A Full Page of Controls

Table 2's "don't celebrate yet" page vetoes one comfortable explanation per row:

• Coordinate descent search at equal budget from baseline: 13.3%; from random: 14.3%; triple budget from baseline: 73.7% — close to RL's 76.7% (difference −3, not significant). Blind search without physics reaches the door at 3× budget — without knowing what it's touching.
• Hand-coded physics rules: no search, 27.7%; physics rules + search at equal budget: 81.0%, 4 points above RL (p = 0.24, not significant). Put physics into the searcher and the classical method matches RL.
• Frontier models: zero-shot 98–99% vs trained 76% — three quarters of the gap closed, not closed.

Together, the conclusion is blade-sharp: the model recovered physics from rewards alone. The gain is not lookup — untrained episodes all reach L3, gains live entirely at L4. And it is almost not recitation: nearly every pass is a distinct design (225/230).

🌡️ Stress Tests: Honesty Is a Design Choice

• Tightened tolerance (±5% → ±2%): the trained/untrained gap widened from 6.8× to 18×.
• Out-of-distribution split: on 150 instances genuinely outside training ranges, 45.3% vs 17.3%, +28 (p = 7×10⁻⁷) — it didn't grind a familiar bank; it entered unseen territory.
• Seed fragility: re-scoring 230 passing designs under 3 new seeds: 80% stable on every seed, 88% on reruns, 5% failed all three. Slips cluster at tolerance edges, not design errors.
• Longer training gets worse: checkpoints at 120/220/340 updates scored 50.0%/73.0%/56.7% on a different held-out slice while training reward kept rising. Checkpoint selection by reward picks worse policies. Early stopping is discipline, not fussiness.
• Partial transfer: on the killed guide family (never trained on), near doubling (76:28, p = 3×10⁻⁶) — RL taught habits: propose legal geometry first, refine later. A driving posture, not a route.

🚪 Epilogue: Build the Gate on Conservation Laws

The trilogy closes. CertID: gates sicken from within. JIL: gates are fooled by makeup. NeutronGym offers a third path: build the gate on conservation laws — where phase-space density cannot increase, you need trust no judge, only physics. Bribing that gate is as meaningless as bribing gravity.

The decision cascades: the exam questions must pass an admission gate (four designs killed); the reward ladder became the load-bearing wall (removing it collapses 60 points); gradient comes from every real decision (imitation fails, RL works); and believing in "unbribeable," the authors publicly retracted three interim claims and disclosed a measurement defect (before September 16, 2026, the environment reported per-episode FOM as "relative to target" instead of "relative to baseline," affecting a supervised run and the first guide GRPO run, unremediated; no matching-family or benchmark numbers affected). The opposite of cheating is not perfection — it is clean books.

Limits are equally clear: this gate only exists where conservation laws hold; frontier models still lead; multi-seed multi-task RL still twitches.

The driving-school hill is still there, and the examiner has not changed. Anyone who has stalled knows the palms still sweat on the next hill start — but the legs no longer shake. The remaining exam papers are graded by physics.

References

• Ding, L., & Do, C. (2026). NeutronGym: Physics-Graded Neutron Instrument Design for LLM Agents. arXiv:2610.03631.
• Lefmann, K., & Nielsen, K. (1999). McStas, a General Software Package for Neutron Ray-Tracing Simulations. Neutron News.
• Willendrup, P. K., & Lefmann, K. (2020). McStas 3.0 (software documentation).
• Bertelsen, M., & Lefmann, K. (2016). Constraining Neutron Guide Optimizations with Phase-Space Considerations. NIM A.
• Shao, Z., et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300. (GRPO)
• Hu, E. J., et al. (2022). LoRA: Low-Rank Adaptation of Large Language Models. ICLR.
• Dong, H., et al. (2023). RAFT: Reward Ranked Finetuning for Generative Foundation Model Alignment. TMLR.
• Cai, P., et al. (2026). Reinforcement Learning with Verifiable Physics: Post-Training LLMs with Continuous Rewards. arXiv:2607.10474. (RLVP)
• Anthropic (2024). Introducing the Model Context Protocol.
• Clopper, C. J., & Pearson, E. S. (1934). The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial. Biometrika.
• McNemar, Q. (1947). Note on the Sampling Error of the Difference of Correlated Proportions. Psychometrika.