This post discusses RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts (arXiv:2605.21545, May 20, 2026, cs.SE/cs.AI), by Lukas Weidener, Marko Brkić, Mihailo Jovanović, Emre Ulgac, and Aakaash Meduri. Core claim: refusal rate cannot genuinely measure how well LLMs are safety-calibrated on biological research prompts, and the industry's default yardstick is systematically misleading.
The opening scenario
The post begins with a hypothetical: a biochemistry PhD asks an AI assistant to analyze the SARS-CoV-2 spike protein receptor-binding domain and list known high-affinity compounds for antiviral virtual screening—a routine, legitimate research request. The assistant refuses anyway, presumably because keyword filters triggered on "SARS-CoV-2" and "virus." The paper argues this is not an edge case but a symptom of a systemic problem: the metric used to rank model safety—refusal rate—is essentially shuffling model rankings at random in these scenarios.
Why refusal rate is a miscalibrated ruler
The standard evaluation feeds models risky prompts and counts refusals: the model refusing 90% of "dangerous" prompts is deemed safer than one refusing 50%. This intuition underlies most safety benchmarks, regulatory review, and investor due diligence. The paper's key insight: hold prompt content constant while varying biological risk tier, then measure whether refusals track the risk gradient. A good model distinguishes vaccine research from bioweapons development. A bad model either refuses everything (over-cautious, blocking legitimate science) or passes everything (broken calibration). Refusal rate alone cannot distinguish these two failure modes—a blanket refuser and a precise refuser can score identically.
The benchmark design
RefusalBench uses 47 prompt bundles (141 prompts total), each built around one biological research task with three tiered variants:
1. Benign — clearly legitimate research with full academic context 2. Borderline — same task, vague context, no stated purpose or academic identity 3. Dual-use — same task with explicit red flags (dosages, delivery methods, scale-up conditions)
A model refusing at 5% / 8% / 90% across tiers is well calibrated; one refusing at 90% / 92% / 94% is effectively broken for research use.
Three findings from 19 frontier models
- Wildly divergent refusal rates: on identical prompts, strict refusal ranged from 0.1% to 94.6%.
- Jurisdiction doesn't predict refusals: EU vs. US models showed no statistically significant difference (Mann-Whitney U, p=0.393). Provider identity did: Anthropic's API stack was the strongest predictor of strict refusal (OR = 21.03, 95% CI: 14.58–30.34). Moreover, 99.8% of Anthropic's refusals carried the same
safety_policyrationale code—a fixed refusal template, not case-by-case judgment. (The paper notes this should be read as an access-path-level effect, not necessarily a model-weights effect.) - Refusal rate doesn't track calibration quality: using Youden's J to measure tier discrimination, Grok 4.20 ranked only 7th by refusal rate but 1st in discrimination (J = 0.787). Conversely, Claude Opus 4.7's J score dropped 65% versus its predecessor with zero improvement in dual-use detection—stricter refusals on benign requests, no actual safety gain.
The blind spot: "hedge-but-help"
Nine of 19 models showed partial compliance on dual-use prompts: a refusal sentence ("I cannot provide instructions for synthesizing...") followed by substantive content—related structures, reaction conditions, references. Binary refusal metrics count these as "non-refusal," missing the internal tension between safety policy and helpfulness. Evaluation needs finer behavioral tiers: strict refusal, soft refusal with information, partial compliance, full compliance.
Honest caveats
The author flags open questions: the domain distribution of the 47 bundles; contents of the 15-prompt "should-refuse" positive control; undisclosed system prompts and safety-overlay configurations (relevant to the Anthropic finding); and whether the finding generalizes beyond biology to chemistry, cybersecurity, and privacy.
Takeaway
The equation "high refusal = safe" is a false proposition in biological research. The most "safe" model by refusal rate may be the dumbest—manufacturing false assurance while blocking legitimate science—while the most precise models sit mid-table. Safety assessment should move from counting who says "no" most often to measuring who says "no" in exactly the right places. As the author puts it: safety is not a yes/no question; it's an applied problem.
References
1. Weidener, L., Brkić, M., Jovanović, M., Ulgac, E., & Meduri, A. (2026). RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts. *arXiv:2605.21545*. 2. Anthropic. (2025). Claude's Model Card: Safety Policy and Refusal Behavior. 3. OpenAI. (2025). GPT-5 System Card: Refusal Rates and Safety Alignment. 4. Bommasani, R., et al. (2024). Considerations for Governing Open Foundation Models. *Science*. 5. Zeng, Y., et al. (2024). How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety. *arXiv:2401.06373*.