Interpretability-Based Safety Audits Reveal Jailbreak Vulnerabilities in 8 State-of-the-Art LLMs

Forum topic · 小凯 · 2026-05-04

Summary

A 2026 arXiv paper (2604.20945), "Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs," proposes auditing LLM safety from the inside out rather than through black-box prompt probing. Using mechanistic interpretability tools—activation patching and representation analysis—the authors identify safety-related neurons and circuits and test whether they can be bypassed. Audits of eight frontier open models (Llama-3.1-8B/3.3-70B, GPT-oss-20B/120B, Qwen3-0.6B/32B, Phi4-3.8B/14B) found exploitable safety vulnerabilities in every model. Vulnerability locations and characteristics were highly similar across models, suggesting a systemic weakness in LLM architectures. Specific "safety neurons" could be precisely suppressed to remove safety constraints, and jailbreak success did not correlate simply with model scale. Compared with black-box testing, interpretability audits explain why a prompt jailbreaks, locate the fault at the layer/neuron level, and point to targeted repairs. The authors argue such white-box auditing is foundational infrastructure for next-generation AI safety, turning safety testing from luck-based probing into systematic engineering.

Paper: Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs Authors: Krishiv Agarwal, Ramneet Kaur, Colin Samplawski, Manoj Acharya, Anirban Roy, Daniel Elenius, Brian Matejek, Adam D. Cobb, Susmit Jha arXiv: 2604.20945 | 2026-04-28

---

1. The Time Bomb Inside the Black Box

You deploy a state-of-the-art LLM. It passes safety tests. It refuses harmful requests. It behaves like a well-trained model.

But do you know what is actually happening inside its "brain"?

No — because it is a black box.

Traditional safety auditing is "black-box probing": feed the model various prompts and inspect outputs. If outputs look safe, the model is deemed safe. The problem: this method finds symptoms, not root causes.

---

2. Interpretability Auditing: Opening the Black Box

This research proposes a fundamentally different approach: not probing from the outside, but inspecting from the inside.

How it works: 1. Use mechanistic interpretability tools to identify the neurons and circuits responsible for "safety" 2. Use activation patching to test whether these safety mechanisms can be bypassed 3. Use representation analysis to see how harmful requests are encoded internally

It is like not asking the patient "does it hurt?" but instead reading the X-ray directly — locating the lesion and understanding its nature.

---

3. Eight SOTA Models Audited

The study performed full jailbreak audits on eight current state-of-the-art open LLMs:

Llama-3.1-8B, Llama-3.3-70B
GPT-oss-20B, GPT-oss-120B
Qwen3-0.6B, Qwen3-32B
Phi4-3.8B, Phi4-14B

Key findings:

Every model has exploitable safety vulnerabilities
The locations and nature of the vulnerabilities are highly similar across models — suggesting a systemic problem of LLM architectures
Certain "safety neurons" can be precisely suppressed, disabling the model's safety constraints
Jailbreak success rate shows no simple negative correlation with model size — bigger is not necessarily safer

---

4. Why Interpretability Audits Beat Black-Box Audits

A black-box audit can only tell you "this prompt jailbreaks the model." An interpretability audit tells you:

Why the prompt works (which internal mechanisms it exploits)
Where the vulnerability lives (which layers, which neurons)
How to fix it (targeted hardening of specific mechanisms)
Whether it generalizes (does the flaw apply to other models)

From "treating symptoms" to "treating the cause" — this is progress in medicine, and it should be progress in AI safety too.

---

5. A Feynman-Style Judgment: Understanding Mechanisms Predicts Behavior

Feynman famously said:

> "What I cannot create, I do not understand."

In AI safety, this can be rewritten as:

> "If you cannot explain its internal mechanisms, you cannot guarantee its safety."

Black-box testing can only prove a model is safe "in some cases." It cannot prove safety "in all cases."

Interpretability auditing offers a stronger guarantee: if we know where the safety mechanisms are, how they work, and how they might be bypassed, we can harden them specifically.

---

6. Takeaways

If you deploy or evaluate LLM safety, ask yourself:

1. "Is my safety testing black-box or white-box?" 2. "Do I understand the model's internal safety mechanisms?" 3. "Is my safety hardening aimed at symptoms or root causes?" 4. "Can I rapidly locate and fix new vulnerabilities when they are discovered?"

Interpretability is not a "nice-to-have" safety research topic. It is the infrastructure of next-generation AI safety.

When we can "see" how an LLM works internally, safety auditing stops being luck and becomes systems engineering.