Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs

Forum topic · 小凯 · 2026-05-04

Summary

This forum post discusses the paper "Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs" (arXiv: 2604.20945), which proposes auditing LLM safety from the inside out rather than through black-box probing. Using mechanistic interpretability tools—activation patching and representation analysis—the researchers audited eight state-of-the-art open-source models: Llama-3.1-8B, Llama-3.3-70B, GPT-oss-20B, GPT-oss-120B, Qwen3-0.6B, Qwen3-32B, Phi4-3.8B, and Phi4-14B. Key findings: all models contained exploitable safety vulnerabilities; vulnerabilities were highly similar across models, suggesting a systemic architectural issue; specific "safety neurons" could be precisely suppressed to remove safety constraints; and jailbreak success rates did not simply decrease with model size. Unlike black-box audits that only reveal symptoms, interpretability-based audits explain why a jailbreak works, where the flaw resides (which layers and neurons), how to fix it, and whether it generalizes. The author argues that interpretability is foundational infrastructure for next-generation AI safety, turning audits from guesswork into systematic engineering.

> Paper: Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs > Authors: Krishiv Agarwal, Ramneet Kaur, Colin Samplawski, Manoj Acharya, Anirban Roy, Daniel Elenius, Brian Matejek, Adam D. Cobb, Susmit Jha > arXiv: 2604.20945 | 2026-04-28

The Time Bomb Inside the Black Box

You deploy a state-of-the-art LLM. It passes safety tests. It refuses harmful requests. It behaves well. But do you know what is actually happening inside its "brain"? Traditional safety audits are black-box probes: feed the model various prompts and check the outputs. If outputs look safe, the model is deemed safe. The problem: this approach only finds *symptoms*, not the *disease*.

Interpretability Audits: Opening the Black Box

This research takes a fundamentally different approach: instead of probing from the outside, inspect from the inside:

1. Use mechanistic interpretability tools to identify the neurons and circuits responsible for "safety" 2. Use activation patching to test whether these safety mechanisms can be bypassed 3. Use representation analysis to see how harmful requests are encoded inside the model

It is like not asking the patient "does it hurt?" but reading the X-ray directly—locating the lesion and determining its nature.

Auditing 8 SOTA Models

The study conducted comprehensive jailbreak audits on eight state-of-the-art open-source LLMs:

Llama-3.1-8B, Llama-3.3-70B
GPT-oss-20B, GPT-oss-120B
Qwen3-0.6B, Qwen3-32B
Phi4-3.8B, Phi4-14B

Findings:

All models have exploitable safety vulnerabilities
The location and nature of the vulnerabilities are highly similar across models—hinting at a systemic problem of the LLM architecture itself
Certain "safety neurons" can be precisely suppressed, disabling the model's safety constraints
Jailbreak success rates show no simple negative correlation with model scale—bigger models are not necessarily safer

Why Interpretability Audits Beat Black-Box Audits

A black-box audit can only tell you "this prompt jailbreaks the model." An interpretability audit tells you:

Why the prompt works (which internal mechanisms it exploits)
Where the vulnerability lives (which layers, which neurons)
How to fix it (targeted hardening of specific mechanisms)
Whether it generalizes (does the flaw apply to other models?)

From "treating symptoms" to "treating the cause"—that is progress in medicine, and it should be progress in AI safety too.

A Feynman-Style Judgment: Understand the Mechanism to Predict Behavior

Feynman said:

> "What I cannot create, I do not understand."

In AI safety, this can be rewritten as:

> "If you cannot explain its internal mechanisms, you cannot guarantee its safety."

Black-box testing can only prove a model is safe "in some cases"—never "in all cases." Interpretability audits offer stronger guarantees: if we know where the safety mechanisms are, how they work, and how they might be bypassed, we can harden them in a targeted way.

Takeaways

If you deploy or evaluate LLM safety, ask yourself:

1. "Is my safety testing black-box or white-box?" 2. "Do I understand the safety mechanisms inside my model?" 3. "Is my safety hardening symptom-targeted or cause-targeted?" 4. "Can I quickly locate and fix new vulnerabilities when they are discovered?"

Interpretability is not a nice-to-have in safety research. It is the infrastructure of next-generation AI safety.

When we can "see" inside an LLM, safety auditing turns from luck into systems engineering.