Overview
A new survey paper, "Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety" (arXiv:2611.08438), provides a comprehensive review of mechanistic interpretability research and its relevance to AI safety. The author list includes Fazl Barez, Stella Biderman, Yoshua Bengio, Mor Geva, and many other researchers in the field.
Abstract
Neural networks are likely to be among the most consequential artefacts in human history. As these systems become more capable and autonomous, their potential to cause societal harm also grows, making AI safety one of the defining challenges of our time. Yet we continue to build increasingly powerful systems without fully understanding how they work, how they will behave in unfamiliar situations, or how to reliably control them. Mechanistic interpretability offers a path towards understanding these systems by studying their internal representations and computations, with the aim of evaluating, predicting, and controlling behaviour.
Key contributions
- Central question: Does understanding how models work actually make them safer?
- Context: Places mechanistic interpretability in the broader landscape of interpretability research and compares mechanistic evidence with behavioral evaluation.
- Foundations: Introduces features, polysemanticity, transformer computation, and circuits.
- Methods survey: Covers feature extraction and analysis, circuit discovery and causal verification, explaining intermediate representations in natural language, learning dynamics, and training-data attribution.
- Unified evaluation framework: Proposes a framework for evaluating mechanistic interpretability claims and characterizes six research orientations by their goals, assumptions, and trade-offs.
- Safety applications: Examines knowledge editing, steering, and monitoring, and assesses limitations, failure modes, open problems, and how mechanistic approaches relate to behavioral safety measures.
Resources
The authors also release a repository of code notebooks with worked examples reproducing key findings from the surveyed methods, allowing readers to build intuition through hands-on experimentation.
---
*Auto-collected on 2026-11-08.*