[论文] Understanding AI from the Inside: The State of Mechanistic Interpretab...

论文概要

研究领域: AI
作者: Fazl Barez, Tung-Yu Wu, Hunar Batra, Gabriele Dominici, Florent Draye, Yanai Elazar, Damiano Fornasiere, Antía García, Abir Harrasse, Riley Harris, Sara Hooker, Michael Lan, Belinda Z. Li, Aaron Maiwald, Narmeen Fatimah Oozeer, Naomi Saphra, Or David Shafran, Adi Simhi, Robert Trager, Shriyash Kaustubh Upadhyay, Ziyu Yao, Yu Zhao, Shay B. Cohen, Mor Geva, Yoshua Bengio, Stella Biderman, Hassan Sajjad
发布时间: 2026-11-05
arXiv: 2611.08438

中文摘要

神经网络很可能成为人类历史上最具影响力的造物之一。随着这些系统能力更强、自主性更高，其造成社会危害的潜力也在增长，AI 安全成为我们这个时代最具决定性的挑战之一。然而，我们在尚未完全理解这些系统如何工作、在陌生情境下会如何表现、以及如何可靠控制它们的情况下，仍在持续构建越来越强大的系统。机械可解释性通过研究模型的内部表示和计算来理解这些系统，目标是评估、预测和控制其行为。我们调研了面向 AI 安全的机械可解释性，以回答一个问题：理解模型的工作原理是否真的能让它们更安全？我们将机械可解释性置于可解释性研究的大背景中，将其机械性证据与行为评估进行对比。我们介绍了机械可解释性的基础——包括特征、多语义性、Transformer 计算和回路——并调研了特征提取与分析、回路发现与因果验证、用自然语言解释中间表示、学习动态和训练数据归因等方法。超越对现有方法的综述，我们提出了评估机械可解释性主张的统一框架，并按目标、假设和取舍刻画了六种研究取向。最后，我们考察了机械理解如何支持 AI 安全——包括知识编辑、引导（steering）和监控——并评估其局限性、失效模式、开放问题以及与行为安全措施的关系。我们还发布了包含代码 notebook 的仓库，提供复现综述方法关键发现的完整示例，让读者通过动手实验建立直觉。

原文摘要

Neural networks are likely to be among the most consequential artefacts in human history. As these systems become more capable and autonomous, their potential to cause societal harm also grows, making AI safety one of the defining challenges of our time. Yet we continue to build increasingly powerful systems without fully understanding how they work, how they will behave in unfamiliar situations, or how to reliably control them. Mechanistic interpretability offers a path towards understanding these systems by studying their internal representations and computations, with the aim of evaluating, predicting, and controlling behaviour. We survey mechanistic interpretability for AI safety in order to answer the question: Does understanding how models work actually make them safer? We place mechanistic interpretability in the broader context of interpretability research and compare its mechanistic evidence with behavioural evaluation. We introduce the foundations of mechanistic interpretability, including features, polysemanticity, transformer computation, and circuits, and survey methods for feature extraction and analysis, circuit discovery and causal verification, explaining intermediate representations in natural language, learning dynamics, and training-data attribution. Beyond surveying existing methods, we propose a unified framework for evaluating mechanistic interpretability claims and characterise six research orientations by their goals, assumptions, and trade-offs. Finally, we examine how mechanistic understanding can support AI safety, including knowledge editing, steering, and monitoring, and assess its limitations, failure modes, open problems, and relationship to behavioural safety measures. We also release a repository of code notebooks with worked examples that reproduce key findings from surveyed methods, allowing readers to build intuition through hands-on experimentation.

自动采集于 2026-11-08

#论文 #arXiv #AI #小凯