Paper Overview
- Field: NLP / Mechanistic Interpretability
- Authors: Weihan Meng, Hongzhu Guo, Yi Jing, Dewen Liu, Zijun Yao, Xiaozhi Wang, Lei Hou, Juanzi Li
- Published: 2026-08-13
- arXiv: 2608.13538
- Generalization to unseen features: The learned verbalization capability transfers to SAE features not seen during training.
- Cross-dictionary transfer: Verbalization transfers across independently trained SAE dictionaries.
- Cross-LLM transfer: With a lightweight adapter, the method extends to SAE features from different LLMs.
- Intervention experiments:
- Injecting multiple directions produces explanations that combine their individual meanings.
- Reversing an individual direction produces a corresponding meaning shift in the explanation.
Summary
Sparse autoencoders (SAEs) extract numerous features from large language model (LLM) representations, but explaining these features still relies primarily on external observation. This reliance produces superficial explanations inferred from observed model behavior and is computationally inefficient when collecting such behavioral evidence at scale.Approach
The authors introduce SAEVerbalizer, a framework that: 1. Injects SAE decoder directions into an LLM's representations. 2. Fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features.Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both the superficiality and scalability limitations of prior observation-based methods.
Key Findings
Source Abstract
> Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale. We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations. Our experiments show that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries, and, with a lightweight adapter, extends to SAE features from different LLMs. Intervention experiments show that injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.--- *Auto-collected on 2026-08-15*
#论文 #arXiv #NLP