SAEVerbalizer: Generating Natural-Language Explanations for Sparse Autoencoder Features

Forum topic · 小凯 · 2026-08-15

Summary

Sparse autoencoders (SAEs) extract numerous features from large language model (LLM) representations, but explaining these features has traditionally relied on external observation of model behavior, which yields superficial explanations and is computationally inefficient at scale. SAEVerbalizer, a framework by Meng et al. (arXiv:2608.13538), addresses both limitations by injecting SAE decoder directions directly into an LLM's representations and fine-tuning the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the verbalizer explains SAE features directly from their decoder directions. Experiments show the learned verbalization capability generalizes to unseen features, transfers across independently trained SAE dictionaries, and extends to SAE features from different LLMs via a lightweight adapter. Intervention experiments demonstrate that injecting multiple directions produces explanations combining their meanings, while reversing individual directions causes corresponding meaning shifts. This work offers a scalable, behavior-independent approach to interpreting SAE features in LLMs.

SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features

Field: NLP Authors: Weihan Meng, Hongzhu Guo, Yi Jing, Dewen Liu, Zijun Yao, Xiaozhi Wang, Lei Hou, Juanzi Li Published: 2026-08-13 arXiv: 2608.13538

Abstract

Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale.

We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations.

Key Findings

Generalization: The learned verbalization capability generalizes to unseen features.
Transfer: It transfers across separately trained SAE dictionaries.
Portability: With a lightweight adapter, it extends to SAE features from different LLMs.
Interventions: Injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.

Significance

SAEVerbalizer offers a scalable, behavior-independent approach to interpreting SAE features, removing the need to collect large amounts of behavioral evidence and reducing the risk of superficial feature explanations.

---

*Auto-collected on 2026-08-15.*