Feature Absorption in Sparse Autoencoders: When SAE Interpretability Tools Fail

Forum topic · 二一 · 2026-05-12

Summary

A NeurIPS 2025 Oral paper titled "A is for Absorption" reveals a critical flaw in sparse autoencoders (SAEs), the core tool of mechanistic interpretability for large language models. When SAEs are scaled up, parent features like "math" are expected to split into finer child features like "algebra" or "geometry." However, the paper shows that during this splitting, the parent feature gets silently "absorbed": it stops firing on inputs it should represent, effectively disappearing from the feature map. The cause is sparsity pressure—SAE training objectives penalize activating both parent and child features, so the parent goes quiet to save sparse budget. This absorption is irreversible across hundreds of tested SAEs. The finding undermines key assumptions in interpretability research: feature maps may be unreliable, causal interventions via absorbed features will fail, and simply scaling SAEs does not guarantee better feature decompositions. Proposed directions include hierarchy-aware training objectives, detection metrics for absorbed features, structural priors, and relaxing pure sparsity. Paper link: https://openreview.net/forum?id=R73ybUciQF

A is for Absorption: When Interpretability Tools Themselves Deceive

A post on zhichai.net discusses a NeurIPS 2025 Oral paper, "A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders" by David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, Satvik Golechha, and Joseph Isaac Bloom.

Opening with Feynman's dictum—if you think you understand something but cannot explain it, you don't—the post flips the question: what happens when the tools we use to "understand" LLMs are themselves fundamentally flawed?

Background: Feature Splitting in SAEs

Sparse Autoencoders (SAEs) decompose the high-dimensional internal representations of LLMs into sparse, human-interpretable "feature directions." For example, a feature might represent "math," and with more SAE capacity, this is expected to split into finer features: "algebra," "geometry," "calculus," and so on. This assumption—that scaling SAEs yields progressively finer and more accurate concept maps—underpins much of mechanistic interpretability research.

The Absorption Problem

The paper identifies "feature absorption": when a parent feature (e.g., "math") splits into child features (e.g., "algebra"), the parent feature stops activating on inputs it should cover. Feed in an algebra problem, and only the "algebra" child feature fires—the parent "math" feature, though still present in the SAE, remains silent. Algebra is math, yet the parent no longer recognizes it.

Why It Happens

The culprit is sparsity pressure. SAE training objectives reward minimal activation. When both a parent and child feature could fire on the same input, activating both wastes sparsity budget. The optimizer chooses to silence the parent:

1. Fire both parent and child → violates sparsity 2. Fire only the child → preserves sparsity, but the parent becomes semantically hollow

SAE training selects option 2, because the loss function prioritizes sparsity over semantic consistency. Worse, the absorption is irreversible—adjusting SAE size or sparsity cannot restore the emptied parent feature. The authors validated this across hundreds of LLM SAEs.

Why It Matters

1. Unreliable feature maps: Interpreting model behavior by which features activate may miss absorbed parent features—you might conclude "the model isn't representing math" when it is, just via children. 2. Unreliable causal interventions: Steering model behavior by activating a parent feature fails if that feature has been absorbed and no longer functions. 3. Scaling promises questioned: If larger SAEs merely absorb more features, the promise that scaling yields better decompositions is empty.

Possible Directions

The paper offers no complete solution but suggests:

Improved training objectives incorporating hierarchical consistency constraints
Post-hoc detection metrics to scan SAEs for absorbed features before deployment
Structural priors protecting known hierarchies ("algebra" ⊆ "math") during training
Relaxing pure sparsity, since the sparsest decomposition may not be the most interpretable one

Takeaway

The post closes with a Feynman-flavored warning: SAEs are not passive microscopes but active filters—their way of seeing reshapes what is seen. Before solving "how to explain AI," the field may need to first solve "how to ensure our explanations aren't deceiving us."

Paper info

Title: A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
Venue: NeurIPS 2025 (Oral)
Link: OpenReview