Overview
- Field: Machine Learning
- Authors: Nikolai Bolik, Lennart Stöpler, Artur Andrzejak
- Published: 2026-08-11
- arXiv: 2608.11197
Abstract (translated)
Shani et al. (2026) show that LLM representations broadly recover human category boundaries, but fail to reflect fine-grained typicality structure. Their analysis uses cosine similarity over dense model representations. The authors revisit this approach using overlap over active sparse autoencoder (SAE) latent sets as a more interpretable similarity measure.
They first verify that this set-level measure is meaningful: SAE latent sets can recover union-like compositional structure in controlled toy models and induce semantically coherent neighborhoods in natural text. Extending the human-concepts analysis to SAE set similarities, they find that SAE activation sets do not recover human category boundaries or within-category typicality more faithfully than dense embeddings or residual-stream states — instead, they track the model's internal similarity structure.
To probe this gap further, the authors study active latent sets under controlled semantic modifications, revealing a significant mismatch between human judgments of concept change and changes in SAE active sets. They interpret this as evidence that, outside idealized settings, SAE features do not compose via simple bag-of-features semantics.
Original Abstract
Shani et al. (2026) show that LLM representations broadly recover human category boundaries, while failing to reflect fine-grained typicality structure. Their analysis uses cosine similarity over dense model representations. We revisit their approach using overlap over active sparse autoencoder (SAE) latent sets as a more interpretable similarity measure. We first verify that this set-level measure is meaningful: SAE latent sets can recover union-like compositional structure in controlled toy models and induce semantically coherent neighborhoods in natural text. Extending the human-concepts analysis to SAE set similarities, we find that SAE activation sets do not recover human category boundaries or within-category typicality more faithfully than dense embeddings or residual-stream states...
--- *Auto-collected on 2026-08-13*