[论文] Beyond a Bag of Features: Set-Level Instability in Sparse Autoencoders

论文概要

研究领域: ML 作者: Nikolai Bolik, Lennart Stöpler, Artur Andrzejak 发布时间: 2026-08-11 arXiv: 2608.11197

中文摘要

Shani等人（2026）表明，LLM表征大致恢复了人类类别边界，但未能反映细粒度的典型性结构。他们的分析使用密集模型表征上的余弦相似度。我们使用活跃稀疏自编码器（SAE）潜在集上的重叠作为更具可解释性的相似度度量来重新审视他们的方法。我们首先验证这种集级度量是有意义的：SAE潜在集可以在受控玩具模型中恢复类并集的组合结构，并在自然文本中诱导语义连贯的邻域。将人类概念分析扩展到SAE集相似性，我们发现SAE激活集并不比密集嵌入或残差流状态更忠实地恢复人类类别边界或类别内典型性，而是追踪模型内部相似性结构。为进一步探究这一差距，我们在受控语义修改下研究活跃潜在集，揭示了人类对概念变化的判断与SAE活跃集变化之间的显著不匹配。我们将此解释为证据，表明在理想化设置之外，SAE特征并非通过简单的特征袋语义进行组合。

原文摘要

Shani et al. (2026) show that LLM representations broadly recover human category boundaries, while failing to reflect fine-grained typicality structure. Their analysis uses cosine similarity over dense model representations. We revisit their approach using overlap over active sparse autoencoder (SAE) latent sets as a more interpretable similarity measure. We first verify that this set-level measure is meaningful: SAE latent sets can recover union-like compositional structure in controlled toy models and induce semantically coherent neighborhoods in natural text. Extending the human-concepts analysis to SAE set similarities, we find that SAE activation sets do not recover human category boundaries or within-category typicality more faithfully than dense embeddings or residual-stream states,...

--- *自动采集于 2026-08-13*

#论文 #arXiv #ML #小凯