[论文] User Embeddings are Superpositions of Interpretable Behavioral Modes

论文概要

研究领域: ML 作者: Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen, Keivan Majidi, Eddie K. H. Ng, Chris Porter, Qi Zhao, Jia Geng, Changmao Li, Brandon R. Feng 发布时间: 2026-11-05 arXiv: 2611.08487

中文摘要

用户嵌入是现代推荐、个性化和行为建模系统的核心组件。尽管这些嵌入能够紧凑地编码丰富的行为信息，但它们本质上是稠密且不透明的，限制了可解释性。为弥补这一差距，我们认为用户嵌入可以被理解为可解释行为模式的稀疏叠加。对于线性嵌入，这种结构直接由用户行为的生成模型预测，并通过叠加假设近似扩展到深度学习表示。叠加模式可以通过稀疏字典学习方法（如稀疏自编码器 SAE）自然恢复，恢复的模式可以用自然语言自动命名和查询。我们在合成数据上对照已知真值验证了模式恢复效果，SAE 的表现优于经典稀疏基线方法，包括在 Transformer 编码的嵌入上。随后我们将该方法应用于来自两个真实数据集的经典和基于 Transformer 的用户嵌入。我们还展示了相应的自然语言标签如何实现在从未训练过文本的嵌入上进行零样本受众检索和特征刻画。该框架是事后分析的且与模型无关，不需要对上游嵌入系统进行任何修改。

原文摘要

User embeddings are central components of modern recommendation, personalization, and behavioral modeling systems. Though these embeddings can compactly encode rich behavioral information, they are inherently dense and opaque, limiting interpretability. To address this gap, we argue that user embeddings can be understood as sparse superpositions of interpretable behavioral modes. This structure is directly predicted by generative models of user behavior for linear embeddings and extends approximately to deep-learning representations by the superposition hypothesis. Superimposed modes can naturally be recovered by sparse dictionary learning methods, such as sparse autoencoders (SAEs), and recovered modes can be automatically named and queried in natural language. We validate mode recovery a...

#论文 #arXiv #ML #小凯