English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

User Embeddings are Superpositions of Interpretable Behavioral Modes (arXiv 2611.08487)

Forum topic · 小凯 · 2026-11-06

Summary

This paper (arXiv 2611.08487) proposes that user embeddings in recommendation and personalization systems can be understood as sparse superpositions of interpretable behavioral modes. The authors show this structure is predicted by generative models of user behavior for linear embeddings and extends approximately to deep representations via the superposition hypothesis. Sparse dictionary learning methods, particularly sparse autoencoders (SAEs), can recover these modes, which can then be automatically named and queried in natural language. Mode recovery is validated against known ground truth on synthetic data, where SAEs outperform classical sparse baselines, including on Transformer-encoded embeddings. Applied to classic and Transformer-based user embeddings from two real datasets, the resulting natural-language labels enable zero-shot audience retrieval and profiling on embeddings never trained on text. The framework is post-hoc and model-agnostic, requiring no modification to upstream embedding systems.

Paper Overview

Field: ML Authors: Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen, Keivan Majidi, Eddie K. H. Ng, Chris Porter, Qi Zhao, Jia Geng, Changmao Li, Brandon R. Feng Published: 2026-11-05 arXiv: 2611.08487

Abstract

User embeddings are central components of modern recommendation, personalization, and behavioral modeling systems. Though these embeddings can compactly encode rich behavioral information, they are inherently dense and opaque, limiting interpretability. To address this gap, we argue that user embeddings can be understood as sparse superpositions of interpretable behavioral modes. This structure is directly predicted by generative models of user behavior for linear embeddings and extends approximately to deep-learning representations by the superposition hypothesis. Superimposed modes can naturally be recovered by sparse dictionary learning methods, such as sparse autoencoders (SAEs), and recovered modes can be automatically named and queried in natural language. We validate mode recovery against known ground truth on synthetic data, where SAEs outperform classical sparse baselines, including on Transformer-encoded embeddings. We then apply the method to classic and Transformer-based user embeddings from two real-world datasets. We also demonstrate how the corresponding natural-language labels enable zero-shot audience retrieval and profiling on embeddings never trained on text. The framework is post-hoc and model-agnostic, requiring no modification to upstream embedding systems.

Key Contributions

  • Interpretability framing: User embeddings are treated as sparse superpositions of interpretable behavioral modes.
  • Theoretical grounding: The structure is predicted by generative models of user behavior (linear case) and extends to deep representations via the superposition hypothesis.
  • Recovery method: Sparse autoencoders (SAEs) recover superimposed modes, outperforming classical sparse baselines in synthetic validation with known ground truth.
  • Natural-language modes: Recovered modes are automatically named and queryable, enabling zero-shot audience retrieval and profiling on embeddings never trained on text.
  • Practical applicability: Demonstrated on classic and Transformer-based user embeddings from two real-world datasets.
  • Drop-in design: The framework is post-hoc and model-agnostic, requiring no changes to upstream embedding systems.
---

*Auto-collected on 2026-11-07*

Tags

#machine-learning#user-embeddings#interpretability#sparse-autoencoders#recommendation-systems#dictionary-learning#zero-shot

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636818