Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations (Baidu, 2025)
Metadata
| Field | Content | |-------|---------| | Title | Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations | | Authors / Affiliation | Yuhao Yang, Zhi Ji, Zhaopeng Li, Yi Li, Zhonglin Mo, Yue Ding, et al. (11 authors), Baidu | | Published | March 2025 | | Link | https://arxiv.org/abs/2503.02453 | | Type | Academic paper | | Section | Recommender Engines |
One-line Summary
A Baidu research paper proposing a unified generative recommendation framework built on cascaded sparse-dense representations, bridging collaborative-filtering-style sparse signals and semantically rich dense embeddings.
Background and Motivation
Large-scale search and recommendation systems have long faced challenges around efficiency, scalability, and user intent understanding. Traditional pipelines tend to treat retrieval, ranking, and generation as separate stages, which limits adaptability in the LLM era where users expect natural language interaction, multi-hop reasoning, and up-to-date knowledge. This paper targets that gap by unifying generative recommendation with cascaded sparse-dense representations.
The core motivation behind sparse-dense hybridization:
- Sparse representations (e.g., item IDs) preserve collaborative filtering signals learned from user behavior.
- Dense representations capture semantic content and improve generalization to long-tail items and cold-start scenarios.
- A cascaded design combines both, balancing the efficiency of sparse lookup with the expressiveness of dense embeddings inside a single generative model.
- 360Brew: A Decoder-only Foundation Model for Personalized Ranking
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers
- Augmenting Netflix Search with In-Session Adapted Recommendations
- Bridging Language and Items for Retrieval and Recommendation
- Data-efficient Fine-tuning for LLM-based Recommendation, SIGIR 2024
- DiffKG: Knowledge Graph Diffusion Model for Recommendation, WSDM 2024
- Researchers: verify quantitative claims, ablations, and compute costs against the original PDF.
- Engineers: evaluate how the cascaded sparse-dense module could integrate with existing retrieval-ranking stacks, considering index update frequency and embedding version compatibility.
- Product managers: focus on user-perceivable benefits (latency, recommendation relevance) rather than offline metrics alone.
- Original paper: Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations, arXiv:2503.02453.
Context Within the Field
The evolution of recommendation architectures — from matrix factorization and deep CTR models, through sequential Transformers, to LLM-based and generative recommendation (Gen-Rec) — centers on a recurring tension: user behavior is sparse, the item catalog is huge, and business objectives require multi-dimensional trade-offs. LLMs contribute semantic priors and cold-start capability, but online inference cost and hallucination risk demand careful system design. This work sits at that intersection, addressing how the responsibilities of retrieval, ranking, and generation should be re-partitioned in the LLM era.