A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys), KDD 2024
Overview
This post introduces the KDD 2024 survey "A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)".
- Source: https://dl.acm.org/doi/abs/10.1145/3637528.3671474
- Type: Survey
- Category: Recommender Engine Surveys
- Dense retrieval: high recall, low latency, suited for first-stage retrieval.
- Late interaction (e.g., ColBERT): higher accuracy but larger indexes.
- Generative IR: directly "generates" documents via tokens or docids, simplifying cascades.
- Agentic search: models search as sequential decision-making, supporting multi-hop and self-reflection.
- 2019–2021: BERT re-ranking and DPR establish neural retrieval foundations.
- 2022–2023: RAG and FreshLLM drive retrieval-generation fusion.
- 2024 onward: Conversational/agentic search and Gen-RecSys flourish.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth areas.
- Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora, public recommendation datasets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost.
- Baselines: BM25, dense retrieval, cross-encoder re-ranking, retrieval-free LLMs, commercial search APIs.
- A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges
- A Comprehensive Survey on Cross-Domain Recommendation
- A Comprehensive Survey on Retrieval Methods in Recommender Systems
- A Survey of Generative Search and Recommendation in the Era of Large Language Models
- A Survey on LLM-powered Agents for Recommender Systems
- A comprehensive review of recommender systems: from theory to deployment
Background and Motivation
At scale, recommendation systems have long faced challenges in efficiency, scalability, and user-intent understanding. Traditional pipeline approaches treat retrieval, ranking, and generation as separate stages, which struggles to meet LLM-era requirements for natural-language interaction, multi-hop reasoning, and real-time knowledge. The survey aims to systematically organize the theory and practice of generative recommendation, covering open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures that combine external knowledge sources with generative models.
Taxonomy
| Dimension | Subclasses | Representative ideas | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative vs. generative retrieval | Two-tower, cross-encoders, DSI | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce labels | | Evaluation | Offline / Online / Human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Divergence from real satisfaction |
The survey generally juxtaposes four main research lines:
Evolution Timeline
Evaluation Paradigms
Typical benchmarks and trends covered:
Note: specific numerical results should be verified against the original PDF.
Key Insights for Search / Rec / Personalization
1. Architecture: cascade retrieval + re-rank + generation remains mainstream, but agentic paradigms make "how many retrieval steps and what strategy" itself learnable. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: latency, cost, explainability, and safety are hard industrial constraints — one cannot optimize academic benchmarks alone.
Open Problems and Future Directions
Commonly cited gaps: insufficient unified benchmarks, irreproducible private data, LLM evaluation bias, and safety/cost constraints for agentic systems. Future work includes finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, multimodal and cross-lingual consistency, more efficient test-time compute allocation, deeper fusion with knowledge graphs/structured databases, and causal and fairness constraints for recommenders.
Engineering Checklist
| Item | Question | Suggestion | |---|---|---| | Data | PII in training/index? Version management? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? Number of retrieval steps? | Cascade + early stop, hot-query cache, async re-rank | | Quality | Does offline gain translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation checks | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Token/GPU cost per query? | Small-model routing, distillation, hybrid sparse+dense |
Glossary
| Term | Meaning | |---|---| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |