A Survey on Large Language Models for Recommendation (WWW 2024, Springer)
Overview
This survey, published at WWW 2024 and hosted by Springer, systematically reviews research on applying large language models (LLMs) to recommender systems.
- Title: A survey on large language models for recommendation
- Venue: WWW 2024 / Springer (World Wide Web journal)
- Link: https://link.springer.com/article/10.1007/s11280-024-01291-2
- Type: Survey
- Unified perspective: consolidates scattered LLM-for-recommendation work into a comparable framework.
- Component decomposition: clearly separates representations, retrievers, re-rankers, planners, generators, and feedback mechanisms for engineering practice.
- Taxonomy and benchmarks: provides reproducible classification tables and protocol coverage that lower the entry barrier for new researchers.
- Interfaces to emerging paradigms: discusses connections to LLM tool calling, reinforcement learning, and multi-agent collaboration, plus paths from research prototypes to industrial systems.
- Open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- 2019–2021: BERT re-ranking and dense passage retrieval (DPR) establish neural retrieval foundations.
- 2022–2023: RAG and FreshLLM drive retrieval-generation fusion.
- 2024 onwards: Conversational/agentic search and generative recommender systems (Gen-RecSys) surge.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth areas.
- Dense retrieval: high recall, low latency; suited to first-stage retrieval.
- Late interaction (e.g., ColBERT): higher precision but larger indexes.
- Generative IR: directly "generates" documents via tokens or docids, simplifying cascades.
- Agentic search: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation datasets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost.
- Baselines: BM25, dense retrieval, cross-encoder re-ranking, retrieval-free LLMs, commercial search APIs.
- Ablations: contributions of retrieval steps, re-ranking depth, and training data scale.
- A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender Systems Challenges (arXiv 2507.21117)
- A Comprehensive Survey on Cross-Domain Recommendation (arXiv 2503.14110)
- A Comprehensive Survey on Retrieval Methods in Recommender Systems (arXiv 2407.21022)
- A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys, DOI 10.1145/3637528.3671474)
- A Survey of Generative Search and Recommendation in the Era of Large Language Models (arXiv 2404.16924)
- A Survey on LLM-powered Agents for Recommender Systems (arXiv 2502.10050)
Background and Motivation
Traditional recommendation pipelines separate retrieval, ranking, and generation stages, which limits adaptability to the LLM era's demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. This survey positions itself at the intersection of large-scale search, recommendation, and personalization, addressing how responsibilities among retrieval, ranking, generation, and tool invocation should be redistributed when LLMs enter the stack.
Key Points
Taxonomy
| Dimension | Subclasses | Representative Ideas | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Two-tower, cross-encoders, DSI, GPT-indexed | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotation | | Evaluation | Offline / online / human | nDCG, MRR, LLM-as-judge, A/B | Comparable | Deviates from real satisfaction |
Evolution of the Field
Four main methodological lines are typically compared:
Evaluation Paradigms
Common benchmarks and metrics covered in surveys of this area include:
Exact quantitative results should be verified against the original PDF tables.
Insights and Implications
1. Architecture: cascaded retrieval + re-ranking + generation remains mainstream, but the agentic paradigm makes retrieval count and strategy themselves learnable. 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data risks knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Production: latency, cost, interpretability, and safety are hard constraints in industrial deployment—academic benchmarks alone are insufficient.
Limitations and Future Directions
Commonly noted limitations include insufficient unified benchmarks, non-reproducible private data, LLM evaluation bias, and safety/cost constraints of agentic systems. Future work covers finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multimodal/cross-lingual consistency, as well as efficient test-time compute allocation, deeper fusion with knowledge graphs and structured databases, and causal/fairness constraints for recommendation.