How Can Recommender Systems Benefit from Large Language Models: A Survey (ACM TOIS 2025)
This post introduces the survey "How Can Recommender Systems Benefit from Large Language Models: A Survey", published in *ACM Transactions on Information Systems* (2025).
- Source: https://dl.acm.org/doi/abs/10.1145/3678004
- Type: Survey
- A unified perspective that organizes scattered related work into a comparable framework.
- A clear decomposition of method components: representation learning, retrievers, re-rankers, planners, generators, and feedback mechanisms.
- Reproducible benchmarks, datasets, and taxonomy tables that lower the entry barrier for new researchers.
- Discussion of interfaces with LLM tool use, reinforcement learning, and multi-agent collaboration, including paths from research prototypes to production systems.
- Explicit open problems: evaluation trustworthiness, latency and cost, hallucination and safety, and cross-lingual/multimodal extension.
- 2019–2021: BERT re-ranking and DPR establish neural retrieval foundations.
- 2022–2023: RAG and FreshLLMs drive retrieval-generation fusion.
- 2024 onward: Conversational/agentic search and Gen-RecSys proliferate.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth fronts.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation sets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost.
- Baselines: BM25, dense retrieval, cross-encoder re-ranking, retrieval-free LLMs, commercial search APIs.
- A Comprehensive Review on Harnessing Large Language Models to Overcome…
- A Comprehensive Survey on Cross-Domain Recommendation
- A Comprehensive Survey on Retrieval Methods in Recommender Systems
- A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
- A Survey of Generative Search and Recommendation in the Era of Large Language Models
- A Survey on LLM-powered Agents for Recommender Systems
Background and Motivation
Traditional recommender pipelines separate retrieval, ranking, and generation, which limits their ability to meet modern user expectations for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. This survey systematically maps how LLMs can be applied across the recommendation stack, covering open-domain information access, conversational search, semantic understanding in recommendation, and end-to-end architectures that combine external knowledge sources with generative models.
Core Contributions
Taxonomy
| Dimension | Subcategory | Representative Ideas | Strengths | Limitations | |------|------|------|------|------| | Modeling paradigm | Discriminative / generative retrieval | Two-tower, cross-encoders, DSI, GPT-indexed | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / Online / Human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Divergence from real satisfaction |
Research Evolution
Evaluation Landscape
Common benchmarks and metrics covered by the survey include:
Quantitative results should be verified against the original PDF.
Key Insights
1. Architecture: Cascaded retrieval + re-ranking + generation remains mainstream, but agentic paradigms make retrieval strategy itself learnable. 2. Data: High-quality instruction data and click/session logs are both critical; synthetic data risks knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge requires cross-validation with human assessment. 4. Production: Latency, cost, interpretability, and safety are hard constraints—academic benchmarks alone are insufficient.
Limitations and Future Directions
Identified gaps include insufficient unified benchmarks, non-reproducible private data, LLM evaluation bias, and safety/cost constraints for agentic systems. Future work includes finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multimodal/cross-lingual consistency, as well as causal and fairness constraints for recommenders.