A Survey of Personalization: From RAG to Agent
Overview
- Paper: A Survey of Personalization: From RAG to Agent
- Authors: Xiaopeng Li, Pengyue Jia, Derong Xu, Yi Wen, Yingyi Zhang, Wenlin Zhang, et al. (14 authors total)
- Published: 2025-04-14
- Source: https://arxiv.org/abs/2504.10147
- Type: Survey
- Resources: https://github.com/Applied-Machine-Learning-Lab/Awesome-Personalized-RAG-Agent
- Unified perspective: Consolidates fragmented personalization research across RAG and agent-based architectures into a comparable framework.
- Component decomposition: Clearly breaks down method components — representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms — to ease engineering adoption.
- Taxonomy: Organizes work by modeling paradigm (discriminative vs. generative retrieval), LLM integration (RAG / Agent / tool-use), optimization objectives (relevance, diversity, freshness), and evaluation (offline, online, human).
- Datasets and metrics: Summarizes benchmarks such as MS MARCO, BEIR, and Natural Questions, plus metrics like nDCG@10, MRR, Recall@k, LLM-as-judge, and A/B testing.
- 2019–2021: BERT reranking and DPR establish neural retrieval foundations.
- 2022–2023: RAG and FreshLLM drive retrieval-generation fusion.
- 2024: Conversational/agentic search and generative recommender systems (Gen-RecSys) explode.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth fronts.
- Dense retrieval: High recall, low latency; suitable for first-stage retrieval; sensitive to domain shift and long-tail queries.
- Late interaction (e.g., ColBERT): Higher precision but larger index footprint.
- Generative IR: Directly "generates" documents or docids, simplifying cascades but complicating index updates.
- Agentic search: Models search as sequential decision-making, enabling multi-hop reasoning and self-reflection.
- Lack of unified benchmarks; private data is hard to reproduce.
- Evaluation biases in LLM judges; safety and cost constraints of agentic systems on the open web.
- Promising directions: finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multimodal/cross-lingual consistency.
- IR: Information Retrieval
- RAG: Retrieval-Augmented Generation
- LTR: Learning to Rank
- nDCG: Normalized Discounted Cumulative Gain, a ranking quality metric
- Agentic Search: Paradigm modeling search as sequential decisions and tool calls
- Gen-IR: Generative Information Retrieval
- Original paper: A Survey of Personalization: From RAG to Agent — https://arxiv.org/abs/2504.10147
- Awesome list: https://github.com/Applied-Machine-Learning-Lab/Awesome-Personalized-RAG-Agent
Abstract (original)
> Personalization has become an essential capability in modern AI systems, enabling customized interactions that align with individual user preferences, contexts, and goals. Recent research has increasingly concentrated on Retrieval-Augmented Generation (RAG) frameworks and their evolution into more advanced agent-based architectures within personalized settings to enhance user satisfaction. Building on this foundation, this survey systematically examines personalization across the three core stages of RAG: pre-retrieval, retrieval, and generation. Beyond RAG, we further extend its capabilities into the realm of Personalized LLM-based Agents, which enhance traditional RAG systems with agentic functionalities, including user understanding, personalized planning and execution, and dynamic generation. For both personalization in RAG and agent-based personalization, we provide formal definitions, conduct a comprehensive review of recent literature, and summarize key datasets and evaluation metrics. Additionally, we discuss fundamental challenges, limitations, and promising research directions in this evolving field.
Key Points
Research Evolution Timeline
Main Retrieval Paradigms Compared
Insights for Search, Recommendation, and Personalization
1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm makes retrieval count and strategy themselves learnable. 2. Data: High-quality instruction data and click/session logs are both critical; synthetic data requires protection against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Production: Latency, cost, explainability, and safety are hard constraints for industrial deployment — academic benchmarks alone are insufficient.
Open Problems and Future Directions
Engineering Checklist
| Item | Question | Suggestion | |------|----------|------------| | Data | Does training/indexing data contain PII? How are versions managed? | Partitioned indexes, anonymization, rollback-capable embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascade + early stopping, query caching, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Small-model routing, distillation, hybrid sparse+dense retrieval |