A Survey on Knowledge-Oriented Retrieval-Augmented Generation

Overview

• Paper: A Survey on Knowledge-Oriented Retrieval-Augmented Generation
• Authors: Mingyue Cheng, Yucong Luo, Jie Ouyang, Qi Liu, Huijie Liu, Li Li, et al. (12 authors total)
• Published: 2025-03-11
• Link: https://arxiv.org/abs/2503.10677
• Type: Survey

Original abstract:

> Retrieval-Augmented Generation (RAG) has gained significant attention in recent years for its potential to enhance natural language understanding and generation by combining large-scale retrieval systems with generative models. RAG leverages external knowledge sources, such as documents, databases, or structured data, to improve model performance and generate more accurate and contextually relevant outputs. This survey aims to provide a comprehensive overview of RAG by examining its fundamental components, including retrieval mechanisms, generation processes, and the integration between the two. We discuss the key characteristics of RAG, such as its ability to augment generative models with dynamic external knowledge, and the challenges associated with aligning retrieved information with generative objectives. We also present a taxonomy that categorizes RAG methods, ranging from basic retrieval-augmented approaches to more advanced models incorporating multi-modal data and reasoning capabilities. Additionally, we review the evaluation benchmarks and datasets commonly used to assess RAG systems, along with a detailed exploration of its applications in fields such as question answering, summarization, and information retrieval. Finally, we highlight emerging research directions and opportunities for improving RAG systems, such as enhanced retrieval efficiency, model interpretability, and domain-specific adaptations. This paper concludes by outlining the prospects for RAG in addressing real-world challenges and its potential to drive further advancements in natural language processing.

Background and Motivation

In large-scale search, recommendation, and personalization systems, RAG has long faced challenges around efficiency, scalability, and user intent understanding. Traditional pipeline-style approaches often separate retrieval, ranking, and generation, which struggles to meet the combined demands of the LLM era: natural language interaction, multi-hop reasoning, and real-time knowledge. This survey systematically maps the theoretical and practical boundaries of this intersection, covering open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures that coordinate external knowledge sources with generative models.

Core Contributions

• Provides a unified perspective that organizes scattered related work into a comparable framework.
• Clearly decomposes method components: representation learning, retriever, reranker, planner, generator, and feedback mechanisms.
• Offers reproducible benchmarks, datasets, and taxonomy tables, lowering the entry cost for later researchers.
• Discusses interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, pointing to paths from research prototypes to industrial systems.
• Explicitly lists open problems: evaluation trustworthiness, latency and cost, hallucination and safety, and cross-lingual / multi-modal extension.

Taxonomy

| Dimension | Subcategory | Representative Ideas | Strengths | Limitations | |-----------|-------------|----------------------|-----------|-------------| | Modeling paradigm | Discriminative / generative retrieval | Dual-tower, cross-encoder, DSI, GPT indexing | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / online / human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Diverges from real satisfaction |

Representative Approaches

The survey places four main lines side by side: dense retrieval, late interaction, generative IR, and agentic search:

• Dense retrieval: high recall, low latency; suited to first-stage retrieval.
• Late interaction (e.g., ColBERT): higher precision but larger indexes.
• Generative IR: directly "generates" documents via tokens or docids, simplifying the cascade.
• Agentic search: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection.

Timeline of Research Evolution

• 2019–2021: BERT reranking and DPR lay the foundation of neural retrieval.
• 2022–2023: RAG and FreshLLM drive retrieval-generation fusion.
• 2024 onward: Conversational / agentic search and Gen-RecSys explode in popularity.
• 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth frontiers.

Evaluation Paradigms

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora, public recommendation sets.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
• Baselines: BM25, dense retrieval, cross-encoder reranking, no-retrieval LLMs, commercial search APIs.
• Ablations: contribution of each module (retrieval steps, reranking depth, training data scale).

Note: for specific numeric results, consult the original PDF; this report summarizes experimental design based on the abstract and public metadata.

Key Insights

1. Architecture: cascade retrieval + reranking + generation remains mainstream, but the agentic paradigm is making "retrieval count and strategy" itself learnable. 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: latency, cost, interpretability, and safety policies are hard constraints for industrial deployment—do not optimize only academic benchmarks.

Open Problems and Future Directions

Commonly cited gaps: insufficient unified benchmarks, non-reproducible private data, LLM evaluation bias, and the safety and cost constraints of agentic systems. Future work includes finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multi-modal / cross-lingual consistency. Limitations may include GPU-budget-constrained experiment scale, benchmarks mismatched to real user distributions, English-centric data with unknown cross-lingual generalization, and safety risks of agentic systems on the open web.

Engineering Checklist

| Check | Question | Recommendation | |-------|----------|----------------| | Data | Does training/indexing contain PII? How are versions managed? | Partitioned indexes, sanitization, rollback-capable embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascade + early stopping, hot-query caching, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelisting, adversarial detection, output filtering | | Cost | Per-query tokens and GPU usage? | Small-model routing, distillation, hybrid sparse+dense |

Actionable Advice for Readers

1. Researchers: reproduce core comparison experiments; check whether statistical significance and compute cost are reported. 2. Engineers: extract pluggable modules (encoder, reranker, planner) and evaluate integration cost with existing stacks. 3. Product managers: identify user-perceivable benefits (latency, answer trustworthiness, multi-turn consistency) rather than offline nDCG alone.

Related Entries

• A Comprehensive Survey on Reinforcement Learning-based Agentic Search
• A Survey of Conversational Search (Oct 2024)
• A Survey of Large Language Model Empowered Agents for Recommendation
• A Survey of Model Architectures in Information Retrieval (Jan 2025)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Paradigm modeling search as sequential decision-making and tool calling | | Gen-IR | Generative Information Retrieval |