Retrieval-Augmented Generation for Large Language Models: A Survey (2023) — Forum Overview

Forum topic · 小凯 · 2026-07-05

Summary

This forum post from zhichai.net presents an overview of the 2023 survey "Retrieval-Augmented Generation for Large Language Models: A Survey", a widely cited RAG survey hosted via BAAI's simg paper archive. The post situates RAG within large-scale search, recommendation, and personalization systems, where efficiency, scalability, and user intent understanding remain core challenges. It organizes the field into a taxonomy covering modeling paradigms (discriminative vs. generative retrieval), LLM integration styles (RAG, agents, tool use), optimization objectives, and evaluation practices (offline metrics, LLM-as-judge, human studies). It contrasts four research lines — dense retrieval, late interaction (e.g., ColBERT), generative IR, and agentic search — and traces the field's evolution from BERT-era reranking and DPR through RAG fusion to agentic search, Deep Research, and GraphRAG. The post also lists open problems such as evaluation reliability, latency and cost, hallucination and safety, and cross-lingual/multimodal extension, and includes a practical engineering checklist covering data privacy, latency budgets, quality assurance, security, and cost. Links to the original PDF and related survey entries are preserved.

Retrieval-Augmented Generation for Large Language Models: A Survey (2023)

This post summarizes the 2023 survey *Retrieval-Augmented Generation for Large Language Models: A Survey*.

Original PDF: https://simg.baai.ac.cn/paperfile/25a43194-c74c-4cd3-b60f-0a1f27f8b8af.pdf
Resource type: Survey
Section: Search Surveys

Key points

Motivation: In large-scale search, recommendation, and personalization systems, RAG faces long-standing challenges around efficiency, scalability, and user intent understanding. Traditional pipelines treat retrieval, ranking, and generation as separate stages, which struggles to meet LLM-era demands for natural language interaction, multi-hop reasoning, and up-to-date knowledge.
Contributed perspective: The post frames the survey as offering a unified view of a scattered literature, decomposing method components — representation learning, retriever, reranker, planner, generator, and feedback mechanisms — for easier engineering adoption.

Taxonomy

| Dimension | Sub-categories | Representative ideas | Strengths | Limitations | |-----------|----------------|----------------------|-----------|-------------| | Modeling paradigm | Discriminative / generative retrieval | Bi-encoders, cross-encoders, DSI, GPT-indexed | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / Online / Human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Diverges from real satisfaction |

Four main research lines

Dense retrieval: high recall, low latency; suited for first-stage retrieval.
Late interaction (e.g., ColBERT): higher accuracy but larger index.
Generative IR: directly "generates" documents via tokens or docids, simplifying cascades.
Agentic search: models search as sequential decision-making, enabling multi-hop and self-reflection.

Timeline of evolution

2019–2021: BERT reranking and DPR lay the foundation for neural retrieval.
2022–2023: RAG and FreshLLM drive retrieval-generation fusion.
2024+: Conversational/agentic search and Gen-RecSys surge.
2025–2026: RL-trained search agents, Deep Research, and GraphRAG emerge as growth frontiers.

Evaluation conventions

Typical benchmarks and metrics covered by surveys in this area:

Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, recommendation datasets.
Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost.
Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.
Ablations: contributions of retrieval steps, reranking depth, and training data scale.

Note: the poster cautions that exact quantitative results should be verified against the original PDF.

Conclusions and insights for Search / Rec / Personalization

1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm makes retrieval count and strategy themselves learnable. 2. Data: High-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Product: Latency, cost, interpretability, and safety are hard industrial constraints — do not optimize only academic benchmarks.

Open problems and future directions

Insufficient unified benchmarks; private data not reproducible; LLM evaluation bias; safety and cost constraints for agentic systems.
Future work: finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, multimodal and cross-lingual consistency, more efficient test-time compute allocation, deeper integration with knowledge graphs/structured databases, and causal/fairness constraints for recommendation.

Engineering checklist (from the post's appendix)

| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? Version management? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? How many retrieval steps? | Cascades + early stop, cache hot queries, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelisting, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Route to smaller models, distillation, hybrid sparse+dense |

Glossary

IR: Information Retrieval
RAG: Retrieval-Augmented Generation
LTR: Learning to Rank
nDCG: Normalized Discounted Cumulative Gain
Agentic Search: Search modeled as sequential decisions and tool calls
Gen-IR: Generative Information Retrieval