Large Language Models for Information Retrieval: A Survey (arXiv 2308.07107)

Forum topic · 小凯 · 2026-07-05

Summary

Large Language Models for Information Retrieval: A Survey is an August 2023 arXiv survey (arXiv:2308.07107) by Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng and colleagues. It systematically reviews how large language models (LLMs) such as ChatGPT and GPT-4 can enhance information retrieval (IR) systems across the full pipeline: query rewriters, retrievers, rerankers, and readers. The survey traces IR's evolution from term-based sparse methods (BM25) through neural models (BERT rerankers, dense retrieval, DPR, late interaction such as ColBERT) to generative IR and emerging agentic search paradigms that model retrieval as sequential decision-making with tool use. It covers evaluation practices (nDCG, MRR, Recall@k, LLM-as-judge, online A/B testing), benchmarks (MS MARCO, BEIR, Natural Questions), and integration patterns like retrieval-augmented generation (RAG). Open challenges highlighted include evaluation reliability, latency and cost constraints, hallucination and safety risks, cross-lingual and multimodal generalization, and the gap between offline metrics and real user satisfaction. The survey serves as a roadmap for researchers and engineers building LLM-powered search, conversational search, and recommender systems.

Large Language Models for Information Retrieval: A Survey

Paper: Large Language Models for Information Retrieval: A Survey — arXiv:2308.07107, August 14, 2023. A survey co-authored by Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, and others (10 authors total).

Overview

Information retrieval (IR) systems such as search engines are a primary means of information acquisition and also serve as components of dialogue, question-answering, and recommender systems. IR has evolved from term-based methods to neural models, which excel at capturing contextual signals and semantic nuances but still face data scarcity, interpretability, and accuracy challenges. The survey examines the confluence of LLMs and IR, covering query rewriters, retrievers, rerankers, and readers, and explores emerging directions such as search agents.

Key points

Unified perspective: Consolidates dispersed LLM-for-IR research into a comparable framework spanning the retrieval, reranking, and generation pipeline.
Component decomposition: Breaks down method components (representation learning, retrievers, rerankers, planners, generators, feedback mechanisms) for easier engineering adoption.
Four main research lines: dense retrieval (high recall, low latency), late interaction (e.g., ColBERT, higher precision but larger indexes), generative IR (generating documents or docids directly), and agentic search (modeling search as sequential decisions with multi-hop reasoning and self-reflection).
Integration paradigms: RAG, agent-based systems, and tool use — flexible and interpretable, but with latency and error-propagation trade-offs.
Evaluation: Benchmarks such as MS MARCO, BEIR, Natural Questions; metrics including nDCG@10, MRR, Recall@k; growing use of LLM-as-judge alongside offline/online/human evaluation.

Timeline of the field

2019–2021: BERT rerankers and DPR establish neural retrieval foundations.
2022–2023: RAG and FreshLLM drive retrieval–generation fusion.
2024 onwards: conversational/agentic search and generative recommender systems (Gen-RecSys) expand rapidly.
2025+: reinforcement learning for search agents, Deep Research, and GraphRAG become new growth areas.

Takeaways for search, recommendation, and personalization

1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms treat retrieval frequency and strategy itself as learnable. 2. Data: High-quality instruction data and click/session logs are both critical; synthetic data needs safeguards against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge requires cross-validation with human assessment. 4. Productization: Latency, cost, interpretability, and safety are hard constraints for industrial deployment — academic benchmarks alone are insufficient.

Open problems and future directions

Lack of unified benchmarks and reproducibility on private data.
Bias in LLM-based evaluation; safety and cost constraints of agentic systems.
Future work: finer-grained process supervision, joint retrieval–reasoning training, enterprise metadata governance, multimodal and cross-lingual consistency, efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases.

Original abstract (excerpt)

> As a primary means of information acquisition, information retrieval (IR) systems, such as search engines, have integrated themselves into our daily lives. ... In this survey, we delve into the confluence of LLMs and IR systems, including crucial aspects such as query rewriters, retrievers, rerankers, and readers. Additionally, we explore promising directions, such as search agents, within this expanding field.

Related surveys

A Comprehensive Survey on Reinforcement Learning-based Agentic Search (arXiv:2510.16724)
A Survey of Conversational Search (arXiv:2410.15576)
A Survey of LLM Empowered Agents for Recommendation (arXiv:2503.05659)
A Survey of Model Architectures in Information Retrieval (arXiv:2502.14822)
A Survey on Knowledge-Oriented Retrieval-Augmented Generation (arXiv:2503.10677)