A Survey on AI Search with Large Language Models (July 2025, Preprint)
> Source: preprints.org download link — Note: preprint, not peer reviewed.
Overview
This survey addresses AI search with large language models (LLMs), motivated by the limits of traditional pipelines that treat retrieval, ranking, and generation as separate stages. Modern users expect natural-language interaction, multi-hop reasoning, and real-time knowledge — requirements that force a rethink of how responsibilities are divided across the search stack. Core scenarios covered include open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures coupling external knowledge sources with generative models.
Key Contributions
- A unified perspective that brings scattered related work into a comparable framework.
- A clear decomposition of method components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms.
- Reproducible benchmarks, datasets, and classification tables to lower the entry barrier for new researchers.
- Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, including paths from research prototypes to industrial systems.
- Explicitly listed open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- Dense retrieval: high recall, low latency; suited to first-stage retrieval.
- Late interaction (e.g., ColBERT): higher precision but larger indexes.
- Generative IR: directly "generates" documents via tokens or docids, simplifying the cascade.
- Agentic search: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection.
- 2019–2021: BERT reranking and DPR lay the foundations of neural retrieval.
- 2022–2023: RAG and FreshLLM drive retrieval–generation fusion.
- 2024: Conversational/agentic search and Gen-RecSys surge.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth fronts.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, recommendation datasets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, no-retrieval LLMs, commercial search APIs.
- Insufficient unified benchmarks; non-reproducible private data; LLM evaluation bias; safety and cost constraints for agentic systems.
- Future work: finer-grained process supervision, joint retrieval–reasoning training, enterprise metadata governance, multimodal and cross-lingual consistency.
- A Comprehensive Survey on Reinforcement Learning-based Agentic Search
- A Survey of Conversational Search, Oct 2024, arXiv
- LLM-Empowered Agents for Recommendation
- A Survey of Model Architectures in Information Retrieval, Jan 2025
- Knowledge-Oriented Retrieval-Augmented Generation, Mar 2025
- Cross-Modal Retrieval: A Systematic Review
Taxonomy
| Dimension | Subclasses | Representative Approaches | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Dual-tower, cross-encoders, DSI, GPT indexing | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization objective | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / Online / Human | nDCG, MRR, LLM-as-judge, A/B | Comparable | Deviates from real satisfaction |
Four Main Technical Lines
Evolution Timeline
Evaluation Paradigms
Quantitative results should be verified against the original PDF; this report summarizes design logic based on the abstract and public metadata.
Key Insights
1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm makes "retrieval count and policy" itself learnable. 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data risks knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human assessment. 4. Product: latency, cost, interpretability, and safety are hard constraints for industrial deployment — academic benchmarks alone are insufficient.
Open Problems and Future Directions
Limitations
Potential limitations include experiment scale constrained by GPU budgets, mismatch between benchmarks and real user distributions, English-centric data leaving cross-lingual generalization unknown, and safety risks of agent systems on the open web.