Agentic Information Retrieval: A Next-Generation IR Paradigm Driven by LLM Agents
Source: arXiv:2410.09713 — Published 2024-10-13. Authors: Weinan Zhang, Junwei Liao, Ning Li, Kounianhua Du, Jianghao Lin.
Overview
This paper introduces agentic information retrieval (Agentic IR), a next-generation paradigm for IR driven by large language models (LLMs) and AI agents. It systematically discusses the topic from task formulation, architecture, evaluation, case studies, and challenges to future prospects.
Key Points
- Redefinition of "information": Traditional IR (since the 1970s) retrieves relevant information items from a pre-defined corpus. Agentic IR shifts the goal from acquiring static items to achieving a target information state — the dynamic, context-dependent situation a user is in, including real-time preferences, contextual factors, and decision-making processes.
- From queries to instructions: Retrieval based on user queries naturally extends to *achieving the target information state given a user instruction*, which defines Agentic IR.
- System architecture decomposition: The pipeline typically includes encoders for dense/sparse representations, retrievers, rerankers, planners, memory modules, and tool interfaces, combined serially or in parallel.
- Learning strategies: Supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), and bootstrapped data synthesis.
- Inference strategies: Single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping, and compute/budget control.
- Datasets/benchmarks: MS MARCO, BEIR, Natural Questions, domain corpora, recommendation datasets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.
- Evaluation is shifting from static nDCG toward process-oriented metrics: task success rate, citation accuracy, and multi-hop reasoning chain completeness.
- Evaluation credibility, latency and cost trade-offs, hallucination and safety risks, and cross-lingual / multimodal extension remain open.
- Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommender systems.
- A Systematic Framework for Enterprise Knowledge Retrieval (arXiv:2512.05411)
- Retrieval Augmented Generation and Understanding in Vision: A Survey (arXiv:2503.18016)
- Synergizing RAG and Reasoning: A Systematic Review (arXiv:2504.15909)
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via RL (arXiv:2509.24193)
Evaluation Landscape
Implications for Search / Recommendation / Personalization
1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm makes *when and how much to retrieve* itself a learnable decision. 2. Data: High-quality instruction data and click/session logs are critical; synthetic data requires guarding against leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Production: Latency, cost, interpretability, and safety are hard industrial constraints — do not optimize only academic benchmarks.
Limitations and Open Problems
Engineering Checklist
| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? Versioning? | Partitioned indexes, anonymization, rollback-capable embedding versions | | Latency | p99 budget? Retrieval steps? | Cascades + early stop, cache popular queries, async reranking | | Quality | Do offline gains transfer to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Route to smaller models, distillation, hybrid sparse+dense retrieval |