Agentic Information Retrieval: A Next-Generation IR Paradigm Driven by LLMs and AI Agents

This post summarizes the paper "Agentic Information Retrieval" (arXiv:2410.09713) by Weinan Zhang, Junwei Liao, Ning Li, Kounianhua Du, and Jianghao Lin (October 2024), categorized under Agentic Search.

Key Points

• New paradigm: The paper introduces Agentic Information Retrieval (Agentic IR), a transformative next-generation IR paradigm driven by large language models (LLMs) and AI agents.
• Central shift: The definition of "information" evolves from static, pre-defined information items to dynamic, context-dependent information states — the user's live information context, including acquired items, real-time preferences, contextual factors, and decision-making processes.
• Reformulated task: Traditional IR (retrieving relevant items for a query) is naturally extended to achieving a target information state given a user instruction.
• Scope: The paper systematically discusses task formulation, architecture, evaluation, case studies, and challenges/future prospects.

Background

Since the 1970s, IR has been defined as acquiring relevant items from a pre-defined corpus. Traditional systems (e.g., web search) are constrained by static information items and can't meet LLM-era demands for natural-language interaction, multi-hop reasoning, and real-time knowledge. Target scenarios include open-domain search, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures coupling external knowledge with generative models.

System Decomposition

Typical agentic IR components discussed:

1. Input & representation: dense/sparse encodings of queries, documents, and user context, or structured prompts; 2. Core modules: retriever, reranker, planner, memory, tool interfaces; 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (incl. process rewards), synthetic data bootstrapping; 4. Inference strategies: single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping and budget control.

Evaluation Landscape

• Datasets: MS MARCO, BEIR, Natural Questions, proprietary corpora, recommendation benchmarks;
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost;
• Baselines: BM25, dense retrieval, cross-encoder rerankers, retrieval-free LLMs, commercial search APIs.

Quantitative results should be verified against the original PDF.

Implications for Search, Recommendation, and Personalization

1. Architecture: cascaded retrieve-rerank-generate remains mainstream, but agentic paradigms make *retrieval count and policy* themselves learnable; 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data risks leakage and distribution shift; 3. Evaluation: the gap between offline metrics and online satisfaction widens; LLM-as-judge needs cross-validation with human evaluation; 4. Product: latency, cost, explainability, and safety are hard production constraints, not just benchmark optimizations.

Limitations and Future Directions

Potential limitations include experiment scale constrained by compute budget, benchmark–real-user distribution mismatch, English-centric data limiting cross-lingual generalization, and safety risks of open-web agent systems. Future work: more efficient test-time compute allocation, deeper integration with knowledge graphs/structured databases, and causal/fairness constraints for recommendation.

Engineering Checklist

| Item | Question | Recommendation | |------|----------|----------------| | Data | PII in training/index? Version control? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? Retrieval steps? | Cascade + early stop, query caching, async reranking | | Quality | Do offline gains convert to CTR/satisfaction? | Interleaving experiments, human audits, citation checks | | Safety | Poisoning/bias from open retrieval? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query tokens and GPU usage? | Small-model routing, distillation, hybrid sparse+dense |

Glossary

• IR: Information Retrieval
• RAG: Retrieval-Augmented Generation
• LTR: Learning to Rank
• nDCG: Normalized Discounted Cumulative Gain
• Agentic Search: search modeled as sequential decision-making and tool invocation
• Gen-IR: Generative Information Retrieval

Original Abstract (English, verbatim)

> Since the 1970s, information retrieval (IR) has long been defined as the process of acquiring relevant information items from a pre-defined corpus to satisfy user information needs. Traditional IR systems, while effective in domains like web search, are constrained by their reliance on static, pre-defined information items. To this end, this paper introduces agentic information retrieval (Agentic IR), a transformative next-generation paradigm for IR driven by large language models (LLMs) and AI agents. The central shift in agentic IR is the evolving definition of "information" from static, pre-defined information items to dynamic, context-dependent information states. Information state refers to a particular information context that the user is right in within a dynamic environment, encompassing not only the acquired information items but also real-time user preferences, contextual factors, and decision-making processes. In such a way, traditional information retrieval, focused on acquiring relevant information items based on user queries, can be naturally extended to achieving the target information state given the user instruction, which thereby defines the agentic information retrieval. We systematically discuss agentic IR from various aspects, i.e., task formulation, architecture, evaluation, case studies, as well as challenges and future prospects. We believe that the concept of agentic IR introduced in this paper not only broadens the scope of information retrieval research but also lays the foundation for a more adaptive, interactive, and intelligent next-generation IR paradigm.