From Matching to Generation: A Survey on Generative Information Retrieval (ACM TOIS, Feb 2025)

Source: https://dl.acm.org/doi/abs/10.1145/3722552 Type: Survey, journal version, ACM Transactions on Information Systems, February 2025

Overview

This post summarizes the journal-version survey *From Matching to Generation: A Survey on Generative Information Retrieval*, published in ACM Transactions on Information Systems (Feb 2025). The survey systematically organizes the shift in information retrieval from traditional discriminative matching pipelines to generative paradigms driven by large language models (LLMs).

Background and Motivation

Traditional search pipelines separate retrieval, ranking, and generation, which struggles to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. The survey addresses this by providing a unified, comparable framework for the field, spanning open-domain search, enterprise knowledge retrieval, conversational search, recommendation semantics, and end-to-end architectures combining external knowledge with generative models.

Core Contributions

• A unified perspective that consolidates scattered prior work into a comparable framework.
• Clear decomposition of method components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms.
• Reproducible benchmarks, datasets, and classification tables to lower the entry barrier for new researchers.
• Discussion of interfaces with emerging paradigms: LLM tool use, reinforcement learning, and multi-agent collaboration.
• Explicit open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.

Taxonomy

| Dimension | Subcategory | Representative Ideas | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Bi-encoder, cross-encoder, DSI, GPT-indexing | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization objectives | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Label scarcity | | Evaluation | Offline / online / human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Divergence from real satisfaction |

Main Research Lines Compared

• Dense retrieval: high recall, low latency; suited to first-stage retrieval.
• Late interaction (e.g., ColBERT): higher precision but larger indexes.
• Generative IR: directly "generates" documents via tokens or docids, simplifying cascades.
• Agentic search: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection.

Timeline of the Field

• 2019–2021: BERT reranking and DPR establish neural retrieval foundations.
• 2022–2023: RAG and FreshLLMs drive retrieval-generation fusion.
• 2024 onward: conversational/agentic search and generative recommender systems (Gen-RecSys) surge.
• 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth areas.

Evaluation Paradigms

Typical benchmarks and metrics covered include:

• Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, recommendation datasets.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost.
• Baselines: BM25, dense retrieval, cross-encoder reranking, no-retrieval LLMs, commercial search APIs.

Key Takeaways

1. Architecture: cascaded retrieve-rerank-generate remains mainstream, but the agentic paradigm makes retrieval count and policy learnable. 2. Data: high-quality instruction data and click/session logs are critical; synthetic data risks leakage and distribution shift. 3. Evaluation: offline metrics increasingly diverge from online satisfaction; LLM-as-judge needs cross-validation with human assessment. 4. Product: latency, cost, explainability, and safety are hard constraints for industrial deployment.

Open Problems and Future Directions

The survey highlights insufficient unified benchmarks, irreproducible private data, LLM evaluation bias, and the safety/cost constraints of agentic systems. Future directions include finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, multimodal and cross-lingual consistency, deeper fusion with knowledge graphs, and causal/fairness constraints for recommendation.

Related Entries

• A Comprehensive Survey on Reinforcement Learning-based Agentic Search
• A Survey of Conversational Search (Oct 2024, arXiv)
• A Survey of LLM-Empowered Agents for Recommendation
• A Survey of Model Architectures in Information Retrieval (Jan 2025)
• A Survey on AI Search with Large Language Models (July 2025)
• A Survey on Knowledge-Oriented Retrieval-Augmented Generation (Mar 2025)

Glossary

| Term | Meaning | |---|---| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |