Overview
This post summarizes the ACM survey A Survey of Conversational Search (September 2025), a systematic review of conversational search at the intersection of large-scale search, recommendation, personalization, and LLM-driven systems.
Background
Conversational search has long faced challenges in efficiency, scalability, and user intent understanding. Traditional pipelines treat retrieval, ranking, and generation as separate stages, which struggles to meet LLM-era demands for natural language interaction, multi-hop reasoning, and real-time knowledge. The survey addresses open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommender systems, and end-to-end architectures combining external knowledge sources with generative models.
Core Contributions
- Provides a unified perspective that places scattered prior work into a comparable framework.
- Decomposes method components — representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms — for engineering practice.
- Offers reproducible benchmarks, datasets, and classification tables to lower the entry cost for researchers.
- Discusses interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, and paths from research prototypes to industrial systems.
- Identifies open problems: evaluation trustworthiness, latency and cost, hallucination and safety, and cross-lingual/multimodal extension.
- Dense retrieval: high recall, low latency, suited to first-stage retrieval.
- Late interaction (e.g., ColBERT): higher precision but larger indexes.
- Generative IR: directly "generates" documents via tokens or docids, simplifying cascades.
- Agentic search: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection.
- 2019–2021: BERT reranking and DPR establish neural retrieval.
- 2022–2023: RAG and FreshLLMs drive retrieval-generation fusion.
- 2024 onward: conversational/agentic search and Gen-RecSys surge.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth fronts.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation sets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.
- Engineering Conversational Search Systems: A Review of Applications
- Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning
- CTR-Guided Generative Query Suggestion in Conversational Search (EMNLP)
- ChatRetriever: Adapting LLMs for Generalized and Robust Retrieval
- CoSearchAgent: A Lightweight Collaborative Search Agent
- ConvGQR: Generative Query Reformulation for Conversational Search
Taxonomy
| Dimension | Subclasses | Representative Approaches | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Bi-encoders, cross-encoders, DSI, GPT indexing | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / Online / Human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Divergence from real satisfaction |
Four Main Research Lines
Timeline of the Field
Evaluation Paradigms
Key Insights
1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms make "how many times and when to retrieve" itself learnable. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Products: latency, cost, interpretability, and safety are hard industrial constraints — academic benchmarks alone are insufficient.
Open Problems
The survey notes the lack of unified benchmarks, irreproducible private data, LLM evaluation bias, and safety/cost constraints for agentic systems. Future directions include finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multimodal/cross-lingual consistency.