Engineering Conversational Search Systems: A Review of Applications, Architectures, and Functional Components
Source: arXiv:2407.00997 | Authors: Phillip Schneider, Wessel Poelman, Michael Rovatsos, Florian Matthes | Published: 2024-07-01 | Type: Survey
Original Abstract
> Conversational search systems enable information retrieval via natural language interactions, with the goal of maximizing users' information gain over multiple dialogue turns. The increasing prevalence of conversational interfaces adopting this search paradigm challenges traditional information retrieval approaches, stressing the importance of better understanding the engineering process of developing these systems. We undertook a systematic literature review to investigate the links between theoretical studies and technical implementations of conversational search systems. Our review identifies real-world application scenarios, system architectures, and functional components. We consolidate our results by presenting a layered architecture framework and explaining the core functions of conversational search systems. Furthermore, we reflect on our findings in light of the rapid progress in large language models, discussing their capabilities, limitations, and directions for future research.
Background and Motivation
Conversational search systems aim to maximize users' information gain over multiple dialogue turns, but the growing prevalence of conversational interfaces challenges traditional information retrieval pipelines that treat retrieval, ranking, and generation in isolation. This survey systematically reviews the literature to bridge theoretical studies and technical implementations, covering open-domain information access, enterprise knowledge retrieval, and end-to-end architectures that couple external knowledge sources with generative models.
Key Contributions
- A unified perspective that organizes dispersed related work into a comparable framework.
- A clear decomposition of method components: representation learning, retrievers, re-rankers, planners, generators, and feedback mechanisms.
- Reproducible taxonomies and coverage of benchmarks, datasets, and classification tables, lowering the entry cost for future researchers.
- Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, including migration paths from research prototypes to industrial systems.
- Explicit open problems: evaluation trustworthiness, latency and cost, hallucination and safety, and cross-lingual/multimodal extension.
- Dense retrieval: high recall, low latency; suited for first-stage retrieval.
- Late interaction (e.g., ColBERT): higher accuracy but larger indexes.
- Generative IR: directly "generates" documents via tokens or docids, simplifying cascades.
- Agentic search: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection.
- 2019–2021: BERT re-ranking and DPR lay the foundation of neural retrieval.
- 2022–2023: RAG and FreshLLM drive retrieval-generation fusion.
- 2024 onward: conversational/agentic search and Gen-RecSys explode.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth frontiers.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, recommendation datasets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost.
- Baselines: BM25, dense retrieval, cross-encoder re-ranking, no-retrieval LLMs, commercial search APIs.
- Ablations: contributions of retrieval steps, re-ranking depth, and training data scale.
- A Survey of Conversational Search, Sep 2025, ACM
- Agentic Conversational Search with Contextualized Reasoning
- CTR-Guided Generative Query Suggestion in Conversational Search, EMNLP
- ChatRetriever: Adapting Large Language Models for Generalized and Robust Retrieval
- CoSearchAgent: A Lightweight Collaborative Search Agent with Large Language Models
- ConvGQR: Generative Query Reformulation for Conversational Search
- Original paper: Engineering Conversational Search Systems: A Review of Applications, Architectures, and Functional Components. https://arxiv.org/abs/2407.00997
Taxonomy
| Dimension | Sub-class | Representative Approaches | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Bi-encoders, cross-encoders, DSI, GPT-indexed | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization objectives | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / online / human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Divergence from real satisfaction |
Main Research Lines
The survey juxtaposes four main lines of work:
Timeline of Development
Evaluation Paradigm
Common benchmarks and metrics covered by such surveys:
Quantitative results should be checked against the original PDF tables.
Conclusions and Insights
1. Architecture: cascaded retrieve → re-rank → generate remains mainstream, but the agentic paradigm makes retrieval count and policy themselves learnable. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human assessment. 4. Product: latency, cost, interpretability, and safety are hard industrial constraints—optimizing academic benchmarks alone is insufficient.
Open Problems and Future Directions
Noted gaps include insufficient unified benchmarks, non-reproducible private data, LLM evaluation bias, and safety/cost constraints for agentic systems. Future work includes finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multimodal/cross-lingual consistency.
Engineering Checklist
| Item | Question | Recommendation | |---|---|---| | Data | PII in training/index? Version management? | Partitioned indexes, anonymization, rollback-able embedding versions | | Latency | p99 budget? Retrieval steps? | Cascade + early stopping, query caching, async re-ranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Poisoning/bias from open retrieval? | Source whitelists, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Small-model routing, distillation, hybrid sparse+dense |
Glossary
| Term | Meaning | |---|---| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool calling | | Gen-IR | Generative Information Retrieval |