A Survey on Multi-Turn Interaction Capabilities of Large Language Models
Source: https://arxiv.org/abs/2501.09959 · Published 2025-01-17 · Type: Survey
Overview
This survey provides a focused review of the multi-turn capabilities of large language models (LLMs), which are critical for downstream applications including conversational search and recommendation, consultation services, and interactive tutoring. As stated in the original abstract:
> Multi-turn interaction in the dialogue system research refers to a system's ability to maintain context across multiple dialogue turns, enabling it to generate coherent and contextually relevant responses. Recent advancements in large language models (LLMs) have significantly expanded the scope of multi-turn interaction, moving beyond chatbots to enable more dynamic agentic interactions with users or environments.
The survey explores four key aspects: 1. The core model capabilities that contribute to effective multi-turn interaction 2. How multi-turn interaction is evaluated in current practice 3. General algorithms used to enhance multi-turn interaction 4. Potential future research directions
Key Points
- Unified perspective: Consolidates dispersed work on LLM multi-turn interaction into a comparable framework spanning retrieval, ranking, generation, and feedback mechanisms.
- Four research threads in modern search:
- *Dense retrieval*: high recall, low latency, suited for first-stage retrieval
- *Late interaction* (e.g., ColBERT): higher precision but larger index
- *Generative IR*: directly "generates" documents via tokens or docids, simplifying cascades
- *Agentic search*: models search as sequential decision-making, supporting multi-hop reasoning and self-reflection
- LLM integration paradigms: RAG, agents, and tool-use offer flexibility and interpretability but introduce latency and error propagation.
- Timeline of evolution: 2019–2021 BERT reranking and DPR established neural retrieval; 2022–2023 RAG and FreshLLMs drove retrieval-generation fusion; from 2024, conversational/agentic search and Gen-RecSys surged; 2025–2026 trends include RL-trained search agents, Deep Research, and GraphRAG.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation sets
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs
- Ablations: Contributions of retrieval steps, reranking depth, and training data scale
- Lack of unified benchmarks; limited reproducibility with private data
- Bias in LLM-based evaluation
- Safety and cost constraints for agentic systems on the open web
- Future directions: finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, multimodal and cross-lingual consistency
- Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey (arXiv:2503.22458)
- An Empirical Analysis on Multi-turn Conversational Recommender Systems (DOI: 10.1145/3626772.3657893)
- Beyond Whole Dialogue Modeling: Contextual Disentanglement for Conversation (arXiv:2504.17427)
- CHIQ: Contextual History Enhancement for Improving Query Rewriting (arXiv:2406.05013)
- Few-Shot Conversational Dense Retrieval, SIGIR 2021 (DOI: 10.1145/3404835.3462856)
Evaluation Paradigms
Insights for Search, Recommendation, and Personalization
1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms make retrieval count and strategy themselves learnable. 2. Data: High-quality instruction data and click/session logs are both essential; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge requires cross-validation with human evaluation. 4. Product: Latency, cost, interpretability, and safety are hard constraints for industrial deployment.