Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning
Overview
This entry covers the paper *Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning* (arXiv: 2601.13115), by Fengran Mo, Yifan Gao, Sha Li, Hansi Zeng, Xin Liu, Zhaoxuan Tan, et al. (10 authors), published 2026-01-19. It belongs to the Conversational Search section of this collection.
Problem and Motivation
LLMs have become a popular interface for human-AI interaction, supporting information seeking and task assistance through natural, multi-turn dialogue. In multi-turn settings, the context-dependent user intent evolves across interactions, requiring contextual interpretation, query reformulation, and dynamic coordination between retrieval and generation. Existing studies usually follow static rewrite-retrieve-generate pipelines that optimize each procedure separately, overlooking mixed-initiative action optimization. Recent deep search agents jointly optimize retrieval and generation via reasoning, but focus on single-turn scenarios and lack the ability to handle multi-turn interactions.
Approach
The authors introduce a conversational agent that interleaves search and reasoning across turns. Exploratory and adaptive behaviors are learned through reinforcement learning (RL) training with tailored rewards oriented toward evolving user goals, rather than fixed multi-stage pipelines.
Results
Experiments across four widely used conversational benchmarks demonstrate the method's effectiveness, surpassing several existing strong baselines.
Context and Takeaways
The post situates this work at the intersection of agentic search and large-scale search/recommendation systems: the new variables in the LLM era are the reasoning budget and the action space (whether to retrieve, how many times, and which tools to call). Key insights discussed include:
- Architecture: cascaded retrieve-rerank-generate remains mainstream, but the agentic paradigm makes retrieval count and strategy themselves learnable.
- Data: high-quality instruction data and session logs matter; synthetic data risks leakage and distribution shift.
- Evaluation: offline metrics diverge from online satisfaction; LLM-as-judge needs cross-validation with human assessment.
- Deployment: latency, cost, interpretability, and safety are hard constraints for production systems.
- A Survey of Conversational Search, Sep 2025, ACM
- Engineering Conversational Search Systems: A Review of Applications
- CTR-Guided Generative Query Suggestion in Conversational Search, EMNLP
- ChatRetriever: Adapting Large Language Models for Generalized and Robust Retrieval
- CoSearchAgent: A Lightweight Collaborative Search Agent
- ConvGQR: Generative Query Reformulation for Conversational Search
Related Entries
Note on Details
Quantitative results should be verified against the original PDF; this summary is based on the abstract and public metadata.