DecoupleSearch: Decoupling Planning and Search via Hierarchical Reward Modeling
Paper: https://arxiv.org/abs/2510.21712 Authors: Hao Sun, Zile Qiao, Bo Wang, Guoxin Chen, Yingyan Hou, Yong Jiang, et al. (9 authors) Published: 2025-09-07 Category: Agentic Search
Overview
Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by dynamically integrating external knowledge. Agentic RAG extends this by introducing autonomous agents into the workflow, but faces three key challenges:
1. The success of each step depends on both high-quality planning and accurate search. 2. There is a lack of supervision for intermediate reasoning steps. 3. The candidate space for planning and searching grows exponentially.
The DecoupleSearch Framework
DecoupleSearch decouples planning and search using dual value models, enabling independent optimization of plan reasoning and search grounding. The approach:
- Constructs a reasoning tree, where each node represents a planning or search step.
- Uses Monte Carlo Tree Search (MCTS) to assess the quality of each step.
- At inference, applies Hierarchical Beam Search to iteratively refine planning and search candidates with the dual value models.
- Architecture: Agentic paradigms are turning retrieval count and policy into learnable objects, beyond fixed cascade pipelines.
- Evaluation: Metrics are shifting from static nDCG toward task success rate, citation accuracy, and multi-hop reasoning chain completeness.
- Deployment: Latency, cost, interpretability, and safety remain hard constraints for industrial adoption—offline benchmark gains must be validated with interleaving experiments and human auditing.
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Synergizing RAG and Reasoning: A Systematic Review (arXiv:2504.15909)
- Agentic Information Retrieval (arXiv:2410.09713)
Extensive experiments across policy models of varying parameter sizes demonstrate the effectiveness of the method.
Original Abstract
> Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs) through the dynamic integration of external knowledge. To further improve RAG's flexibility, Agentic RAG introduces autonomous agents into the workflow. However, Agentic RAG faces several challenges: (1) the success of each step depends on both high-quality planning and accurate search, (2) the lack of supervision for intermediate reasoning steps, and (3) the exponentially large candidate space for planning and searching. To address these challenges, we propose DecoupleSearch, a novel framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. Our approach constructs a reasoning tree, where each node represents planning and search steps. We leverage Monte Carlo Tree Search to assess the quality of each step. During inference, Hierarchical Beam Search iteratively refines planning and search candidates with dual value models. Extensive experiments across policy models of varying parameter sizes demonstrate the effectiveness of our method.
Context and Implications
The work sits at the intersection of agentic search and large-scale search/recommendation systems. While classic search stacks follow a funnel of recall, ranking, and generation, the LLM era introduces new variables: reasoning budget and action space (whether to retrieve, how many times, and which tools to call). DecoupleSearch treats retrieval strategy itself as a learnable, optimizable component.
Broader takeaways for the field: