DecoupleSearch: Decoupling Planning and Search via Hierarchical Reward Modeling

• Paper: DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling
• Authors: Hao Sun, Zile Qiao, Bo Wang, Guoxin Chen, Yingyan Hou, Yong Jiang, et al. (9 authors total)
• Date: 2025-09-07
• Category: Agentic Search

Original Abstract

> Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs) through the dynamic integration of external knowledge. To further improve RAG's flexibility, Agentic RAG introduces autonomous agents into the workflow. However, Agentic RAG faces several challenges: (1) the success of each step depends on both high-quality planning and accurate search, (2) the lack of supervision for intermediate reasoning steps, and (3) the exponentially large candidate space for planning and searching. To address these challenges, we propose DecoupleSearch, a novel framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. Our approach constructs a reasoning tree, where each node represents planning and search steps. We leverage Monte Carlo Tree Search to assess the quality of each step. During inference, Hierarchical Beam Search iteratively refines planning and search candidates with dual value models. Extensive experiments across policy models of varying parameter sizes demonstrate the effectiveness of our method.

Key points

• Problem: Agentic RAG pipelines struggle because (1) step success depends jointly on planning quality and search accuracy, (2) intermediate reasoning steps lack supervision signals, and (3) the combined planning/search candidate space grows exponentially.
• Core idea: Decouple planning and search optimization using dual value models, so plan reasoning and search grounding can each be improved independently.
• Method:
• Build a reasoning tree whose nodes represent planning and search steps.
• Use Monte Carlo Tree Search (MCTS) to estimate per-step quality (hierarchical reward modeling).
• At inference, apply Hierarchical Beam Search to iteratively refine planning and search candidates using the dual value models.
• Evaluation: Extensive experiments across policy models of varying parameter sizes show consistent effectiveness. Specific quantitative results should be verified against the original PDF tables.

Context

The work sits at the intersection of agentic search and large-scale retrieval systems. In the LLM era, retrieval is no longer a one-shot step but an iterative, verifiable, plannable process, so search agents must learn *when and how* to retrieve. DecoupleSearch contributes a structured way to supervise and optimize the two intertwined capabilities (planning vs. searching) that drive multi-hop reasoning and tool-augmented search.

Notes for practitioners

• Quantitative claims should be cross-checked against the paper's original tables; this summary is based on the abstract and public metadata.
• Related literature to read alongside: agentic information retrieval surveys, RAG + reasoning synergies, and RL-based search agent bootstrapping (e.g., AceSearcher).

References

• Original paper: DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling. arXiv:2510.21712