DecomposeR: Planner-Centric RL with Typed DAG Plans for Deep Research

Forum topic · 小凯 · 2026-06-01

Summary

A team at the National University of Singapore proposes DecomposeR, a planner-centric reinforcement learning framework that decouples planning from execution in AI deep research systems. Instead of treating an agent rollout as a flat ReAct-style trajectory with a single terminal reward, DecomposeR externalizes the research plan as a typed directed acyclic graph (DAG) containing search nodes, aggregation nodes, and a single terminal answer node. Training proceeds in staged phases: cold-start SFT, planner RL rewarded for rubric coverage, search quality, and graph expressiveness, then answerer RL rewarded for aggregation execution, rubric judgment, and branch utilization. Built on Qwen3-8B, DecomposeR achieves 41.8 on DeepResearchBench, 42.0 on HealthBench, and 71.4 on ResearchQA-Mini, outperforming same-scale baselines, and staged training beats joint training by 3.7–4.7 points. In a fixed-external-answerer experiment using GPT-4.1, the 8B planner's plans approach or exceed GPT-5 prompt-only planning quality. Ablations show typed DAGs beat linear and tree structures, and revision after retrieval is essential. The framework also makes planning auditable, enabling structured diagnosis of failures in high-stakes domains like clinical evidence aggregation. Limitations include JSON parsing fragility, complex training pipelines, and open questions about evidence quality and generalization.

DecomposeR: Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward

| Property | Detail | |---|---| | Authors | Mustafa Anis Hussain, Xinle Wu, Yao Lu | | Institution | National University of Singapore | | arXiv ID | 2605.30824 | | Date | 2026-05-29 | | Category | cs.AI |

Key points

Core idea: Planning and execution in deep research should be decoupled. The research plan is externalized as a typed DAG, making the plan itself a policy object that can be explicitly rewarded and optimized.
Problem addressed: Existing RL-trained deep research systems (ReAct-style flat trajectories) suffer from ambiguous credit assignment and sparse rewards — a single scalar reward from the final answer cannot distinguish a bad plan from poor execution. Planning also remains implicit and unauditable, which is dangerous in high-stakes domains.

Method

DecomposeR replaces flat trajectories with an explicit plan 𝒢 mediating between a planner policy πP and an answerer policy πA:

1. The planner emits an initial typed DAG 𝒢₀ from query x. 2. The environment ℰ executes all search nodes, returning observations Z. 3. The planner consumes Z and emits a revised plan 𝒢₁ plus a fetch set F. 4. The answerer fills aggregation nodes in topological order, producing intermediate outputs and a cited final report y.

The DAG has three node types: search nodes (query + key points), aggregation nodes (synthesis brief), and exactly one terminal answer node. Unlike linear lists or trees, DAGs natively support cross-branch evidence reuse and hierarchical synthesis. A validity gate rejects malformed graphs before reward computation.

Two-stage training

Planner RL rewards: rubric coverage (Cᵣᵤᵦ), search quality (Qₛₑₐᵣcₕ), and graph expressiveness (Eᵧᵣₐₚₕ). Removing graph expressiveness collapses structure and hurts performance.
Answerer RL rewards: aggregation execution (Rₑₓₑc, the most critical), rubric judgment (Jᵣᵤᵦ), and branch utilization (Uᵦᵣₐₙcₕ — without it, content overlap drops from 0.82 to 0.61).
Staged training isolates plan quality from execution noise and avoids the non-stationarity of joint training.

Results

On three long-form benchmarks (DeepResearchBench, HealthBench, ResearchQA-Mini), DecomposeR-8B (SFT+RL) achieves 41.8 / 42.0 / 71.4 — the strongest open 8B model, gaining 7.4 / 22.2 / 14.4 points over Qwen3-8B + Search.

Staged vs joint training: staged wins by 3.7–4.7 points; the gap is largest on the longest-rollout benchmark (DRBench), where end-of-trajectory credit assignment hurts most.
Fixed external answerer: with GPT-4.1 executing plans, the DecomposeR planner's plans approach or exceed the GPT-5 prompt-only planning baseline — with only an 8B backbone — confirming gains trace to the planner itself.
Structure ablations: linear structure drops to 34.2/31.5/57.8; tree to 38.6/39.8/65.3; single-turn planning (no retrieval-informed revision) to 35.1/36.7/67.5 — confirming that initial plans must be calibrated against retrieval reality.

Case study

A full rollout on the query "How do tree roots in agroforestry intercropping systems reduce subsurface nitrogen loss?" shows the planner revising the DAG after seeing search results — adding a node on preferential flow, restructuring aggregation nodes so one conclusion feeds multiple downstream nodes — and producing a final report citing 11 references. The DAG makes failures diagnosable per-node rather than buried in a black-box trajectory.

Significance and limitations

DecomposeR argues for a paradigm shift: optimize research plan quality, not just final answers. An 8B model with structure-aligned training signals can rival far larger models on planning quality, and DAG plans provide human-auditable transparency crucial for clinical, policy, and scientific use.

Honest limitations include JSON parsing fragility (failed parses score zero), evaluation restricted to auto-gradable benchmarks, added training complexity, and open questions: whether the typed-DAG prior generalizes to unfamiliar domains, how to assess evidence correctness (not just relevance) against misinformation, and the deeper problem that a structurally perfect plan can still lead to false conclusions if its core assumptions are biased.

References

1. Hussain, M. A., Wu, X., & Lu, Y. (2026). *DecomposeR: Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward*. arXiv:2605.30824 [cs.AI]. 2. OpenAI. (2025). *Deep Research*. OpenAI Blog. 3. Jin, Q., et al. (2025). *Search-R1*. arXiv preprint. 4. Li, X., et al. (2026). *WebWeaver*. arXiv preprint. 5. Yao, S., et al. (2023). *ReAct: Synergizing Reasoning and Acting in Language Models*. ICLR 2023.