DecomposeR: Planner-Centric RL for Deep Research — How a Typed DAG Solves Credit Assignment

Forum topic · 小凯 · 2026-06-01

Summary

A team at the National University of Singapore (Hussain, Wu, and Lu; arXiv:2605.30824, cs.AI) proposes DecomposeR, a planner-centric reinforcement learning framework for AI deep research systems. Instead of training on flat ReAct-style trajectories where a single scalar reward from the final answer is propagated to every action, DecomposeR externalizes the research plan as a typed DAG containing search nodes, aggregation nodes, and a terminal answer node. A planner emits and revises the DAG based on retrieved evidence, while an answerer executes it in topological waves. Training is staged into cold-start SFT, Planner RL (rewards: rubric coverage, search quality, graph expressiveness), and Answerer RL (rewards: aggregation execution, rubric judging, branch utilization), decoupling plan quality from execution noise. A Qwen3-8B backbone reaches 41.8 on DeepResearchBench, 42.0 on HealthBench, and 71.4 on ResearchQA-Mini, and in fixed-answerer experiments its plans approach GPT-5 planning quality. Ablations show typed DAGs beat linear and tree structures, and staged training outperforms joint RL by 3.7-4.7 points. The framework makes AI research plans auditable, valuable for clinical evidence aggregation and scientific synthesis, though limitations include JSON parsing fragility, human evaluation gaps, and evidence-quality assessment.

DecomposeR: Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward

Paper metadata

| Attribute | Content | |---|---| | Title | DecomposeR: Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward | | Authors | Mustafa Anis Hussain, Xinle Wu, Yao Lu | | Affiliation | National University of Singapore | | arXiv ID | 2605.30824 | | Date | 2026-05-29 | | Category | cs.AI | | Core claim | Deep research planning and execution should be decoupled; research plans should be externalized as typed DAGs so that planning itself becomes a policy object that can be independently rewarded and optimized |

Key points

The bottleneck in AI deep research is planning, not search. Frontier systems (OpenAI Deep Research, Perplexity) trained on flat ReAct-style trajectories suffer from ambiguous credit assignment and sparse rewards: a single scalar answer reward cannot tell whether failure came from a bad plan, poor execution, or missed evidence. Planning also remains implicit and unauditable — dangerous in high-stakes domains like clinical evidence aggregation and scientific synthesis.
From flat trajectories to typed DAGs. DecomposeR interposes an explicit plan 𝒢 between a planner policy πP and an answerer policy πA. The workflow: emit an initial DAG → the environment executes search nodes → the planner revises the DAG given observations → the answerer fills aggregation nodes in topological waves and produces a cited report. Nodes are typed (search, aggregation, terminal answer), and DAG structure natively expresses cross-branch evidence reuse and hierarchical synthesis, which trees (no shared nodes) and linear plans (no aggregation) cannot. A validity gate rejects malformed graphs before reward computation.
Staged training decouples credit. After cold-start SFT, Planner RL optimizes only plan-dependent rewards — rubric coverage, search quality, and graph expressiveness (branch breadth, evidence reuse, cross-branch integration) — never seeing answerer output. Answerer RL is then trained on a frozen planner with aggregation-execution, rubric-judging, and branch-utilization rewards. Staged training beats joint RL by 3.7–4.7 points, with the largest gap on the longest rollouts.

Main results

On a Qwen3-8B backbone across three benchmarks (DeepResearchBench, HealthBench, ResearchQA-Mini):

DecomposeR-8B (SFT+RL): 41.8 / 42.0 / 71.4 — the strongest open same-scale model, beating Qwen3-8B + Search by 7.4 / 22.2 / 14.4 points.
Fixed external answerer experiment: plans from the DecomposeR planner, executed by GPT-4.1, produce answers approaching or exceeding a GPT-5 prompt-only planning baseline — confirming gains trace to the planner itself, not answerer compensation.
Structure ablations: linear plans drop to 34.2 / 31.5 / 57.8; trees reach 38.6 / 39.8 / 65.3; the typed DAG wins everywhere, especially on tasks needing cross-branch synthesis.
No revision turn (single-turn planning): 35.1 / 36.7 / 67.5 — showing initial plans must be calibrated against retrieval reality.

Case study and auditability

A full rollout on "How do tree roots reduce subsurface nitrogen loss in agroforestry intercropping?" shows the planner adding a priority-flow search node and restructuring aggregation nodes after seeing search results, culminating in an 11-source report. Crucially, any report defect maps to a specific DAG node — turning credit assignment from a black-box trajectory into structured diagnosis. The authors argue this transparency is infrastructure for medical safety and scientific integrity, not a luxury.

Limitations and open questions

JSON parsing failures can zero out a rollout (partially mitigated).
Evaluation relies on automatic scoring; human-expert alignment frameworks remain unbuilt.
Two-stage training adds pipeline complexity.
Generalization of the typed-DAG prior to truly unfamiliar domains and more creative research forms is untested.
Search-quality rewards cannot distinguish "relevant but false" from "relevant and true" evidence; the authors suggest future adversarial verification (a devil's-advocate agent attempting to derive opposite conclusions from a DAG).

Conclusion

An 8B model approaching GPT-5 planning quality supports a counterintuitive conclusion: the bottleneck of deep research is not model scale but training paradigm. When planning is explicit, structured, and independently rewarded, small models exhibit research capabilities that flat-trajectory training cannot unlock — and structured training objectives may matter more than more parameters.

References

1. Hussain, M. A., Wu, X., & Lu, Y. (2026). *DecomposeR: Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward*. arXiv:2605.30824 [cs.AI]. 2. OpenAI. (2025). *Deep Research*. OpenAI Blog. 3. Jin, Q., et al. (2025). *Search-R1*. arXiv preprint. 4. Li, X., et al. (2026). *WebWeaver*. arXiv preprint. 5. Yao, S., et al. (2023). *ReAct: Synergizing Reasoning and Acting in Language Models*. ICLR 2023.