Deep Research Is Replacing Traditional RAG: From Retrieval-Augmented Generation to Autonomous Research

> Target audience: engineers and researchers already familiar with RAG who want to understand the next stage of technical evolution.

This post (translated and adapted from zhichai.net) argues that the industry is undergoing a paradigm shift from "Retrieval-Augmented Generation" (RAG) to "Deep Research"—from "the model retrieves when it needs external knowledge" to "the model is given the ability to autonomously explore the information space through multi-turn querying, verification, and reasoning."

Key points

• Deep Research is a different species, not an upgraded RAG. RAG treats external information as raw material for answers; Deep Research treats it as fuel for an extended reasoning process.
• Traditional RAG has three structural limitations: single-turn retrieval (no feedback loop between retrieval and reasoning), static fixed corpora (no real-time or cross-domain access), and zero verification of retrieved content quality.
• Reinforcement learning—not SFT or DPO—is the dominant training paradigm. SFT is expensive and imitative; DPO struggles with dynamic, path-dependent preferences. RL supports exploration and delayed rewards. R1-Searcher (arXiv:2503.05592) showed a 7B model trained with pure RL outperforms GPT-4o-mini on search-augmented QA.
• GRPO is adopted by 60%+ of Deep Research systems because it eliminates the PPO critic network (halving training cost), suits sparse rewards via group-relative baselines, and is simple to implement. Limitations include high variance at small group sizes and issues with homogeneous groups and long horizons.

Core components of a Deep Research system

| Component | Function | Representative implementations | |---|---|---| | Query Planner | Decompose research questions into executable query sequences | ReAct, Plan-and-Solve, Tree-of-Thought | | Information Gatherer | Execute retrieval, browsing, API calls | WebSearch API, Browser Tool, DB Connector | | Memory Manager | Store and organize intermediate findings | Short-term / Long-term / External Memory | | Reasoning Engine | Analyze, synthesize, verify | CoT, Self-Critique, Verifier | | Output Synthesizer | Generate structured research reports | Markdown, JSON, cited long-form text |

Development timeline: ReAct (ICLR 2023) → Self-RAG (2024) → Search-o1 (mid-2024) → OpenAI Deep Research (late 2024) → R1-Searcher (arXiv:2503.05592, early 2025) → DeepResearcher (arXiv:2505.22648, May 2025), the latter trained via large-scale RL in the real web environment.

Query planning strategies

• Sequential planning (ReAct): low overhead, fine-grained adjustment, but prone to local optima and compounding early mistakes.
• Parallel planning: high coverage via concurrent queries, but resource-intensive and requires result merging.
• Tree/Graph planning (Tree-of-Thought, Graph-of-Thought): handles complex dependencies with backtracking and cross-validation, at high implementation cost.
• Current consensus: hybrid strategies—parallel broad exploration, serial deep dives, graph-based cross-validation and synthesis.

Information acquisition: API vs. browser vs. hybrid

• Search APIs: fast, structured, cheap, parallelizable—but coarse-grained and unable to handle interactive sites.
• Browser simulation (Playwright/Selenium): full page content, dynamic loading, form/login interactions—but slow, costly, fragile.
• Hybrid (OpenAI Deep Research, DeepResearcher): API scan for candidate URLs, browser for high-value pages. DeepResearcher's experiments show hybrid is Pareto-optimal on the effect–cost curve.

Memory management: three routes and bottlenecks

1. In-context (short-term): simple but capped by context length; "needle in a haystack" degradation. 2. External (long-term) (MemGPT, ACE): write–read trade-off, heterogeneous data structuring, and forgetting/versioning challenges. 3. Hierarchical: raw fragments → local summaries → global synthesis → conclusions; layer design, cross-layer consistency, and compression loss remain unsolved.

Reward design and two-stage training

• Outcome reward: simple but sparse; credit assignment fails over long research trajectories.
• Process reward: "Which is Better for Agentic RAG Reinforcement Learning" (arXiv:2505.14069v1) finds outcome RL suffers from low exploration efficiency, gradient conflict, and sparse rewards; the best strategy is a hybrid—process rewards for early exploration, outcome rewards for final quality.
• Structured reward: multi-dimensional (coverage, conflict detection, citation accuracy, reasoning coherence, completeness). ToolRL found length-based rewards are "poison," producing redundant output.
• Pipeline: (1) cold-start SFT with a small set of high-quality human research trajectories, then (2) RL (GRPO or similar) in real or simulated environments. R1-Searcher uses an offline simulated retrieval environment; DeepResearcher trains directly against real web search and pages.

Three-phase roadmap

1. Agent Search (2024–2025, current mainstream): multi-turn retrieval + simple reasoning; humans define the question. Bottlenecks: no autonomous question-asking, session-limited depth. 2. Autonomous Research (2025–2026): hypothesis generation, research planning, cross-modal fusion, self-critique (R1-Searcher, DeepResearcher, EvoScientist). 3. Full-Stack AI Scientist (2026+): autonomous question-asking, experiment design, execution, and paper writing; original hypothesis generation and true paradigm shifts still require humans.

Commercial systems comparison

| System | Provider | Base model | Information sources | Training | |---|---|---|---|---| | OpenAI Deep Research | OpenAI | o3 / o4-mini | Web + live search | RL + human feedback | | Google Deep Research | Google | Gemini 2.5 Pro | Google Search + academic sources | SFT + RL | | DeepSeek-R1-Searcher | community | DeepSeek-R1-7B/32B | Simulated retrieval env | Pure GRPO RL | | Manus | Monica.im | Claude/GPT mix | Browser + API | Workflow orchestration |

Takeaways: OpenAI leads in product polish and citation quality but is a black box and costly; Google excels at cost and academic retrieval; DeepSeek-R1 proves open-source 7B models with pure RL reach usable levels; Manus is a general agent platform with deliverable outputs but limited availability.

Conclusions and advice for engineers

1. Deep Research is not a RAG upgrade but a fundamentally different architecture. 2. RL is the mainstream training paradigm, but reward design remains the biggest open problem. 3. Memory management is the next bottleneck for long, cross-session research. 4. The open-source ecosystem is catching up fast—Deep Research capability is about to be democratized.

Practical roadmap: short term, add multi-turn retrieval and verification (Self-RAG / ReAct) to existing RAG; medium term, introduce query planners (Tree-of-Thought / Plan-and-Solve) and hierarchical memory; long term, train domain-specific research models with GRPO.

Key paper index

1. ReAct: Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models", ICLR 2023 2. R1-Searcher: arXiv:2503.05592, "Incentivizing the Search Capability in LLMs via Reinforcement Learning", 2025-03 3. DeepResearcher: arXiv:2505.22648, "Towards Autonomous Information Seeking Agency", 2025-05 4. Process vs Outcome RL: arXiv:2505.14069v1, "Which is Better for Agentic RAG Reinforcement Learning", 2025-05 5. Deep Research Survey: arXiv:2508.12752v1, "Deep Research: A Survey of Autonomous Research Agents", 2025-08 6. GRPO: DeepSeek-R1 technical report, 2025-01

> Author's website: TGLTommy.com