DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL

Source: arXiv:2509.10446 (September 2025) Authors: Rui Lu, Zhenyu Hou, Zihan Wang, Hanchen Zhang, Xiao Liu, Yujiang Li, et al. (9 authors) Category: Deep Research

Key points

• DeepDive targets agentic search systems, aiming to advance deep search agents by integrating knowledge graphs with multi-turn reinforcement learning.
• It addresses efficiency, scalability, and user intent understanding—challenges that arise when traditional pipelines keep retrieval, ranking, and generation separate, while LLM-era users expect natural language interaction, multi-hop reasoning, and up-to-date knowledge.
• Core scenarios include open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures coupling external knowledge sources with generative models.
• The work discusses interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, and outlines open problems: evaluation trustworthiness, latency and cost, hallucination and safety, and cross-lingual/multimodal extension.

> Note: Quantitative results should be verified against the original PDF; this analysis is based on the abstract and public metadata.

Method outline

The approach follows a four-step pattern: problem formulation → model/system design → training or construction → inference pipeline.

1. Input and representation — encode queries, documents, and user context into dense or sparse representations, or structured prompts. 2. Core modules — retriever, reranker, planner, memory, and tool interfaces, chained or combined per task. 3. Learning strategy — supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), and bootstrapped data synthesis. 4. Inference strategy — single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

Positioning in the field

The paper sits at the intersection of agentic search and large-scale search/recommendation systems. In the classic search funnel, recall handles coverage, ranking handles discrimination, and generation handles presentation; the LLM era adds two variables: inference budget and action space (whether to retrieve, how many times, which tools to call).

RAG and agentic search extend external knowledge access from one-shot retrieval to an iterative, verifiable, plannable process. Evaluation correspondingly shifts from static nDCG toward process-oriented metrics such as task success rate, citation accuracy, and multi-hop reasoning chain completeness.

Engineering checklist

| Item | Question | Suggestion | |------|----------|------------| | Data | Does training/indexing data contain PII? How is it versioned? | Partitioned indexes, sanitization, rollback-capable embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascading + early stop, query caching, async reranking | | Quality | Do offline gains translate to online satisfaction? | Interleaving experiments, manual audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelisting, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Small-model routing, distillation, sparse+dense hybrid retrieval |

Related entries

• A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications
• A Survey of LLM-based Deep Search Agents
• A Survey of Scientific Large Language Models
• Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents
• Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |