DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL

This post summarizes an arXiv paper (September 2025) indexed in a Deep Research section of a Chinese tech forum's reading list.

Metadata

| Field | Content | |-------|---------| | Title | DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL | | Authors | Rui Lu, Zhenyu Hou, Zihan Wang, Hanchen Zhang, Xiao Liu, Yujiang Li, et al. (9 authors total) | | Published | September 2025 (arXiv) | | Source | https://arxiv.org/abs/2509.10446 | | Type | Academic paper | | Section | Deep Research |

Key points

• The paper targets agentic search: rather than a fixed retrieve–rerank–generate pipeline, modern LLM-based search agents treat the number of retrieval steps and the choice of tools as learnable decisions, constrained by inference budget and action space.
• Core application scenarios include open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures that couple external knowledge sources with generative models.
• The post describes a typical four-stage methodology: problem formalization → model/system design → training pipeline → inference pipeline.
1. Input & representation: encode queries, documents, and user context into dense/sparse representations or structured prompts. 2. Core modules: retriever, reranker, planner, memory, and tool interfaces, chained or combined per task. 3. Learning: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), and bootstrapped data synthesis. 4. Inference: single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.
• As the title indicates, the approach combines knowledge graphs (structured synthetic knowledge sources) with multi-turn reinforcement learning for training deep search agents. Specific quantitative results should be checked against the original PDF; the forum post is largely template-based and notes that numbers must be verified from the source.

Evaluation context

The post outlines standard experimental protocol for this area:

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
• Baselines: BM25, dense retrieval, cross-encoder rerankers, retrieval-free LLMs, commercial search APIs.
• Ablations: contribution of retrieval steps, reranking depth, and training data scale.

Insights for search / recommendation practitioners

1. Architecture: cascaded retrieval + rerank + generation remains mainstream, but the agentic paradigm makes retrieval strategy itself learnable. 2. Data: high-quality instruction data and click/session logs are critical; synthetic data requires safeguards against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Productization: latency, cost, interpretability, and safety are hard industrial constraints, not just academic benchmark optimization.

Limitations and open problems

Potential limitations include experiment scale limited by GPU budget, benchmark/real-user distribution mismatch, English-centric data with unknown cross-lingual generalization, and safety risks of agents operating on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommendation.

Cross-references from the post

• A Comprehensive Survey of Deep Research (arXiv:2506.12594)
• A Survey of LLM-based Deep Search Agents (arXiv:2508.05668)
• A Survey of Scientific Large Language Models (arXiv:2508.21148)
• Towards Scientific Intelligence: LLM-based Scientific Agents (arXiv:2503.24047)
• AgentIR: Reasoning-Aware Retrieval for Deep Research Agents (arXiv:2603.04384)
• Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning (arXiv:2502.04644)

Engineering checklist (from the post)

| Check | Question | Suggestion | |-------|----------|------------| | Data | Does training/index contain PII? How are versions managed? | Partitioned indexes, anonymization, rollback-capable embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascading + early stop, hot-query caching, async reranking | | Quality | Do offline gains translate to online CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Route to smaller models, distillation, hybrid sparse+dense |

Glossary

• IR: Information Retrieval
• RAG: Retrieval-Augmented Generation
• LTR: Learning to Rank
• nDCG: Normalized Discounted Cumulative Gain
• Agentic Search: modeling search as sequential decision-making and tool calling
• Gen-IR: Generative Information Retrieval

Reference

• Original paper: DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL