Open Deep Search: Democratizing Search with Open-source Reasoning Agents

Overview

| Field | Content | |------|------| | Title | Open Deep Search: Democratizing Search with Open-source Reasoning Agents | | Authors | Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, et al. (12 authors) | | Published | 2025-03-26 | | Source | https://arxiv.org/abs/2503.20201 | | Category | Agentic Search |

One-line Summary

ODS closes the gap between proprietary search AI solutions and open-source alternatives by augmenting open-source LLMs with reasoning agents that judiciously use web search tools.

Background and Motivation

In large-scale search, recommendation, and personalization systems, agentic search has long faced challenges around efficiency, scalability, and intent understanding. Traditional pipelines treat retrieval, ranking, and generation as separate stages, which struggles to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and real-time knowledge. ODS targets open-domain information access, enterprise knowledge retrieval, conversational search, and end-to-end architectures that combine external knowledge sources with generative models.

Core Contributions

• A unified perspective that brings scattered related work into a comparable framework.
• A clear decomposition of method components (representation learning, retrievers, rerankers, planners, generators, feedback mechanisms) for engineering practice.
• Reproducible benchmarks, datasets, and taxonomies that lower the entry barrier for follow-up research.
• Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, outlining paths from research prototypes to industrial systems.
• Explicit open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.

Method / System Architecture

The approach follows four steps: problem formalization → model/system design → training or construction → inference pipeline.

1. Input and representation: encode queries, documents, and user context into dense or sparse representations, or structured prompts. 2. Core modules: retrievers, rerankers, planners, memory modules, and tool interfaces, chained or combined by task. 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), bootstrapped data synthesis. 4. Inference strategies: single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

As stated in the paper's abstract, ODS consists of two components that work with a user-chosen base LLM:

• Open Reasoning Agent — interprets the given task and completes it by orchestrating a sequence of actions that includes calling tools.
• Open Search Tool — a novel web search tool that outperforms proprietary counterparts.

Experimental Results

Together with powerful open-source reasoning LLMs such as DeepSeek-R1, ODS nearly matches and sometimes surpasses existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES.

| Model | SimpleQA | FRAMES | |-------|----------|--------| | DeepSeek-R1 (base) | 82.4% | 30.1% | | ODS (with DeepSeek-R1) | 88.3% | 75.3% |

On the FRAMES benchmark, ODS improves on the best existing baseline, the recently released GPT-4o Search Preview, by 9.7% in accuracy.

Key Insights for Search / Rec / Personalization

1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm makes "number of retrieval steps and strategy" itself a learnable object. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: latency, cost, interpretability, and safety policies are hard constraints for industrial deployment — one cannot optimize academic benchmarks alone.

Limitations and Future Work

Possible limitations include experiment scale constrained by GPU budgets, mismatch between benchmarks and real user distributions, unknown cross-lingual generalization due to English-centric data, and safety risks of agent systems operating on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommendation systems.

Related Entries

• A Systematic Framework for Enterprise Knowledge Retrieval
• Retrieval Augmented Generation and Understanding in Vision: A Survey
• Synergizing RAG and Reasoning: A Systematic Review (arXiv:2504.15909)
• AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
• Agentic Information Retrieval (arXiv:2410.09713)

Original Abstract

> We introduce Open Deep Search (ODS) to close the increasing gap between the proprietary search AI solutions, such as Perplexity's Sonar Reasoning Pro and OpenAI's GPT-4o Search Preview, and their open-source counterparts. The main innovation introduced in ODS is to augment the reasoning capabilities of the latest open-source LLMs with reasoning agents that can judiciously use web search tools to answer queries. Concretely, ODS consists of two components that work with a base LLM chosen by the user: Open Search Tool and Open Reasoning Agent. Open Reasoning Agent interprets the given task and completes it by orchestrating a sequence of actions that includes calling tools, one of which is the Open Search Tool. Open Search Tool is a novel web search tool that outperforms proprietary counterparts. Together with powerful open-source reasoning LLMs, such as DeepSeek-R1, ODS nearly matches and sometimes surpasses the existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES. For example, on the FRAMES evaluation benchmark, ODS improves the best existing baseline of the recently released GPT-4o Search Preview by 9.7% in accuracy. ODS is a general framework for seamlessly augmenting any LLMs -- for example, DeepSeek-R1 that achieves 82.4% on SimpleQA and 30.1% on FRAMES -- with search and reasoning capabilities to achieve state-of-the-art performance: 88.3% on SimpleQA and 75.3% on FRAMES.