Open Deep Search: Democratizing Search with Open-source Reasoning Agents

Overview

• Paper: Open Deep Search: Democratizing Search with Open-source Reasoning Agents
• Authors: Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, et al. (12 authors total)
• Published: 2025-03-26
• Source: https://arxiv.org/abs/2503.20201
• Category: Agentic Search

TL;DR

Open Deep Search (ODS) closes the gap between proprietary search AI solutions (e.g., Perplexity's Sonar Reasoning Pro, OpenAI's GPT-4o Search Preview) and open-source alternatives by augmenting open-source LLMs with reasoning agents that can judiciously use web search tools.

System Architecture

ODS consists of two components that work with any base LLM chosen by the user:

1. Open Search Tool — a novel web search tool that outperforms proprietary counterparts. 2. Open Reasoning Agent — interprets the given task and completes it by orchestrating a sequence of actions, including calling tools such as the Open Search Tool.

Key Results

Together with powerful open-source reasoning LLMs such as DeepSeek-R1, ODS nearly matches and sometimes surpasses existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES.

• On the FRAMES evaluation benchmark, ODS improves the best existing baseline (GPT-4o Search Preview) by 9.7% in accuracy.
• DeepSeek-R1 alone achieves 82.4% on SimpleQA and 30.1% on FRAMES.
• With ODS, the same model reaches 88.3% on SimpleQA and 75.3% on FRAMES — state-of-the-art performance.

ODS is a general framework for seamlessly augmenting any LLM with search and reasoning capabilities.

Insights for Search / Rec / Personalization

1. Architecture: Cascaded retrieve-rerank-generate remains mainstream, but the agentic paradigm treats retrieval count and strategy itself as learnable. 2. Data: High-quality instruction data and click/session logs remain critical; synthetic data requires protection against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Product: Latency, cost, explainability, and safety policies are hard constraints for industrial deployment — one cannot optimize academic benchmarks alone.

Limitations and Future Work

Potential limitations include experiment scale constrained by GPU budgets, benchmarks that may not match real user distributions, unknown cross-lingual generalization due to English-centric data, and safety risks of agent systems operating on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommender systems.

Related Entries

• A Systematic Framework for Enterprise Knowledge Retrieval (arXiv:2512.05411)
• Retrieval Augmented Generation and Understanding in Vision: A Survey (arXiv:2503.18016)
• Synergizing RAG and Reasoning: A Systematic Review (arXiv:2504.15909)
• AceSearcher: Bootstrapping Reasoning and Search for LLMs via RL (arXiv:2509.24193)
• Agentic Information Retrieval (arXiv:2410.09713)

Original Abstract

> We introduce Open Deep Search (ODS) to close the increasing gap between the proprietary search AI solutions, such as Perplexity's Sonar Reasoning Pro and OpenAI's GPT-4o Search Preview, and their open-source counterparts. The main innovation introduced in ODS is to augment the reasoning capabilities of the latest open-source LLMs with reasoning agents that can judiciously use web search tools to answer queries. Concretely, ODS consists of two components that work with a base LLM chosen by the user: Open Search Tool and Open Reasoning Agent. Open Reasoning Agent interprets the given task and completes it by orchestrating a sequence of actions that includes calling tools, one of which is the Open Search Tool. Open Search Tool is a novel web search tool that outperforms proprietary counterparts. Together with powerful open-source reasoning LLMs, such as DeepSeek-R1, ODS nearly matches and sometimes surpasses the existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES. For example, on the FRAMES evaluation benchmark, ODS improves the best existing baseline of the recently released GPT-4o Search Preview by 9.7% in accuracy. ODS is a general framework for seamlessly augmenting any LLMs -- for example, DeepSeek-R1 that achieves 82.4% on SimpleQA and 30.1% on FRAMES -- with search and reasoning capabilities to achieve state-of-the-art performance: 88.3% on SimpleQA and 75.3% on FRAMES.