Open Deep Search: Democratizing Search with Open-source Reasoning Agents
Paper: https://arxiv.org/abs/2503.20201 Authors: Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, et al. (12 authors) Published: 2025-03-26 Category: Agentic Search
Overview
Open Deep Search (ODS) aims to close the increasing gap between proprietary search AI solutions—such as Perplexity's Sonar Reasoning Pro and OpenAI's GPT-4o Search Preview—and their open-source counterparts. The core innovation is augmenting the reasoning capabilities of the latest open-source LLMs with reasoning agents that can judiciously use web search tools to answer queries.
Architecture
ODS consists of two components that work with any user-chosen base LLM:
1. Open Search Tool — a novel web search tool that outperforms proprietary counterparts. 2. Open Reasoning Agent — interprets the given task and completes it by orchestrating a sequence of actions that includes calling tools, one of which is the Open Search Tool.
Key Results
Together with powerful open-source reasoning LLMs such as DeepSeek-R1, ODS nearly matches and sometimes surpasses existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES.
- On the FRAMES evaluation benchmark, ODS improves the best existing baseline (GPT-4o Search Preview) by 9.7% in accuracy.
- DeepSeek-R1 alone achieves 82.4% on SimpleQA and 30.1% on FRAMES.
- Augmented with ODS, the same model reaches 88.3% on SimpleQA and 75.3% on FRAMES.
- A Systematic Framework for Enterprise Knowledge Retrieval
- Retrieval Augmented Generation and Understanding in Vision: A Survey
- Synergizing RAG and Reasoning: A Systematic Review
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
- Agentic Information Retrieval
ODS is a general framework for seamlessly augmenting any LLM with search and reasoning capabilities to achieve state-of-the-art performance.
Context and Significance
The work sits at the intersection of agentic search and large-scale search/recommendation systems, addressing how to redistribute responsibilities among retrieval, ranking, generation, and tool calling in the LLM era. In the classic search stack, recall handles coverage, fine ranking handles discrimination, and generation handles presentation; the new variables in the LLM era are reasoning budget and action space (whether to retrieve, how many times, and which tools to call).
Broader implications for Search / Rec / Personalization
1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm makes retrieval count and strategy themselves learnable. 2. Data: High-quality instruction data and click/session logs remain critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: Latency, cost, explainability, and safety policies are hard constraints for industrial deployment.
Limitations and Future Directions
Potential limitations include experiment scale constrained by GPU budgets, benchmark–real-user distribution mismatch, unknown cross-lingual generalization due to English-centric data, and safety risks of agent systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs/structured databases, and causal/fairness constraints for recommender systems.
Original Abstract
> We introduce Open Deep Search (ODS) to close the increasing gap between the proprietary search AI solutions, such as Perplexity's Sonar Reasoning Pro and OpenAI's GPT-4o Search Preview, and their open-source counterparts. The main innovation introduced in ODS is to augment the reasoning capabilities of the latest open-source LLMs with reasoning agents that can judiciously use web search tools to answer queries. Concretely, ODS consists of two components that work with a base LLM chosen by the user: Open Search Tool and Open Reasoning Agent. [...] For example, on the FRAMES evaluation benchmark, ODS improves the best existing baseline of the recently released GPT-4o Search Preview by 9.7% in accuracy. ODS is a general framework for seamlessly augmenting any LLMs -- for example, DeepSeek-R1 that achieves 82.4% on SimpleQA and 30.1% on FRAMES -- with search and reasoning capabilities to achieve state-of-the-art performance: 88.3% on SimpleQA and 75.3% on FRAMES.