Open Deep Search: Democratizing Search with Open-source Reasoning Agents
Paper: https://arxiv.org/abs/2503.20201 Published: 2025-03-26 Authors: Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, et al. (12 authors) Category: Agentic Search
TL;DR
Open Deep Search (ODS) closes the gap between proprietary search AI solutions (e.g., Perplexity's Sonar Reasoning Pro, OpenAI's GPT-4o Search Preview) and open-source counterparts, by augmenting open-source LLMs with reasoning agents that judiciously use web search tools.
Architecture
ODS consists of two components that work with any base LLM chosen by the user:
- Open Search Tool — a novel web search tool that outperforms proprietary counterparts.
- Open Reasoning Agent — interprets the given task and completes it by orchestrating a sequence of actions, including tool calls (one of which is the Open Search Tool).
- On the FRAMES evaluation benchmark, ODS improves the best existing baseline (GPT-4o Search Preview) by 9.7% in accuracy.
- DeepSeek-R1 alone achieves 82.4% on SimpleQA and 30.1% on FRAMES.
- Augmented with ODS, it reaches 88.3% on SimpleQA and 75.3% on FRAMES — state-of-the-art performance.
- Retrieval Augmented Generation and Understanding in Vision: A Survey — arXiv:2503.18016
- Synergizing RAG and Reasoning: A Systematic Review — arXiv:2504.15909
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via RL — arXiv:2509.24193
- Agentic Information Retrieval — arXiv:2410.09713
Results
Together with powerful open-source reasoning LLMs such as DeepSeek-R1, ODS nearly matches and sometimes surpasses existing state-of-the-art baselines on two benchmarks: SimpleQA and FRAMES.
ODS is a general framework for seamlessly augmenting any LLM with search and reasoning capabilities.
Context and Significance
The work sits at the intersection of agentic search and large-scale retrieval systems, addressing how to redistribute the responsibilities of retrieval, ranking, generation, and tool calling in the LLM era. Key takeaways:
1. Architecture: The agentic paradigm turns retrieval frequency and strategy itself into a learnable, orchestrated decision process rather than a fixed pipeline. 2. Evaluation: Benchmarks shift from static nDCG toward task success rate, citation accuracy, and multi-hop reasoning chain completeness. 3. Deployment: Latency, cost, explainability, and safety remain hard constraints for production systems — open-source parity with commercial APIs lowers adoption barriers.
Limitations and Open Questions
Potential limitations include benchmark-vs-real-user distribution mismatch, English-centric data limiting cross-lingual generalization, GPU budget constraints on experiment scale, and safety risks when agents operate on the open web (prompt injection, source poisoning, bias). Future directions include more efficient test-time compute allocation and deeper integration with knowledge graphs and structured databases.