Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction

| Field | Content | |---|---| | Authors | Zhuofeng Li, Haoxiang Zhang, Cong Wei, Pan Lu, Ping Nie, Yi Lu, et al. (19 authors) | | Published | 2026-05-03 | | Source | https://arxiv.org/abs/2605.05242 | | Type | Academic paper | | Section | Agentic Search |

TL;DR

Modern retrieval pipelines compress corpus access into a single top-k similarity step before reasoning. This paper proposes Direct Corpus Interaction (DCI): instead of using an embedding model, vector index, or retrieval API, the agent searches the raw corpus directly with general-purpose terminal tools (grep, file reads, shell commands, lightweight scripts). This simple setup outperforms strong sparse, dense, and reranking baselines on several BRIGHT and BEIR datasets and achieves strong accuracy on BrowseComp-Plus and multi-hop QA, with no offline indexing and no conventional semantic retriever.

Background and Motivation

Traditional pipelines split retrieval, ranking, and generation, which struggles to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. The paper's abstract identifies the core problem:

> Modern retrieval systems, whether lexical or semantic, expose a corpus through a fixed similarity interface that compresses access into a single top-k retrieval step before reasoning. This abstraction is efficient, but for agentic search, it becomes a bottleneck: exact lexical constraints, sparse clue conjunctions, local context checks, and multi-step hypothesis refinement are difficult to implement by calling a conventional off-the-shelf retriever, and evidence filtered out early cannot be recovered by stronger downstream reasoning.

Agentic tasks exacerbate this: they require discovering intermediate entities, combining weak clues, and revising plans after observing partial evidence — all poorly served by a single-shot similarity interface.

Method: Direct Corpus Interaction (DCI)

• The agent operates directly on the raw corpus via terminal tools: grep, file reads, shell commands, and lightweight scripts.
• No embedding model, no vector index, no retrieval API; no offline indexing is required.
• The approach adapts naturally to evolving local corpora (e.g., locally evolving codebases).
• Agentic orchestration handles multi-step discovery: searching, reading, combining weak clues, and refining hypotheses.

Experiments and Results

• Benchmarks: BRIGHT, BEIR (IR benchmarks); BrowseComp-Plus and multi-hop QA (end-to-end agentic tasks).
• Baselines: strong sparse, dense, and reranking methods.
• Findings: DCI substantially outperforms the baselines on several BRIGHT and BEIR datasets and attains strong accuracy on BrowseComp-Plus and multi-hop QA without any conventional semantic retriever.

Key Insight

As language agents become stronger, retrieval quality depends not only on reasoning ability but also on the resolution of the interface through which the model interacts with the corpus. DCI opens a broader interface-design space for agentic search beyond fixed similarity interfaces.

Implications for Search / Rec / Personalization

1. Architecture: cascaded retrieve-rank-generate remains mainstream, but the agentic paradigm treats retrieval count and strategy itself as a learnable object. 2. Evaluation: static nDCG is giving way to task success rate, citation accuracy, and multi-hop reasoning-chain completeness. 3. Engineering: latency, cost, explainability, and safety remain hard constraints for deployment; DCI trades offline indexing for per-query terminal tool usage, which shifts the cost profile. 4. Open problems: benchmark fidelity, cross-lingual generalization, and the security risks of agents operating over open corpora.

Related Entries

• A Systematic Framework for Enterprise Knowledge Retrieval
• Retrieval Augmented Generation and Understanding in Vision: A Survey
• Synergizing RAG and Reasoning: A Systematic Review
• AceSearcher: Bootstrapping Reasoning and Search for LLMs via RL
• AgentX: Agent-Driven Self-Iteration of Industrial Recommender Systems
• Agentic Information Retrieval

Glossary

| Term | Meaning | |---|---| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making and tool use by an agent | | DCI | Direct Corpus Interaction — searching raw corpora with general-purpose terminal tools |

References

• Original paper: Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction