Is Grep All You Need? Why a 1974 Tool Beats Vector Retrieval in Agentic Search

Background: RAG and the vector retrieval consensus

In modern RAG (Retrieval-Augmented Generation) systems, the standard pipeline is: chunk documents, embed them into 768- or 1536-dimensional vectors with models like BERT or text-embedding-ada-002, store them in vector databases (FAISS, Milvus, Pinecone) with approximate nearest neighbor (ANN) indexes, and retrieve the top-K similar chunks per query.

This approach offers semantic understanding, scalability, and generality — but carries hidden costs:

• Semantic drift: embedding is lossy compression; it handles broad topics well but often struggles with exact matches (specific function names, error codes).
• Indexing overhead: every new document requires embedding and index updates, hurting real-time use cases.
• Black-box results: irrelevant retrievals are hard to explain or debug.
• Domain adaptation: general-purpose embeddings underperform in specialized domains like law, medicine, and specific codebases.

The surprise: grep wins

A Google DeepMind team's new paper poses a heretical question: what if plain grep — the 1974 Unix text-search tool — simply works better? Grep is deterministic, fully explainable, extremely fast on GB-scale text, and requires zero indexing: new documents are searchable instantly.

Experimental setup

The paper asks how retrieval strategy interacts with agent architecture and tool-call paradigms:

• Dataset: 116 samples from LongMemEval, a long-context memory benchmark.
• Agent harnesses: Chronos (the team's custom framework), Claude Code, Codex, and Gemini CLI.
• Retrieval strategies: grep (keyword) vs. vector (semantic similarity).
• Tool-result presentation: inline vs. file-based (where the agent must actively read a file).

A second experiment progressively mixes in more irrelevant conversation history to test robustness under noise.

Key findings

1. grep > vector, consistently. Across all harnesses — both the custom Chronos and the vendor CLIs — grep achieved generally higher accuracy than vector retrieval. 2. The harness matters more than the retrieval strategy. Even with identical underlying dialogue data, overall scores depend heavily on which harness and tool-calling style is used. 3. Noise amplifies the gap. As irrelevant history grows, grep's deterministic matching stays precise, while vector retrieval's semantic drift lets similar-but-irrelevant passages crowd out the relevant ones.

Why grep thrives in agentic search

• Agent agency compensates for grep's "dumbness". Unlike passive RAG, agents can query repeatedly, refine keywords, and cross-validate — turning a crude tool into an effective one.
• Exact matching fits technical queries. For queries like "Connection timeout", grep finds the right error line, while vector search may rank semantically-similar-but-irrelevant results (e.g., connection pooling docs) higher.
• Vector retrieval's "hallucination" problem. Semantic generalization becomes semantic confusion; grep never "overthinks".
• Tool-call style matters. File-based results (requiring an explicit read action) versus inline results significantly affected outcomes, suggesting retrieval must be designed as part of the whole agent workflow.

Limitations and caveats

• LongMemEval mostly covers technical dialogue; grep's advantage may not transfer to open-domain QA, creative writing, or medical scenarios.
• Grep cannot handle synonyms, cross-language queries, or semantic reasoning (e.g., "models faster than GPT-4" → Claude 3.5 Sonnet); vector retrieval remains necessary there.
• The paper's conclusion is not "replace vector retrieval with grep", but that retrieval strategy should be evaluated systematically together with agent architecture and tool-call paradigm. Hybrid retrieval (grep for exact match, vectors for semantic supplement) is likely the practical optimum.
• Vendor CLIs are black boxes; the precise mechanism of the "harness effect" remains hard to explain.

Outlook: context-aware retrieval selection

The paper's biggest contribution is opening an overlooked research direction: retrieval strategy itself should be adaptive. Future agentic systems may classify queries (exact vs. semantic), pick grep, vectors, or a mix accordingly, switch strategies dynamically on poor first results, and learn per-strategy success rates from feedback.

As the author concludes: this is not regression but evolution — a more mature, pragmatic, outcome-driven engineering mindset. Sometimes a 1974 tool still has an irreplaceable place in 2026 AI systems, because a tool's value depends on the scenario.

References

• Kasturi, A., Lumer, E., Gulati, A., & Subbiah, V. K. (2026). *Is Grep All You Need? How Agent Harnesses Reshape Agentic Search*. arXiv:2605.15184.
• Lewis, P., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. *Advances in Neural Information Processing Systems*, 33, 9459-9474.
• Vaswani, A., et al. (2017). Attention is all you need. *Advances in Neural Information Processing Systems*, 30.