SIRA: Single-Shot Superintelligent Retrieval Beats Multi-Round Agents and Neural Retrievers

Overview

SIRA (SuperIntelligent Retrieval Agent) reframes retrieval-augmented agents: instead of a novice doing multi-round exploratory search that gradually learns corpus quirks, SIRA treats the LLM as a domain expert that predicts the shape of the target evidence before firing a query, then validates predictions with corpus statistics. The result is a training-free single-shot BM25 pipeline that outperforms supervised dense retrievers and RL-trained multi-round agents on BEIR.

Key points

• Single-shot vs multi-round. Most RAG agents (ReAct, IRCoT, Search-R1) rely on retrieval-context advantage, re-writing queries across rounds while accumulating evidence in long context. SIRA eliminates this: no second round, no reading of returned passages, no iterative refinement.
• Dual-side lexical enrichment.
• *Corpus side (offline, once):* An LLM reads each document and proposes candidate search terms that a user might use to find it, with the constraint that these terms must not already appear in the document. A document-frequency filter removes any term appearing in more than τ·|C| documents (too common, no discriminative power). Surviving terms are decomposed into n-grams and injected into the BM25 inverted index.
• *Query side (online, per query):* The LLM generates an "expected answer sketch"—discriminative terms likely to appear in relevant documents but absent from the query. A hard constraint prevents the LLM from guessing the actual answer (e.g., forbidden from outputting "1921" for "What year did Einstein win the Nobel Prize?"), forcing it instead toward contextual terms like "photoelectric effect" or "Royal Swedish Academy." A DF filter with the additional constraint DF > 0 drops any hallucinated term that does not exist in the enriched index.
• Final retrieval. Filtered expansions are merged with the original query into a single weighted BM25 call:
score(d) = BM25(q_orig, d) + w · BM25(q_exp, d)
• BM25 as weapon, not baseline. SIRA uses no neural retriever, no dense embeddings, no vector database, no trained encoder. Two overlooked properties make BM25 effective under LLM control:
• *IDF naturally rewards rare terms*, so domain-specific jargon (often diluted in high-dimensional embedding spaces) becomes the strongest retrieval signal.
• *Transparency and control*: term weights, mandatory inclusion/exclusion, and structured Boolean combinations are all directly tunable, unlike neural retrievers that accept only a vector and return nearest neighbors.
• Benchmark results (BEIR, 10 datasets).
• SIRA average Recall@10: 0.691
• E5 (supervised dense): 0.648
• SPLADE (learned sparse): 0.625
• BM25 baseline: 0.530
• Search-R1 (RL multi-round, E5 backend): 0.616
• SIRA tops 8 of 10 datasets in Recall@10; largest gains on SciDocs (+36%), CQADupStack (+23%), ArguAna (+14%), all datasets with severe vocabulary gap.
• NDCG@10: SIRA averages 0.572, again surpassing all baselines.
• Downstream QA answer coverage (top-10) on NQ and HotpotQA exceeds six RL-trained end-to-end QA systems, even though SIRA is a pure retriever with no answer-generation module.

Why it is counterintuitive

• "The future of retrieval is neural" — A 1970s lexical function beats neural retrievers when controlled by an LLM.
• "Agents need multi-round interaction to adapt to a corpus" — Adaptation can happen before the query is emitted; LLM parametric knowledge provides a prior over "general corpus behavior," anchored to a specific index by DF statistics.
• "Retrieval quality requires supervised training" — No clicks, no relevance labels, no query-document pairs needed. A frozen LLM plus corpus statistics outperforms supervised models.

Limitations

1. Tested only on textual corpora; image, audio, and multimodal retrieval are out of scope because BM25 cannot handle them. 2. Depends on LLM parametric knowledge; for extremely niche domains absent from training, both enrichment sides may fail, though the DF filter still prevents harmful term injection. 3. BEIR corpora are relatively static; live-updating indexes (news, social media) require re-running corpus-side enrichment. 4. Each query triggers one LLM call (cacheable), making it more expensive than vanilla BM25, but still cheaper than multi-round agent search.

Paper details

• Title: Superintelligent Retrieval Agent: The Next Frontier of Information Retrieval
• Authors: Zeyu Yang, Qi Ma, Jason Chen, Anshumali Shrivastava
• Institutions: Meta Superintelligence Labs, Rice University
• arXiv ID: arXiv:2605.06647v1 [cs.IR]
• Submission date: 2026-05-07
• Core method: Single-shot corpus-discriminative retrieval with dual-side lexical enrichment, DF filtering, and weighted BM25
• Backbone LLM: Qwen3.6-35B-A3B-FP8 (frozen)
• Best result: BEIR average Recall@10 0.691, first place on 8 of 10 datasets
• Code: Not yet released as of 2026-05-09

Takeaway

SIRA reframes the retrieval-augmented agent paradigm. The conventional approach trains agents to learn corpora through multi-round interaction. SIRA instead uses LLM parametric knowledge to pre-judge evidence shape, validates pre-judgments with lightweight corpus statistics, and compresses everything into a single BM25 call. The lesson is not that BM25 is strong, but that *who holds the steering wheel matters*: when the LLM controls the retrieval engine rather than merely expanding queries, the transparency and interpretability of lexical retrieval become decisive advantages, and IDF's preference for rare terms complements LLM-supplied domain vocabulary.