CoEvo: Coevolution of LLM and Retrieval Model for Domain-Specific Information Retrieval (EMNLP 2025)
Source: https://aclanthology.org/2025.emnlp-main.757.pdf
> Note: This forum entry is based on the paper's title, abstract metadata, and public listing. The original abstract text was not fully available; quantitative claims should be verified against the PDF.
Overview
CoEvo, published at EMNLP 2025 (main conference), tackles domain-specific information retrieval through a coevolution framework: instead of treating retrieval and generation as fixed, independently trained pipeline stages, the LLM and the retrieval model are iteratively improved against each other. This addresses a known weakness of traditional cascaded search stacks—separate retrieval, ranking, and generation stages that adapt poorly to LLM-era requirements such as natural-language interaction, multi-hop reasoning, and up-to-date knowledge.
Key points from the surrounding analysis
- Problem context: Open-domain search, enterprise knowledge retrieval, conversational search, and recommendation all face tension between efficiency, scalability, and intent understanding. LLMs add new variables: inference budget and action space (whether to retrieve, how many times, which tools to call).
- Architectural framing: Typical systems decompose into input/representation encoding, core modules (retriever, reranker, planner, memory, tool interfaces), learning strategies (supervised fine-tuning, contrastive learning, distillation, RL with process rewards, synthetic data bootstrapping), and inference strategies (single-pass, iterative, parallel sub-queries, early stopping with budget control).
- Evaluation practice: Standard protocols use datasets such as MS MARCO, BEIR, and Natural Questions, with metrics like nDCG@10, MRR, Recall@k, plus latency and token cost. Baselines typically include BM25, dense retrieval, cross-encoder reranking, and retrieval-free LLMs.
- Industry insights: 1. Cascaded retrieve-rerank-generate remains dominant, but agentic paradigms make retrieval count and policy learnable. 2. High-quality instruction data and click/session logs matter as much as model design; synthetic data risks leakage and distribution shift. 3. The gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Latency, cost, explainability, and safety are hard production constraints.
- Open problems: Benchmark-vs-real-user distribution mismatch, English-centric training data limiting cross-lingual generalization, retrieval poisoning and bias in open web access, and efficient test-time compute allocation.
- CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval
- DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers (arXiv:2502.18460)
- ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance
- Fine-Tuning LLaMA for Multi-Stage Text Retrieval (arXiv:2310.08319)
- How Does Generative Retrieval Scale to Millions of Passages? (Google Research, arXiv:2305.11841)
- How to Make Cross Encoder a Good Teacher for Efficient Image-Text Retrieval (arXiv:2407.07479)
Engineering checklist (summarized)
| Area | Concern | Recommendation | |------|---------|----------------| | Data | PII, index versioning | Partitioned indexes, anonymization, rollback-capable embedding versions | | Latency | p99 budget, retrieval steps | Cascades with early stop, query caching, async reranking | | Quality | Offline-to-online transfer | Interleaving experiments, human audits, citation verification | | Safety | Poisoning/bias in open retrieval | Source allowlists, adversarial detection, output filtering | | Cost | Token/GPU per query | Small-model routing, distillation, hybrid sparse+dense retrieval |