CoEvo: Coevolution of LLM and Retrieval Model for Domain-Specific Information Retrieval (EMNLP 2025)

Source: https://aclanthology.org/2025.emnlp-main.757.pdf

Overview

CoEvo, presented at EMNLP 2025, tackles domain-specific information retrieval by jointly evolving a large language model (LLM) and a retrieval model. The work responds to a core question in the LLM era: how should the responsibilities of retrieval, ranking, generation, and tool use be redistributed within modern search and recommendation systems?

Background and Motivation

Traditional pipelines treat retrieval, ranking, and generation as disconnected stages, which struggles to meet user demands for natural language interaction, multi-hop reasoning, and real-time knowledge. CoEvo targets scenarios including open-domain information access, enterprise knowledge retrieval, conversational search, and semantic understanding in recommendation systems.

Core Contributions

• A unified perspective that brings scattered related work into a comparable framework.
• A clear decomposition of method components: representation learning, retriever, reranker, planner, generator, and feedback mechanisms.
• Reproducible benchmarks, datasets, and classification tables to lower entry costs for follow-up research.
• Discussion of interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration.
• Open problems explicitly listed: evaluation credibility, latency and cost, hallucination and safety, and cross-lingual/multimodal extension.

Methodology

The typical workflow follows four steps: problem formalization, model/system design, training or construction, and inference pipeline.

1. Input & representation: encode queries, documents, and user context into dense/sparse representations or structured prompts. 2. Core modules: retriever, reranker, planner, memory modules, and tool interfaces, chained or combined per task. 3. Learning strategies: supervised fine-tuning, contrastive learning, distillation, reinforcement learning (including process rewards), and bootstrapped data synthesis. 4. Inference strategies: single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

Experiments and Evaluation

Relevant datasets and metrics in this research area typically include:

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
• Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.

Quantitative results should be verified against the original PDF tables.

Key Insights for Search / Rec / Personalization

1. Architecture: cascaded retrieval + rerank + generation remains mainstream, but agentic paradigms are making retrieval count and policy themselves learnable. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: latency, cost, interpretability, and safety are hard constraints for industrial deployment — academic benchmarks alone are insufficient.

Limitations and Future Directions

Potential limitations include experiment scale constrained by GPU budgets, mismatch between benchmarks and real user distributions, unknown cross-lingual generalization from English-centric data, and safety risks of agent systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommender systems.

Related Entries

• CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval
• DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers (arXiv:2502.18460)
• ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance
• Fine-Tuning LLaMA for Multi-Stage Text Retrieval (arXiv:2310.08319)
• How Does Generative Retrieval Scale to Millions of Passages? (Google Research, arXiv:2305.11841)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |

Recommendations for Readers

1. Researchers: reproduce the core comparative experiments; check whether statistical significance and compute costs are reported. 2. Engineers: extract pluggable modules (encoder, reranker, planner) and assess integration costs with existing stacks. 3. Product managers: focus on user-perceivable benefits (latency, answer trustworthiness, multi-turn consistency) rather than offline nDCG alone.