InfoGain-RAG: Boosting Retrieval-Augmented Generation via Document Information Gain-based Reranking and Filtering (EMNLP 2025)
- Paper: InfoGain-RAG: Boosting Retrieval-Augmented Generation through Document Information Gain-based Reranking and Filtering
- Venue: EMNLP 2025 (main conference), November 2025
- PDF: https://aclanthology.org/2025.emnlp-main.365.pdf
- Topic area: RAG (Retrieval-Augmented Generation)
- Motivation: Traditional RAG pipelines treat retrieval, ranking, and generation as disconnected stages, which struggles to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge.
- Core idea: Quantify the marginal informational contribution of each retrieved document and use that signal to rerank and filter the context window, so the generator sees only high-value documents.
- Scope of scenarios: open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures coupling external knowledge sources with generative models.
- Framing contributions: a unified perspective for comparing retrieval/reranking components; a decomposition of the RAG stack (retriever, reranker, planner, generator, feedback mechanisms); and discussion of interfaces with LLM tool calling, reinforcement learning, and multi-agent collaboration.
- Typical datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora.
- Typical metrics: nDCG@10, MRR, Recall@k, Hit@k, task success rate, latency, and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, no-retrieval LLMs, and commercial search APIs.
- A Survey of Graph Retrieval-Augmented Generation for Customized LLMs (arXiv:2501.13958)
- A Survey on Retrieval-Augmented Text Generation for LLMs (arXiv:2404.10981)
- Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG (arXiv:2501.09136)
- RAFT: Adapting Language Models to Domain-Specific RAG (Jul 2024)
- RAG vs. GraphRAG: A Systematic Evaluation and Key Insights (arXiv:2502.11371)
Overview
InfoGain-RAG addresses a recurring weakness in standard RAG pipelines: retrieved documents are typically ranked by surface relevance, yet some relevant-looking documents add little new information for answering the query, while noisy passages degrade generation. The paper proposes using document information gain — a measure of how much each document actually contributes toward resolving the query — as the basis for reranking and filtering the retrieved set before generation.
Key points
Pipeline context
The work follows the general RAG design pattern:
1. Input & representation — encode queries, documents, and user context into dense/sparse representations or structured prompts. 2. Core modules — retriever, reranker, planner, memory, and tool interfaces, chained or run in parallel. 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, RL (including process rewards), and synthetic data. 4. Inference strategies — single-pass retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.
InfoGain-RAG's distinguishing step is inserting an information-gain-based reranking/filtering stage between retrieval and generation.
Evaluation considerations
Specific numerical results should be verified against the original PDF.
Takeaways for practitioners
1. Architecture: cascaded retrieve → rerank → generate remains mainstream, but agentic paradigms are making retrieval count and strategy themselves learnable. 2. Data: high-quality instruction data and session logs matter as much as model design; synthetic data needs leakage safeguards. 3. Evaluation: offline metrics increasingly diverge from online satisfaction; LLM-as-judge should be cross-validated with human assessment. 4. Deployment: latency, cost, explainability, and safety are hard constraints in production — do not optimize academic benchmarks alone.