Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (Fusion-in-Decoder, arXiv 2007.01282)

Forum topic · 小凯 · 2026-07-05

Summary

This 2021 arXiv paper by Gautier Izacard and Edouard Grave (FAIR/ENS) proposes an approach to open-domain question answering that combines non-parametric passage retrieval with end-to-end generative sequence-to-sequence models, later known as Fusion-in-Decoder (FiD). Retrieved passages are independently encoded with a T5-based encoder, and their representations are concatenated in the decoder, allowing the model to fuse evidence from a large number of passages efficiently at inference time. Because encoding is done per-passage, the method scales to retrieving many documents without re-encoding the full context, unlike approaches that concatenate all evidence in a single encoder pass. The paper reports strong results on the Natural Questions and TriviaQA benchmarks, achieving state-of-the-art performance at the time with substantial gains over prior retrieval-augmented baselines. The work is influential in the RAG (retrieval-augmented generation) literature, illustrating a decoupled retrieve-then-generate architecture where the retriever and generator can be developed and scaled relatively independently. The forum post includes metadata, a curated list of related entries on ranking and retrieval, and engineering notes on integrating retrieval, reranking, and generation in production search systems.

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2021)

Overview

This entry indexes the arXiv paper Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering by Gautier Izacard and Edouard Grave (2021). The work is widely known by the name of its proposed architecture, Fusion-in-Decoder (FiD), and is a foundational reference in the retrieval-augmented generation (RAG) literature.

Key Points

Problem: Open-domain question answering requires answering natural-language questions using knowledge beyond what is stored in a model's parameters. The paper addresses how to combine an efficient non-parametric retrieval component with a powerful parametric sequence-to-sequence generator.
Approach (Fusion-in-Decoder):

independently

decoder

Why it works: Because evidence fusion happens only in the decoder, passages can be processed in parallel and the method scales to retrieving a large number of passages (e.g., 100) without the quadratic cost of encoding an entire concatenated context.
Results: The paper reports state-of-the-art results on the Natural Questions and TriviaQA open-domain QA benchmarks at the time of publication, outperforming prior retrieval-augmented baselines. The authors also demonstrate a variant trained with distant supervision on unlabelled questions; exact figures should be verified against the original PDF tables.
Significance: FiD showed that decoupling retrieval from generation — with a simple late-fusion mechanism — can be both scalable and highly effective, influencing subsequent RAG and generative information-retrieval systems.

Context in Search / Retrieval Research

The paper sits at the intersection of neural information retrieval and generative modeling. Classic pipelines (recall → rerank → answer) are here simplified into retrieve-then-generate, with the fusion strategy chosen to trade off accuracy against latency and memory. Later work in iterative retrieval, agentic search, and generative IR builds directly on this retrieve-and-fuse paradigm.

Related Entries

Deep Learning to Rank in Industrial Search Engines, Recommender Systems
Multi-Objective Recommendation in the Era of Generative AI: A Survey
A Generative Re-ranking Model for List-level Multi-objective Optimization
A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE

References

Izacard, G., & Grave, E. (2021). *Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering.* arXiv:2007.01282. https://arxiv.org/abs/2007.01282