Scaling Generative Retrieval to Millions of Passages: Google Research's DSI++ Study (arXiv 2305.11841)

Forum topic · 小凯 · 2026-07-05

Summary

This Google Research paper (arXiv:2305.11841, May 2023, by Ronak Pradeep, Kai Hui, Jai Gupta, Adam D. Lelkes, Honglei Zhuang, Jimmy Lin, et al.) examines how generative retrieval — where a sequence-to-sequence model maps queries directly to document identifiers without a separate index — can scale to corpora of millions of passages. The work builds on Differentiable Search Index (DSI) and identifies the key scaling bottlenecks: memorization of document identifiers, efficiency of constrained decoding, and the ability of transformer decoders to retain and retrieve from very large corpora. The authors propose techniques such as curricum-style training strategies, identifier pruning, and training/inference optimizations that enable generative retrieval to approach the performance of strong dense retrieval baselines on large-scale open-domain QA benchmarks like Natural Questions. The paper provides analysis of where generative retrieval remains behind conventional pipeline (retrieve-then-rank) systems, particularly on index freshness and update costs. This forum post reproduces the paper's metadata with commentary on how such architectures fit into modern search stacks combining retrieval, re-ranking, and generation, and lists related work on dense retrieval, LLM-based rerankers, and generative information retrieval.

Scaling Generative Retrieval to Millions of Passages (Google Research, arXiv 2305.11841)

This post on zhichai.net indexes a Google Research paper from May 2023 that investigates how generative retrieval — jointly training a model to search and memorize documents within a single sequence-to-sequence model — scales to corpora containing millions of passages.

Paper overview

| Field | Detail | |------|--------| | Title | How Does Generative Retrieval Scale to Millions of Passages? | | Authors | Ronak Pradeep, Kai Hui, Jai Gupta, Adam D. Lelkes, Honglei Zhuang, Jimmy Lin, et al. (8 authors) | | Source | https://arxiv.org/abs/2305.11841 | | Affiliation | Google Research |

Key points

Generative retrieval replaces the traditional index-then-retrieve pipeline with a single seq2seq model that maps a query directly to a document identifier.
The study identifies scaling challenges: identifier memorization, constrained decoding efficiency, and model capacity limits as corpus size grows to millions of passages.
Training strategies (e.g., curriculum-style data ordering) and inference optimizations (e.g., identifier constraints) are explored to make large-scale generative retrieval practical.
Experiments on open-domain QA benchmarks (e.g., Natural Questions) compare generative retrieval against strong dense retrieval baselines, narrowing but not fully closing the gap.
The paper highlights remaining weaknesses of generative approaches, notably index update cost — adding or modifying documents requires retraining rather than a lightweight index refresh.

Context within information retrieval

Neural IR has evolved from sparse retrieval (BM25) to dense dual-tower encoders, late-interaction models, and cross-encoder re-rankers. Generative retrieval (e.g., the Differentiable Search Index, DSI) is a newer paradigm that removes cascade stages but introduces distinct trade-offs between efficiency, effectiveness, and maintainability. This work provides one of the first systematic studies of whether that paradigm survives web-scale corpora.

Takeaways for practitioners

1. Architecture: cascade retrieve + rank + generate remains the mainstream; generative retrieval is promising but constrained by identifier memorization and update latency. 2. Evaluation: offline metrics like nDCG@10 should be complemented by latency and cost measurements when comparing to pipelines. 3. Open problems: incremental index updates, hallucination control, and cross-lingual generalization remain open.

Related entries

Fine-Tuning LLaMA for Multi-Stage Text Retrieval (arXiv 2310.08319)
DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers (arXiv 2502.18460)
CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval

References

Original paper: https://arxiv.org/abs/2305.11841

> Note: Quantitative results should be verified against the original PDF; this entry summarizes the paper based on its abstract and public metadata.