Accelerating Listwise Reranking: Reproducing and Enhancing FIRST (SIGIR 2025)

Forum topic · 小凯 · 2026-07-05

Summary

This SIGIR 2025 paper (ACM, DOI: 10.1145/3726302.3730287) addresses the efficiency of listwise reranking in modern search pipelines. Listwise reranking with large language models improves ranking quality over pointwise and pairwise approaches, but its heavy computational cost limits practical deployment. The work reproduces FIRST, a listwise reranking method, and proposes enhancements that accelerate the reranking process while preserving effectiveness. The paper situates itself in the broader ranking-for-search landscape, where latency, inference cost, and scalability are hard constraints for industrial systems alongside quality metrics such as nDCG@10 and MRR. The source post summarizes the paper's positioning within the retrieval-to-reranking-to-generation pipeline, discusses how reranking depth and inference budget affect end-to-end quality and cost, and cross-references related work on cross-encoder versus LLM reranking, generative multi-objective re-ranking, and industrial learning-to-rank systems. Readers are advised to consult the original PDF for exact quantitative results, as this summary is based on metadata and abstract-level information.

Accelerating Listwise Reranking: Reproducing and Enhancing FIRST (SIGIR 2025, ACM)

Source: https://dl.acm.org/doi/abs/10.1145/3726302.3730287
Venue: SIGIR 2025, ACM
Track: Ranking for Search

Overview

This paper focuses on accelerating listwise reranking, a family of techniques that use large language models (LLMs) to reorder a set of candidate documents jointly, rather than scoring each document independently (pointwise) or comparing documents in pairs (pairwise). Listwise reranking generally yields stronger ranking quality but incurs significant inference cost, which is a major obstacle to production deployment.

The work reproduces FIRST, an existing listwise reranking approach, and introduces enhancements that reduce its computational overhead while retaining its effectiveness. Reproduction-plus-improvement studies of this kind are valuable for the community because they verify reported results under controlled conditions and clarify which design choices actually drive the speed/quality trade-off.

Context: Reranking in the LLM Era

Classic search pipelines follow a cascade:

1. Retrieval / recall — cheap, high-coverage candidate generation (e.g., BM25, dense retrieval with ANN search). 2. Reranking — discriminative precision on the top candidates (cross-encoders, LLM listwise rerankers). 3. Generation / presentation — answer synthesis or result display.

In the LLM era, two new variables enter the picture: the inference budget and the action space (whether to retrieve, how many rounds, which tools to call). Listwise reranking sits at the expensive end of the cascade, so acceleration directly affects p99 latency and per-query token cost.

Typical Evaluation Setup

Papers in this area are usually evaluated on:

Datasets: MS MARCO, BEIR, Natural Questions, and domain-specific corpora.
Metrics: nDCG@10, MRR, Recall@k, plus latency and token cost.
Baselines: BM25, dense retrieval, cross-encoder rerankers (e.g., monoBERT-style), and retrieval-free LLMs.
Ablations: reranking depth, candidate list size, and training data scale.

> Note: Exact quantitative results should be taken from the original PDF; this summary is compiled from abstract-level metadata.

Practical Takeaways

Latency and cost are hard constraints. Offline nDCG gains do not automatically translate into product value if reranking adds hundreds of milliseconds per query.
Engineering checklist for reranking deployment:
Cascade + early stopping; cache results for popular queries.
Route easy queries to smaller models; distill large rerankers where possible.
Validate that offline improvements hold in interleaved online experiments.
Guard against retrieval poisoning and bias when reranking open-web content.
Reproduction matters. Independent reproductions like this one help separate genuinely effective components from noise in reported benchmarks.

Related Work

Deep Learning to Rank in Industrial Search Engines (ACM)
A Generative Re-ranking Model for List-level Multi-objective Optimization (arXiv 2505.07197)
A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE (arXiv 2403.10407)
Adaptive Neural Ranking Framework (ACM, 10.1145/3589334.3645605)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Listwise reranking | Jointly reordering a candidate list with one model pass | | Agentic Search | Modeling search as sequential decisions and tool calls |

Reference

Accelerating Listwise Reranking: Reproducing and Enhancing FIRST. SIGIR 2025, ACM. https://dl.acm.org/doi/abs/10.1145/3726302.3730287