RankZephyr: Effective and Robust Zero-Shot Listwise Reranking with Open-Source LLMs

Forum topic · 小凯 · 2026-07-05

Summary

RankZephyr is a 7B-parameter open-source large language model, introduced by Pradeep, Sharifymoghaddam, and Lin (University of Waterloo) in December 2023 (arXiv:2312.02724), that performs zero-shot listwise reranking for information retrieval. The authors fine-tune a Mistral-7B model on permutations synthesized by GPT-3.5, converting pairwise reranking instructions into listwise-style data to teach sliding-window listwise reranking behavior. RankZephyr substantially outperforms RankVicuna, an earlier open-source reranking model built on Vicuna-7B, while using roughly one-quarter of the number of documents required by GPT-4-based listwise reranking to achieve strong effectiveness on TREC-DL and BEIR benchmarks. The model is robust to the quality of the first-stage retrieval, performing well whether candidates come from BM25 or dense retrieval, and with other open-source retrievers. Model weights and inference code are released to promote reproducibility and further research on open, efficient LLM-based reranking pipelines. This entry links RankZephyr to related work on cross-encoder rerankers, listwise reranking acceleration, and neural ranking in industrial search systems.

RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!

Authors: Ronak Pradeep, Sahel Sharifymoghaddam, Jimmy Lin (University of Waterloo)
Published: December 2023
Paper: https://arxiv.org/abs/2312.02724
Category: Ranking for Search

One-line summary

RankZephyr is an open-source 7B-parameter LLM that achieves effective and robust zero-shot listwise reranking, outperforming prior open rerankers while using far fewer candidate documents than GPT-4-based approaches.

Key points

Listwise reranking with an open LLM: RankZephyr is fine-tuned to perform sliding-window listwise reranking, following the listwise paradigm popularized by RankGPT but using a fully open-source model instead of a closed API.
Training approach: The model is a fine-tuned Mistral-7B. Training permutations are synthesized by GPT-3.5; the authors convert pairwise-style reranking instructions into listwise-format training data, enabling the model to learn listwise reranking behavior without human labels.
Effectiveness: RankZephyr substantially outperforms RankVicuna (a Vicuna-7B-based reranker from the same group's earlier work) on standard benchmarks.
Efficiency: It achieves strong effectiveness with roughly one-quarter of the number of documents needed by GPT-4-based listwise reranking, reducing inference cost.
Robustness: The model is robust to the quality of first-stage retrieval, working well whether candidates come from BM25 (sparse retrieval) or dense retrieval methods.
Release: Model weights and inference code are made publicly available to support reproducible research on LLM-based reranking.

Positioning in the literature

RankZephyr sits in the neural ranking lineage that progressed from BM25 to BERT cross-encoders, dense bi-encoders, and instruction-following LLM rerankers (RankVicuna, RankGPT). It demonstrates that a relatively small open-source model, trained on synthetic permutations, can approach the reranking effectiveness of much larger commercial models at a fraction of the cost — a key step toward practical, deployable LLM reranking pipelines.

Related entries

Deep Learning to Rank in Industrial Search Engines, Recommender Systems
A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE
Accelerating Listwise Reranking: Reproducing and Enhancing FIRST
Adaptive Neural Ranking Framework: Toward Maximized Business Goal

References

Original paper: RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!