A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE
This post summarizes and contextualizes the paper "A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE" (March 2024) by Hervé Déjean, Stéphane Clinchant, and Thibault Formal.
- Paper: https://arxiv.org/abs/2403.10407
- Topic area: Ranking for Search / neural information retrieval
- Problem: SPLADE is a strong sparse retrieval method; the paper studies which reranking architecture is best suited to re-score SPLADE's top candidates — classical cross-encoders (e.g., BERT/MiniLM-based) versus decoder-only LLMs.
- Comparison axes: effectiveness (nDCG@10, MRR), inference cost, latency, and score calibration across model families and sizes.
- Context: the work sits in the evolution from BM25 → BERT cross-encoders → dense bi-encoders → late interaction → generative and agentic retrieval, where each generation trades off efficiency, effectiveness, and maintainability.
- Researchers: reproduce the comparison with attention to statistical significance and compute cost; exact numbers should be verified against the paper's tables.
- Engineers: treat encoders and rerankers as pluggable modules; measure integration cost with existing stacks.
- Product teams: prioritize user-perceivable gains (latency, answer trustworthiness) over purely offline metric improvements.
- Deep Learning to Rank in Industrial Search Engines and Recommender Systems
- A Generative Re-ranking Model for List-level Multi-objective Optimization
- Accelerating Listwise Reranking: Reproducing and Enhancing FIRST (SIGIR)
- Adaptive Neural Ranking Framework: Toward Maximized Business Goal
- IR: Information Retrieval
- RAG: Retrieval-Augmented Generation
- LTR: Learning to Rank
- nDCG: Normalized Discounted Cumulative Gain, a ranking quality metric
- Agentic Search: modeling search as sequential decision-making with tool calls
- Gen-IR: Generative Information Retrieval
Key points
Typical reranking pipeline
1. Retrieval: SPLADE produces a candidate set via learned sparse representations. 2. Reranking: candidates are re-scored with either a cross-encoder (joint query–document encoding, high accuracy, no precomputable document representations) or an LLM-based scorer/listwise ranker. 3. Serving constraints: p99 latency budget, token cost, caching of popular queries, and early-stopping control practical deployment.
Engineering checklist for reranking systems
| Concern | Recommendation | |---|---| | Latency | Cascade retrieval + rerank with early stopping; cache hot queries | | Quality | Verify offline gains translate to online CTR/satisfaction via interleaving tests | | Safety | Filter poisoned or biased retrieved sources; validate citations | | Cost | Route to smaller models, distill LLM rankers, combine sparse + dense signals |