A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE (Mar 2024)
This post summarizes an academic paper from the *Ranking for Search* section of our reading list.
Paper Information
| Field | Content | |-------|---------| | Title | A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE | | Authors | Hervé Déjean, Stéphane Clinchant, Thibault Formal | | Published | March 2024 | | Link | https://arxiv.org/abs/2403.10407 | | Type | Academic paper |
One-Sentence Summary
The paper provides a systematic comparison of cross-encoder rerankers and LLM-based rerankers applied on top of SPLADE retrieval, examining the accuracy/efficiency trade-offs in modern search pipelines.
Background and Motivation
Large-scale search, recommendation, and personalization systems have long faced challenges around efficiency, scalability, and intent understanding. Traditional pipelines separate retrieval, ranking, and generation, which struggles to meet the demands of the LLM era: natural language interaction, multi-hop reasoning, and up-to-date knowledge. This paper addresses how to reallocate responsibilities among retrieval, reranking, and generation in that new landscape.
Positioning in the Field
Neural information retrieval has evolved from BM25 to BERT cross-encoders, bi-encoder dense retrieval, late interaction models, and now generative retrieval and LLM agents. Each generation balances the triangle of efficiency–effectiveness–maintainability:
- Dense retrieval enables millisecond-level recall via approximate nearest-neighbor search but is sensitive to domain shift and long-tail queries.
- Cross-encoders achieve high precision but cannot precompute document representations.
- LLM-based rerankers (listwise/pointwise prompting) offer strong semantic reasoning at higher inference cost.
- SPLADE provides learned sparse representations that combine lexical matching with semantic expansion.
- Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, latency, token cost
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs
- Ablations: reranking depth, number of retrieved passages, training data scale
- Deep Learning to Rank in Industrial Search Engines and Recommender Systems
- A Generative Re-ranking Model for List-level Multi-objective Optimization
- Accelerating Listwise Reranking: Reproducing and Enhancing FIRST (SIGIR)
- Adaptive Neural Ranking Framework: Toward Maximized Business Goals
Evaluation Setup Typically Reported
Key Insights for Search / Rec / Personalization
1. Architecture: cascade of retrieval + rerank + generate remains mainstream, but agentic paradigms are making retrieval count and strategy themselves learnable. 2. Data: high-quality instruction data and click/session logs remain critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human assessment. 4. Productization: latency, cost, explainability, and safety are hard constraints in industry — do not optimize only academic benchmarks.
Engineering Checklist
| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? Versioning? | Partitioned indexes, anonymization, rollback-capable embedding versions | | Latency | p99 budget? Number of retrieval steps? | Cascade + early stopping, query caching, async reranking | | Quality | Does offline gain translate to online CTR? | Interleaving experiments, human audits, citation checks | | Safety | Poisoning/bias via open retrieval? | Source allowlists, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Route to smaller models, distillation, hybrid sparse+dense |
Related Entries
Glossary
| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |
> Note: This post is an editorial summary based on the paper's abstract and public metadata. For precise quantitative results, consult the original PDF at https://arxiv.org/abs/2403.10407.