RouteLLM: Learning to Route LLMs with Preference Data (arXiv 2406.18665)

Forum topic · 小凯 · 2026-07-05

Summary

RouteLLM is a framework from UC Berkeley and LMSYS researchers (Isaac Ong, Amjad Almahairi, Wei-Lin Chiang, Joseph E. Gonzalez, et al.) presented in a June 2024 arXiv paper (arXiv:2406.18665) that reduces the cost of serving large language models by dynamically routing queries between a stronger, more expensive model and a weaker, cheaper one. The routers are trained purely on preference data, such as human pairwise comparisons from Chatbot Arena, learning a judge that predicts which model can better answer a given query. According to the paper, this approach cuts the cost of LLM inference by up to around 80% while preserving roughly 95% of GPT-4's performance on benchmarks including MT-Bench, MMLU, and GSM8K. The authors also release the router models, the benchmark curation pipeline, and evaluation results, and report that routers trained on one benchmark generalize well to others. This article summarizes the paper's motivation, method, and evaluation, and discusses implications for cost-efficient LLM deployment in production systems.

RouteLLM: Learning to Route LLMs with Preference Data (Jun 2024, arXiv)

Paper: arXiv:2406.18665
Authors: Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, et al. (8 authors, UC Berkeley / LMSYS)
Published: June 2024, arXiv

Summary

RouteLLM addresses a practical problem in LLM deployment: strong models like GPT-4 deliver high-quality answers but are expensive, while weaker models are cheap but less capable. Rather than sending every query to the strongest model, RouteLLM routes each query to the weakest model capable of answering it well, dramatically lowering serving cost.

Key Points

Routers trained on preference data: The framework trains routers purely on *preference data* — pairwise comparisons indicating which of two model responses is better (e.g., human votes from Chatbot Arena). No ground-truth answers are required.
Strong/weak model pairs: The router chooses between a strong model and a weak model per query, balancing quality against cost.
Multiple router designs: The paper explores several router architectures, including a similarity-weighted ranking router, a matrix factorization router, and BERT-style classifier routers, and compares their cost–quality trade-offs.
Reported results: The authors report up to ~2x cost reduction (up to ~80% savings in some settings) while retaining ~95% of GPT-4 quality on benchmarks such as MT-Bench, MMLU, and GSM8K.
Generalization: Routers trained on data from one benchmark transfer reasonably well to other benchmarks, and training on larger preference datasets improves performance.
Open release: The router models, training code, and benchmark curation pipeline are open-sourced so practitioners can adapt the framework to their own model pairs and traffic.

Why It Matters

RouteLLM treats *model selection* as a learnable component of the serving stack, similar to cascade systems in classic information retrieval. For production teams, the implication is that a lightweight judge model can mediate between tiers of LLMs, letting expensive capacity be spent only where it changes outcomes — a direct lever on latency, cost, and scalability.

Caveats

Readers should consult the original PDF for exact quantitative results and router-by-router comparisons; this summary is based on the paper's public abstract and metadata. Reported gains depend on the chosen strong/weak model pair, the amount and distribution of preference data, and the evaluation benchmark.

References

RouteLLM: Learning to Route LLMs with Preference Data. arXiv:2406.18665, June 2024. https://arxiv.org/abs/2406.18665

RouteLLM: Learning to Route LLMs with Preference Data (arXiv 2406.18665)

Summary

RouteLLM: Learning to Route LLMs with Preference Data (Jun 2024, arXiv)

Summary

Key Points

Why It Matters

Caveats

References

Tags