Summary
A new arXiv paper (2608.20316) by Adam Fisch, Shubhendu Trivedi, Fantine Huot, and William W. Cohen frames LLM routing as a Pandora's Box problem from optimal search theory. Routing queries to the cheapest expert that answers them well can improve both quality and efficiency in heterogeneous AI systems, but estimating each expert's expected value has a cost: cheap estimators like embedding-based predictors are fast but noisy, while accurate estimators like fine-tuned models are expensive. The paper formalizes this trade-off under a Gaussian signal model, yielding closed-form value-of-information expressions that determine when refining an estimate is worthwhile for each expert and input. The centralized policy is called Pandora's Router; a decentralized variant, Pandora's Bidder, lets experts independently decide whether to invest in self-assessment before accepting offers. Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show Pandora's Router matches the routing quality of exhaustive estimation while querying expensive estimators far less often.
Paper Overview
Field: ML
Authors: Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen
Release date: 2026-08-22
arXiv: 2608.20316
Abstract
Heterogeneous AI systems composed of multiple models, architectures, frameworks, or inference-time settings can improve quality and efficiency by routing queries to the expert that answers them most effectively at the lowest cost. Routing requires estimating each expert's expected value, but that value estimation has a cost. Cheap estimators (such as embedding-based predictors) are fast but noisy, while accurate estimators (such as fine-tuned models) are expensive.
This paper formalizes the trade-off as a Pandora's Box problem—a classic problem of optimal search with costly inspection. Under a Gaussian signal model, the resulting policies admit closed-form value-of-information expressions that determine whether refining the value estimate is worthwhile for each expert and input.
- Centralized setting: a policy called *Pandora's Router*
- Decentralized setting: *Pandora's Bidder*, where experts independently decide whether to invest in self-assessment before accepting offers
Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show that Pandora's Router matches the routing quality of exhaustive estimation while querying expensive estimators far less often.
*Auto-collected on 2026-08-22*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633816