Summary
A paper by Adam Fisch, Shubhendu Trivedi, Fantine Huot, and William W. Cohen (arXiv:2608.20316) frames LLM routing as a Pandora's Box problem, the classical optimal search problem with costly inspection. Heterogeneous AI systems can route queries to the expert model that answers most effectively at the lowest cost, but estimating each expert's expected value is itself costly: cheap estimators such as embedding-based predictors are fast but noisy, while accurate estimators such as fine-tuned models are expensive. Under a Gaussian signal model, the derived policies admit closed-form value-of-information expressions that decide whether refining the value estimate for each expert-input pair is worthwhile. The centralized strategy is called Pandora's Router; the decentralized variant, Pandora's Bidder, lets experts independently decide whether to invest in self-assessment before accepting offers. Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show Pandora's Router matches the routing quality of exhaustive estimation while querying expensive estimators far less often.
Overview
Research Area: Machine Learning
Authors: Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen
Published: 2026-08-22
arXiv: 2608.20316
Summary
Heterogeneous AI systems composed of multiple models, architectures, frameworks, or inference-time settings can improve quality and efficiency by routing queries to the expert that answers most effectively at the lowest cost. Routing, however, requires estimating the expected payoff for each expert—and that estimation itself has a cost. Cheap estimators (such as embedding-based predictors) are fast but noisy, while accurate estimators (such as fine-tuned models) are expensive.
This paper formalizes the trade-off as a Pandora's Box problem—the classical optimal search problem with costly inspection. Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions, determining for each expert and input whether it is worthwhile to refine the value estimate.
- Pandora's Router: the centralized policy.
- Pandora's Bidder: the decentralized setting, where experts independently decide whether to invest in self-assessment before accepting an offer.
Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show that
Pandora's Router matches the routing quality of exhaustive estimation while querying the expensive estimator far less often.
*Auto-collected on 2026-08-22*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633816