Summary
A paper by Adam Fisch, Shubhendu Trivedi, Fantine Huot, and William W. Cohen (arXiv:2608.20316) formalizes AI model routing as a Pandora's Box problem, the classic optimal search problem with costly inspection. Heterogeneous AI systems can improve quality and efficiency by routing queries to the cheapest expert that answers them well, but estimating each expert's expected value carries a cost: cheap estimators like embedding-based predictors are fast yet noisy, while accurate ones like fine-tuned models are expensive. The paper derives closed-form value-of-information expressions under a Gaussian signal model, determining when it is worthwhile to refine a value estimate for each expert and input. Two policies are proposed: Pandora's Router, a centralized strategy, and Pandora's Bidder, a decentralized setting where experts independently decide whether to invest in self-assessment before accepting offers. Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show Pandora's Router matches the routing quality of exhaustive estimation while querying expensive estimators far less often.
Paper Overview
Research area: ML
Authors: Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen
Published: 2026-08-22
arXiv: 2608.20316
Summary
Heterogeneous AI systems—composed of multiple models, architectures, frameworks, or inference-time settings—can improve quality and efficiency by routing each query to the expert that can answer it most effectively at the lowest cost. Routing, however, requires estimating the expected payoff for every expert, and this value estimation itself has a cost. Cheap estimators (e.g., embedding-based predictors) are fast but noisy, while accurate estimators (e.g., fine-tuned models) are expensive.
This paper formalizes the trade-off as a Pandora's Box problem—the classic optimal search problem with costly inspection. Under a Gaussian signal model, the resulting policies admit closed-form value-of-information expressions, which determine whether it is worthwhile to refine the value estimate for each expert and input.
Two variants are introduced:
- Pandora's Router: a centralized routing policy
- Pandora's Bidder: a decentralized setting where experts independently decide whether to invest in self-assessment before accepting offers
Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show that Pandora's Router matches the routing quality obtained from exhaustive estimation while querying expensive estimators far less frequently.
*Auto-collected on 2026-08-22*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633795