Summary
This arXiv paper (2608.20316) by Adam Fisch, Shubhendu Trivedi, Fantine Huot, and William W. Cohen formalizes a key tradeoff in heterogeneous AI model routing: estimating each expert model's expected value is itself costly. Cheap estimators such as embedding-based predictors are fast but noisy, while accurate estimators like fine-tuned models are expensive. The authors cast this problem as a Pandora's Box problem, a classic optimal search framework with costly inspection. Under Gaussian signal models, the resulting policies admit closed-form value-of-information expressions that determine whether refining a value estimate is worthwhile for each expert and input. They propose a centralized policy, Pandora's Router, and a decentralized variant, Pandora's Bidder, where experts independently decide whether to invest in self-assessment before accepting offers. Experiments across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—show that Pandora's Router matches the routing quality of exhaustive estimation while querying expensive estimators far less often, improving the quality-cost balance of multi-model AI systems.
Paper Overview
Research Area: ML
Authors: Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen
Published: 2026-08-22
arXiv: 2608.20316
Summary
Heterogeneous AI systems composed of multiple models, architectures, frameworks, or inference-time settings can improve quality and efficiency by routing each query to the expert that can answer it most effectively at the lowest cost. Routing, however, requires estimating the expected value of each expert—and this value estimation itself has a cost.
- The tradeoff: Cheap estimators (e.g., embedding-based predictors) are fast but noisy, while accurate estimators (e.g., fine-tuned models) are expensive.
- Formalization: The paper formalizes this tradeoff as a Pandora's Box problem, a classic framework for optimal search with costly inspection.
- Theory: Under Gaussian signal models, the resulting policies admit closed-form value-of-information expressions that determine whether it is worthwhile to refine the value estimate for each expert and input.
- Policies: The centralized strategy is called Pandora's Router; in the decentralized setting, Pandora's Bidder lets experts independently decide whether to invest in self-assessment before accepting an offer.
- Experiments: Across three domains—multi-LLM benchmarks, retrieval-augmented experts, and LLMs with variable inference times—Pandora's Router matches the routing quality of exhaustive estimation while querying the expensive estimator far less often.
*Automatically collected on 2026-08-22.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633795