CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents (arXiv 2507.17084)

Forum topic · 小凯 · 2026-07-23

Summary

CodeRescue (arXiv 2507.17084, by Qijia He, Jiayi Cheng, and Chenqian Le) reformulates failure handling for coding agents as recovery routing over heterogeneous actions rather than a simple cascade. Because coding agents run in executable environments, a failed attempt yields actionable feedback, making further cheap-model recovery valuable before escalating to a stronger, more expensive model. The authors train a supervised router from execution rollouts to decide when to retry with cheap compute and when to escalate. To keep the same router usable under changing budgets, they add a Conformal Risk Control (CRC) layer that selects a deployment-time cost penalty without retraining, providing marginal expected cost guarantees under exchangeability. On held-out failures across five coding benchmarks, cheap recovery and escalation show complementary success patterns, and the calibrated frontier outperforms fixed actions, prompt-only routers, and binary cascade baselines. In a GPT-5.4-nano/GPT-5.4 setting, one CRC-calibrated frontier point exceeds the solve rate of always-escalate while using only 35% of its average recovery cost. Code is open-sourced.

Overview

Research area: ML Authors: Qijia He, Jiayi Cheng, Chenqian Le Published: 2026-07-22 arXiv: 2507.17084

Key Idea

Coding agents increasingly operate in executable environments where a failed attempt produces actionable feedback rather than merely an incorrect answer. Existing cost-aware systems typically treat such failures as cascade decisions: try a cheap model first, then escalate hard cases to a stronger and more expensive model. In coding, however, execution feedback can also make further cheap-model recovery worthwhile, raising a budgeted deployment question: when should an agent spend more cheap compute, and when should it escalate?

Method

Formulates the post-failure decision as recovery routing over heterogeneous actions.
Trains a supervised router from execution rollouts.
Adds a Conformal Risk Control (CRC) layer that selects a deployment-time cost penalty without retraining, providing marginal expected cost guarantees under exchangeability.

Results

Evaluated on held-out failure cases across five coding benchmarks.
Cheap recovery and escalation exhibit complementary success patterns.
The calibrated frontier outperforms fixed actions, prompt-only routers, and binary cascade baselines.
In the main GPT-5.4-nano/GPT-5.4 setting, one CRC-calibrated frontier point exceeds the solve rate of always-escalating while using only 35% of its average recovery cost.

Code has been open-sourced by the authors.