FrugalEvo: Cost-Aware LLM-Guided Program Evolution Matches SOTA at a Fraction of the Cost

Forum topic · 小凯 · 2026-10-06

Summary

FrugalEvo is a cost-aware framework for LLM-guided program evolution presented in arXiv paper 2610.03675 by Chen et al. Unlike prior methods such as AlphaEvolve that maximize performance over a fixed number of iterations, FrugalEvo maximizes gain per unit cost. The framework pairs a stronger, more expensive LLM that explores solution strategies with a cheaper LLM that implements and iteratively refines code. A cache-efficient evolution process maximizes prefix sharing across evolution steps to improve cache reuse. The authors also introduce Budget-Aware Area Under the Curve (BA-AUC), which measures solution quality across a fixed cost budget. Across 10 mathematical and system optimization tasks, FrugalEvo matches or exceeds state-of-the-art baselines including OpenEvolve, ShinkaEvolve, AdaEvolve, and EvoX, achieving higher BA-AUC on 9 tasks and better average performance on 10 ALE-Bench-Lite algorithm optimization tasks. On circle packing, it reaches new state-of-the-art results using GPT-5.6 Terra/Luna at just $1.68, or GLM-5.3 with its Flash variant at $0.55, matching or beating multi-agent baselines like CORAL and SwarmResearch that average roughly $50 per run.

Paper Overview

Field: NLP
Authors: Hui Chen, Xuan Qi, James Xu Zhao, Zhaopeng Feng, Shilong Liu, Kuang Xu, Pang Wei Koh, Bryan Hooi
Published: 2026-10-02
arXiv: 2610.03675

Key Ideas

LLM-guided evolutionary methods such as AlphaEvolve have become powerful tools for hard computational optimization problems like circle packing. However, prior work typically optimizes performance gain over a fixed number of iterations. The FrugalEvo authors argue that practical optimization should instead maximize gain per unit cost, and propose a framework built around that principle.

Method

Cost-aware two-tier LLM design: a stronger, higher-cost LLM explores solution strategies, while a cheaper LLM implements those strategies and iteratively refines the resulting code.
Cache-efficient evolution: the harness and prompt design maximize prefix sharing across evolution steps, improving cache reuse and cutting token costs.
Budget-Aware Area Under the Curve (BA-AUC): a new metric defined as the area under the best evaluation-score curve plotted against cumulative LLM cost, up to a budget cap, measuring solution quality throughout a fixed cost budget.

Results

Across 10 mathematical and system optimization tasks, FrugalEvo matches or exceeds state-of-the-art baselines including OpenEvolve, ShinkaEvolve, AdaEvolve, and EvoX.
It achieves higher BA-AUC on 9 of the 10 tasks.
On 10 ALE-Bench-Lite algorithm optimization tasks, it achieves higher average performance.
On circle packing, FrugalEvo reaches a new state of the art at just $1.68 using GPT-5.6 Terra and Luna, or $0.55 using GLM-5.3 with its Flash variant — matching or beating all baselines, including multi-agent approaches like CORAL and SwarmResearch that average around $50.

Source

Abstract: arXiv:2610.03675

--- *Auto-collected on 2026-10-06*