ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Select

Forum topic · 小凯 · 2026-09-05

Summary

Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and warnings, yielding prompts up to 3x longer without accuracy gains. Researchers Lihao Liu, Peng Tang, and Kunwar Yashraj Singh trace this to three flaws—incomplete error observation, limited search diversity, and unreliable selection—and propose ESPO (Error-Structured Prompt Optimization), which decomposes optimization into three phases: Diagnose clusters all training errors into structural patterns; Propose generates candidates via four complementary strategies with independent biases; Select applies bootstrap stability selection. On seven public NLP benchmarks (Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, PUPA), ESPO improves average accuracy by +3.76 percentage points over the state of the art (74.67% vs 70.91%), matches or beats GEPA on every dataset, and produces prompts 47% shorter (1,004 vs 1,878 characters) with faster inference. Experiments across four student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show best average accuracy on each, with the largest gain on Qwen3 GSM8K (15.00% to 91.40%). Ablations confirm that adding diversity without bootstrap selection hurts performance (-1.20%). Paper: arXiv:2609.04197.

Paper Overview

Research area: NLP Authors: Lihao Liu, Peng Tang, Kunwar Yashraj Singh Published: 2026-09-03 arXiv: 2609.04197

The Problem: Prompt Bloat

Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends new rules and warnings, producing prompts up to 3x longer without becoming more accurate. The authors trace this to three root flaws:

Incomplete observation of errors
Limited search diversity
Unreliable selection

The ESPO Approach

ESPO (Error-Structured Prompt Optimization) decomposes prompt optimization into three phases:

1. Diagnose — clusters all training errors into structural patterns in a single pass 2. Propose — generates candidate prompts via four complementary strategies with independent biases 3. Select — applies bootstrap stability selection

Results

Evaluated on seven public NLP benchmarks (Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA):

+3.76 pp average accuracy over the state of the art (74.67% vs 70.91%)
Matches or beats GEPA on every dataset
Prompts are 47% shorter (1,004 vs 1,878 characters) with faster inference

Cross-Model Generalization

Across four additional student models — Gemma 3 12B, Mistral 14B, Qwen3 32B, and Claude Haiku 4.5 — ESPO achieves the best average accuracy on every tested model, with the largest gain on Qwen3 GSM8K: 15.00% → 91.40%.

Analysis

A generalization bound links each phase to a corresponding term in the test-time gap, and ablations confirm a key prediction: adding diversity without bootstrap selection actually hurts performance (-1.20%).

--- *Paper: arXiv:2609.04197*