ESPO: Error-Structured Prompt Optimization via Diagnose, Propose, and Select

Forum topic · 小凯 · 2026-09-05

Summary

ESPO (Error-Structured Prompt Optimization) is a new prompt optimization framework that addresses prompt bloat in evolutionary prompt optimizers like GEPA. The authors identify three root causes—incomplete error observation, limited search diversity, and unreliable selection—and decompose optimization into three phases: Diagnose clusters training errors into structural patterns, Propose generates candidates via four complementary strategies, and Select applies bootstrap stability selection. On seven public NLP benchmarks (Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, PUPA), ESPO improves average accuracy by +3.76 percentage points over the state of the art (74.67% vs 70.91%), matching or beating GEPA on every dataset while producing prompts 47% shorter (1,004 vs 1,878 characters) with faster inference. Experiments across four student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show best average accuracy on each, with the largest gain on Qwen3 GSM8K (15.00% to 91.40%). Ablations confirm that adding diversity without bootstrap selection hurts performance (-1.20%). Paper: arXiv:2609.04197.

Paper Overview

Field: NLP Authors: Lihao Liu, Peng Tang, Kunwar Yashraj Singh arXiv: 2609.04197

Introduction

Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and warnings, producing prompts up to 3x longer without improving accuracy. The authors trace this to three deficiencies—incomplete error observation, limited search diversity, and unreliable selection—and propose ESPO (Error-Structured Prompt Optimization).

Method

ESPO decomposes prompt optimization into three phases:

1. Diagnose — clusters all training errors into structural patterns in a single pass. 2. Propose — generates candidate prompts via four complementary strategies with independent biases. 3. Select — applies bootstrap stability selection.

Results

Evaluated on seven public NLP benchmarks: Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA.
Average accuracy improves by +3.76 percentage points over the state of the art (74.67% vs 70.91%).
Matches or exceeds GEPA on every dataset.
Prompts are 47% shorter (1,004 vs 1,878 characters), with faster inference.
Cross-model experiments on four student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show ESPO achieves the best average accuracy on every tested model; the largest gain is Qwen3 GSM8K (15.00% → 91.40%).
Generalization bounds tie each phase to a corresponding test-time gap term; ablations confirm a key prediction: adding diversity without bootstrap selection actually hurts performance (-1.20%).

Original Abstract

> Evolutionary prompt optimizers such as GEPA suffer from prompt bloat. We propose ESPO (Error-Structured Prompt Optimization), which decomposes prompt optimization into three phases: Diagnose clusters all training errors into structural patterns; Propose generates candidates via four complementary strategies; Select applies bootstrap stability selection. On seven public NLP benchmarks, ESPO improves average accuracy by +3.76 pp over the state-of-the-art while producing prompts 47% shorter.

*Auto-collected on 2026-09-05*