GPO: Unleashing LLMs as Prompt Optimizers via Analogies with Gradient-Based Model Optimizers (arXiv 2402.17564)

Forum topic · ✨步子哥 · 2025-11-27

Summary

This post reviews the paper "Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers" (arXiv:2402.17564). The authors draw an analogy between LLM-based prompt optimizers and gradient-based parameter optimizers, identifying two key factors in model learning: update direction and update method. Based on systematic analysis of improvement strategies for both factors, they develop GPO (Gradient-inspired Prompt Optimizer). At each optimization step, GPO retrieves relevant prompts from the optimization trajectory as update directions, applies a generation-based update using an LLM, and employs a cosine decay strategy to control the edit distance between generated and original prompts. Experiments show GPO delivers up to 56.8% additional performance gains over baselines on Big-Bench Hard and up to 62.6% on MMLU, while remaining efficient in the number of LLM calls. The work provides a new theoretical framework for automatic prompt engineering and demonstrates the promise of using LLMs to optimize prompts.

This post discusses paper arXiv:2402.17564, *Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers* (Xinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu, Yaliang Li, Ji-Rong Wen, 2024).

Background

LLM performance depends heavily on prompt quality. While manual prompt engineering is costly, automatic prompt optimization has emerged as a key research direction. A recent paradigm uses an LLM itself as a prompt optimizer, iteratively improving task prompts in natural language without touching the target model's parameters. However, how to systematically design such optimizers has remained an open question.

Core Idea: Gradient-Optimizer Analogy

The authors note the similarity between gradient-based parameter optimizers (which update model weights) and LLM prompt optimizers (which update prompt text). They distill model parameter learning into two key factors:

Update direction — analogous to gradients or momentum
Update method — analogous to learning rates, step-size schedules, etc.

The GPO Method

Based on this analogy, the paper proposes GPO (Gradient-inspired Prompt Optimizer), which performs three operations per optimization step:

1. Retrieving update direction: GPO retrieves relevant prompts from the optimization trajectory (past prompts and their scores) to indicate the improvement direction — similar to computing gradients/momentum. 2. Generation-based update: Rather than directly editing text, GPO feeds the retrieved direction prompts to an LLM (e.g., a stronger model like GPT-4) that *generates* a new improved prompt, exploring a richer discrete prompt space. 3. Cosine-based decay strategy: A cosine decay factor controls the edit distance between generated and original prompts — large edits early on, progressively smaller refinements later — mirroring learning-rate decay in gradient optimization.

Experimental Results

Evaluated on Big-Bench Hard (23 challenging reasoning tasks) and MMLU (57-subject multitask benchmark).
Compared with baselines, GPO achieves up to 56.8% additional performance gains on Big-Bench Hard and up to 62.6% on MMLU.
GPO converges within a limited number of LLM calls, remaining efficient.
Ablation studies confirm each component matters: removing direction retrieval or the cosine decay strategy degrades or destabilizes optimization.

Significance

New theoretical framework: transfers mature gradient-optimization concepts (momentum, adaptive schedules) to prompt optimization design.
Strong empirical gains: substantially raises the state of the art on automatic prompt optimization benchmarks.
Generality: applicable across tasks, domains, and model scales, since optimization happens through the LLM itself.
Paradigm shift: reinforces the trend from human-crafted prompts toward algorithmically optimized prompts, lowering the barrier to effective LLM use.

Reference

1. Xinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu, Yaliang Li, Ji-Rong Wen. *Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers*. arXiv:2402.17564, 2024.