GPO: Unleashing LLMs as Prompt Optimizers via Gradient-Based Optimizer Analogies (arXiv 2402.17564)

This post discusses arXiv paper 2402.17564, which treats LLM-based prompt optimization through the lens of gradient-based model optimizers and proposes GPO (Gradient-inspired Prompt Optimizer).

Key points

• Analogy framework: The paper identifies two core factors shared by parameter learning and prompt optimization — update direction and update method — and uses gradient optimization theory to guide prompt optimizer design.
• GPO algorithm: Each optimization step consists of three components:
1. Retrieving update direction: relevant prompts are retrieved from the optimization trajectory (analogous to gradient/momentum information) to guide improvement. 2. Generation-based update: an LLM generates a new prompt based on the retrieved direction prompts, rather than performing direct text edits, enabling richer exploration of the discrete prompt space. 3. Cosine-based decay strategy: the edit distance between generated and original prompts is controlled by a cosine decay schedule — large changes early, fine-grained adjustments later — mirroring learning-rate decay in gradient optimization.

Experimental results

• Evaluated on Big-Bench Hard (23 challenging reasoning tasks) and MMLU (57-subject multi-task benchmark).
• Compared with baselines, GPO achieves up to 56.8% additional performance gain on Big-Bench Hard and up to 62.6% on MMLU.
• GPO converges within limited iteration steps, remaining compute-efficient; ablation studies confirm that both the retrieval mechanism and the cosine decay strategy are necessary for stable, effective optimization.

Significance

• Provides a systematic theoretical framework for designing LLM prompt optimizers, rather than ad-hoc strategies.
• Supports the paradigm shift from manually engineered prompts to algorithmically optimized prompts, lowering the barrier for non-expert users.
• The approach is model- and task-agnostic, applicable across reasoning, QA, and generation scenarios with LLMs of various scales.

Conclusion

GPO demonstrates that classical gradient optimization concepts (direction, update, decay) translate effectively into natural-language prompt optimization, yielding substantial performance and efficiency gains and validating LLMs as powerful prompt optimizers.

Reference: Xinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu, Yaliang Li, Ji-Rong Wen. *Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers*. arXiv:2402.17564, 2024.