SPRIG: Using Genetic Algorithms to Discover Universal System Prompts for LLMs

Forum topic · 小凯 · 2026-06-28

Summary

Task-level prompt tuning is fragile: prompts optimized for one benchmark often hurt performance on others. SPRIG (ICLR 2026) reframes the problem, asking whether a single general-purpose system prompt can lift accuracy across many tasks at once. The method builds a library of 9,000 prompt components drawn from literature and automatic extraction, then trains a ModernBERT-based reward model on 100,000 preference pairs from real benchmark evaluations. A genetic algorithm uses this reward model for cheap scoring and applies six operations—add, delete, swap, rephrase, merge, crossover—to evolve candidate prompts. Sampling 100 candidates per round on 42 benchmarks keeps real evaluation costs bounded. SPRIG alone raises average accuracy from 0.5972 to 0.6287 over chain-of-thought, and combining SPRIG with task-prompt optimizer PROTEGI reaches 0.6714. Overlap analysis shows 28% of problems are solved by only one of the two, confirming complementarity. Gains are largest on math and reasoning tasks and smallest on pure knowledge recall, defining a clear applicability boundary. The paper also notes high compute costs, weaker transfer across model families, and the need to safety-screen evolved prompts.

Key points

Core insight: Optimize the model's underlying "operating system" (system prompt), not task-specific phrasing. A well-tuned system prompt generalizes; task prompts do not.
Method overview: A genetic algorithm evolves system prompts over a 9,000-component library, guided by a ModernBERT reward model trained on 100,000 preference pairs from real benchmark runs.
Genetic operations: Add, delete, swap, rephrase, merge, and crossover—applied to scored candidates, with the bottom 50% culled each round.
Empirical results on 42 benchmarks:
Base chain-of-thought: 0.5972
SPRIG (system prompt only): 0.6287 (+5.3%)
PROTEGI (task prompt only): 0.6541
SPRIG + PROTEGI: 0.6714 (+2.6% over PROTEGI alone)
Complementarity: 54% of problems solved by both, 18% by neither, ~28% solved by only one method—system prompt captures reasoning strategy; task prompt captures format and constraints.
Task-dependent gains: Largest improvements on math and logical reasoning; near-zero gains on knowledge-recall tasks where answers depend on pretraining facts rather than inference.
Position in the landscape: SPRIG complements task-prompt optimizers (APO, PROTEGI) and context optimizers (TextGrad); the three can be stacked as layers.
Boundaries and risks:
Heavy compute (10,000 initial real evaluations, iterative re-training of the reward model).
Optimized prompts transfer poorly across model families (GPT-4 vs Llama, Gemini, Claude) due to different pretraining biases.
The 9,000-component library includes persona, behavioral, and jailbreak-style items—human safety review of the final prompt is required.
Moderate complexity outperforms maximal length; longer prompts do not necessarily help.

Practical implications for developers

System prompts are not freebies: a 5%+ lift is achievable without fine-tuning or extra data.
Invest in a reusable general system prompt containing metacognitive instructions ("identify the problem type first"), verification habits, decomposition strategies, and formatting rules.
Avoid one-size-fits-all prompts: re-evaluate after model upgrades; stack task-prompt optimization on top of the system prompt rather than replacing it.
Use SPRIG-like methods for reasoning-heavy agents (customer support bots, code assistants, analytical tools) where 5% accuracy can determine production readiness.

Why this matters

SPRIG signals a shift from prompt "engraving" to prompt "architecture": system, task, and context prompts are treated as separately optimizable layers. The reported gains are small in absolute terms but effectively free—no additional training, no extra data—and compound when combined with task-level optimizers.

References

Pryzant, R., et al. (ICLR 2026). "SPRIG: Improving Large Language Model Performance by System Prompt Optimization."
OpenReview: https://openreview.net/forum?id=VdVV24KSWK
Related methods: PROTEGI, APO, TextGrad