Eevee Deep Dive: How Routed Prompt Partitions Beat Catastrophic Forgetting for LLM Agents Facing Mixed Real-World Tasks

Forum topic · 小凯 · 2026-06-14

Summary

Eevee is a test-time prompt learning framework for self-improving LLM agents, proposed by researchers from Shanghai Jiao Tong University and Princeton (arXiv:2606.11182). Existing methods like GEPA and ACE optimize a single prompt per dataset, which fails in real-world settings where agents face heterogeneous mixed task streams: optimizing one task type degrades others (cross-dataset interference), with cumulative retention dropping to -15.36 and -18.58 respectively. Eevee maintains a set of specialized prompt slots plus a learned router that assigns each input to the right slot, and co-evolves router and prompts via alternating optimization across three stages (initialization with Pareto-front greedy coverage, lightweight exploration, and convergence refinement). On mixed GPQA Diamond, Formula, TheoremQA, and HumanEval benchmarks, Eevee improves Qwen3-4B by +10.38 points (51.75 average) and DeepSeek-V3.2 by +24.32 (64.07), achieving cumulative retention of +41.53 while costing only ~4.3k tokens per example versus ACE's 21.3k. Learned prompts also transfer across models and tasks. Ablations show learned routing and co-evolution are both essential; manually written routing underperforms even the baseline. Case studies suggest prompt learning excels at reusable procedures but can underweight domain knowledge.

Eevee Deep Dive: How Routed Prompt Partitioning Beats Catastrophic Forgetting for LLM Agents Facing Real-World Task Mixtures

> Paper: *Eevee: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents* > Authors: Weixian Xu (SJTU), Shilong Liu (Princeton), Mengdi Wang (Princeton) > Paper: https://arxiv.org/abs/2606.11182 > Code: https://github.com/Princeton-AI2-Lab/EEVEE

1. The Problem: Why Existing Prompt Learning Fails in the Real World

Test-time prompt learning is a lightweight path for LLM agents — instead of fine-tuning weights, the agent automatically improves its prompts from deployment feedback. Methods like GEPA, ACE, and Combee have proven this works.

But they share a critical blind spot: they are almost all designed for a single dataset.

Real-world agents face heterogeneous tasks simultaneously — math derivations, code completion, knowledge QA, formula calculation — each with different formats, evaluation rules, and capability demands. Absorbing all this feedback into one prompt inevitably causes learning new while forgetting old: improving code performance degrades math reasoning, and vice versa.

Figure 1 of the paper shows this clearly: as GPQA Diamond, Formula, TheoremQA, and HumanEval join the learning stream one by one, the cumulative retention of GEPA and ACE keeps falling, ending at -15.36 and -18.58 — worse than never learning at all.

This is cross-dataset interference: a single prompt cannot simultaneously carry multiple task-specialized behaviors.

2. Eevee's Core Idea: Routing the Mixture into Specialized Slots

Eevee's solution is intuitive: since one prompt cannot hold all tasks, maintain a set of specialized prompts, and use a routing module to decide which prompt each input takes.

Inference flow (Figure 2): 1. Input → Router selects prompt slot z = R(x; P) 2. The slot's specialized prompt pz → target model generates answer ŷ = M(x; pz)

Different task families take different prompt channels, fundamentally avoiding the conflict of one prompt learning math and coding simultaneously.

But routing design is a chicken-and-egg problem:

Routing decides what data each prompt sees → routing quality affects prompt optimization
Prompt quality determines whether routing decisions are correct → prompt capability affects routing evaluation

This motivates Router-Prompt Co-Evolution.

3. Co-Evolution: The Art of Alternating Optimization

Each co-evolution cycle alternates two operations:

3.1 Router Evolution

Fix the current prompt set, search for better routing rules
Sample training data where "at least one slot answers correctly" (ensuring failures are attributable to routing, not prompt capability)
Mutate + reflect on candidate routers using cases where "the routed slot failed but another succeeded"
Routing score = accuracy (60%) + compactness/separability (20%) + balance (20%)
Weights are annealed: early stages favor consistency/balance (preserving routing diversity), later stages favor accuracy (converging to stable partitioning)

3.2 Prompt Evolution

Fix the current router, optimize each slot's prompt independently in parallel
Each slot learns only on its routed data
Uses mutation + reflection for search
Key: a Pareto-front pool — each prompt is represented by its correctness vector on validation data; only non-dominated prompts are kept
Admission gate: s_val(p*) > s_val(p_∅) (must beat the empty prompt)

The essence of this alternation is not seeking per-step optimality. Heavy prompt optimization before routing stabilizes wastes budget; prompts too far behind cause the router to overfit stale behavior. Eevee switches frequently with lightweight budgets, letting the two co-evolve.

4. Three-Stage Training: From Mixed Stew to Refined Simmer

Stage 1: Initialization

Run prompt learning on the mixed training set, keep the Pareto-front pool
Select Top-K prompts via greedy coverage: repeatedly pick the prompt covering the most new validation samples
The resulting prompts are naturally complementary, giving routing learnable behavioral distinctions

Stage 2: Exploration

Alternately run Router Evolution and Prompt Evolution from (R₀, P₀)
Lightweight budgets, frequent switching — quickly explore the coupled space
The annealing mechanism encourages diverse partitions early, converging later

Stage 3: Convergence

Once a stable router R* is identified, fix it
Re-route all training/validation data to slots
Invest larger prompt-learning budgets per slot to find each partition's optimal prompt

5. Results: Overwhelming Gains That Grow With Task Mixture

5.1 Main experiment (four mixed benchmarks)

| Method | Qwen3-4B avg | DeepSeek-V3.2 avg | |--------|-------------|-------------------| | Baseline | 41.37 | 39.75 | | GEPA | 37.73 (-3.64) | 55.83 (+16.08) | | ACE | 34.92 (-6.45) | — | | Eevee | 51.75 (+10.38) | 64.07 (+24.32) |

5.2 Incremental multi-benchmark retention (core selling point)

GEPA: final cumulative retention -15.36
ACE: final cumulative retention -18.58
Eevee: final cumulative retention +41.53 (always positive, growing stronger)

This confirms the core hypothesis: the structural advantage of routed partitioning amplifies exponentially as task mixture increases.

5.3 Ablations: routing and co-evolution are both indispensable

| Variant | Avg score | Note | |---------|-----------|------| | Full Eevee | 51.75 | Full method | | Default Router | 43.58 | Only +2.21 over baseline | | Manual Router (GPT-5.4) | 37.18 | Worse than baseline by 4.19 | | No Co-evolution | 42.88 | Only +1.51 |

1. Learning the router is necessary — manual routing is harmful 2. Co-evolution is necessary — two-stage approaches barely help

5.4 Token cost

Eevee averages 4.32k tokens per example, close to GEPA's 3.47k and far below ACE's 21.30k. ACE's playbook keeps appending bullets, bloating as tasks accumulate; Eevee's router adds only a lightweight selection layer.

5.5 Generalization

Cross-model: prompts learned on Qwen3-4B improve DeepSeek-V3.2 from 39.75 to 54.10 (+14.35); HumanEval gains +34.22
Cross-task: +1.13 on MBPP; only -1.82 on MMLU-Pro (less than GEPA's -1.89)

6. Case Study: What Does Prompt Learning Actually Learn?

The authors retested 6 full runs, comparing empty-prompt vs. final-learned flips:

Positive cases (formula calculation, code generation)

Formula: baseline forgot unit scaling, producing wrong negative numbers; learned version strictly followed dollar computation and output correct values
HumanEval: baseline wrote bare expressions without return statements; learned version produced complete executable function bodies

→ Prompt learning excels at turning feedback into reusable procedures/rules.

Negative cases (knowledge-intensive QA)

GPQA Diamond: baseline correctly used a planetary physics prior; the learned version reinforced general reasoning but misapplied a physical assumption (same composition = same density), flipping to a wrong answer

→ Prompt learning can overreach on domain knowledge — it strengthens reasoning patterns rather than supplying missing knowledge.

Core insight: > "Prompt learning excels at reusable procedures, but can underweight domain knowledge."

This has deep implications for agent design: prompt learning suits teaching agents *how to solve problems* (procedures, formats, strategies), not *what to know* (domain knowledge).

7. Limitations and Future Work

1. Randomness: evolutionary search cannot guarantee consistent results across runs 2. Label dependence: still needs ground-truth or rule-based labels; not purely reflection-driven 3. Distribution shift risk: poor-quality or mismatched adaptation data can degrade learned prompts

8. Conclusion: Why Eevee Is "Real-World" Prompt Learning

Eevee's contribution in one sentence: moving prompt learning from single-task labs to the multi-task real world.

Its core innovation is an architectural redesign — acknowledging the heterogeneity of real-world task streams, using routed partitioning to structurally eliminate cross-task interference, and co-evolving router and prompts to adapt to each other.

Results are solid:

Matches SOTA on single tasks
Crushes SOTA on multi-task mixtures (retention +41.53 vs. -15/-18)
Controlled token cost (4.3k vs. ACE's 21.3k)
Reasonable cross-model and cross-task generalization

For developers building LLM agents, Eevee offers a key design principle: when an agent faces diverse task streams, rather than letting one prompt grow ever more bloated, let tasks find their own prompts.

References:

Xu, W., Liu, S., & Wang, M. (2026). Eevee: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents. *arXiv preprint arXiv:2606.11182*.
Agrawal, L. A., et al. (2026). GEPA: Reflective prompt evolution can outperform reinforcement learning. *ICLR 2026*.
Liu, S., et al. (2026). ACE: Evolving contextual encyclopedia for debugging language agents. *arXiv*.
Li, H., et al. (2026). Combee: Scaling prompt learning with parallel trace aggregation. *arXiv*.