Eevee Deep Dive: How Routed Prompt Partitions Avoid "Learn New, Forget Old" in Real-World LLM Agents

> Paper: *Eevee: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents* > Authors: Weixian Xu (SJTU), Shilong Liu (Princeton), Mengdi Wang (Princeton) > Paper: https://arxiv.org/abs/2606.11182 > Code: https://github.com/Princeton-AI2-Lab/EEVEE

The Problem: Single-Prompt Learning Breaks on Mixed Tasks

Test-time prompt learning lets agents improve their prompts from feedback after deployment, without fine-tuning. Methods like GEPA, ACE, and Combee have proven the approach works — but they are designed for single datasets.

Real agents face heterogeneous task streams: a coding assistant may simultaneously receive math derivations, code completion, knowledge QA, and formula computation. Forcing one prompt to absorb all this heterogeneous feedback causes cross-dataset interference — improving coding hurts math, and vice versa.

In the paper's Figure 1, as GPQA Diamond, Formula, TheoremQA, and HumanEval are added one by one to the learning stream, GEPA and ACE's cumulative retention falls to -15.36 and -18.58 respectively — after learning all four tasks, they perform *worse* on earlier tasks than before training.

Core Idea: Routing Splits the Mix into Specialized Slots

Eevee maintains a set of specialized prompts plus a routing module:

1. Input → Router selects a prompt slot: z = R(x; P) 2. The slot's specialized prompt generates the answer: ŷ = M(x; p_z)

The catch is a chicken-and-egg problem: routing quality affects prompt optimization, while prompt quality affects routing decisions. This motivates Router-Prompt Co-Evolution.

Co-Evolution: Alternating Optimization

Each co-evolution cycle alternates:

Router Evolution (prompts fixed)

• Samples training data where at least one slot answers correctly (so errors are attributable to routing, not prompt capability)
• Applies mutation + reflection on cases where the routed slot failed but another succeeded
• Router score = accuracy (60%) + compactness/separability (20%) + balance (20%), with annealed weights: early phases favor diversity, later phases favor accuracy

Prompt Evolution (routing fixed)

• Each slot's prompt is optimized in parallel, only on its own routed data
• Uses a Pareto-front pool: prompts are represented by correctness vectors on validation data; only non-dominated prompts are kept, and candidates must beat the empty prompt

The design deliberately avoids optimizing either side to the max each step — lightweight budgets and frequent switching let both co-evolve.

Three-Stage Training

1. Initialization: Run prompt learning on the mixed training set, keep the Pareto-front pool, and use Greedy Coverage to pick Top-K complementary prompts covering all validation samples 2. Exploration: Alternate router and prompt evolution with light budgets and frequent switching; annealed scoring drives early diversity, later convergence 3. Convergence: Once routing stabilizes (R*), fix it, re-route all data, and invest larger prompt budgets per slot

Results: Dominant Gains, Especially as Task Mix Grows

Main results (four benchmarks mixed)

| Method | Qwen3-4B avg | DeepSeek-V3.2 avg | |---|---|---| | Baseline | 41.37 | 39.75 | | GEPA | 37.73 (-3.64) | 55.83 (+16.08) | | ACE | 34.92 (-6.45) | — | | Eevee | 51.75 (+10.38) | 64.07 (+24.32) |

Eevee beats GEPA and ACE by 37.2% and 48.2% respectively.

Retention (the core claim)

• GEPA final cumulative retention: -15.36
• ACE: -18.58
• Eevee: +41.53 — always positive and growing

This confirms the structural advantage: the more heterogeneous the task mix, the more a routed partition beats a single shared prompt.

Ablations

| Variant | Avg score | Note | |---|---|---| | Full Eevee | 51.75 | — | | Default Router | 43.58 | only +2.21 over baseline | | Manual Router (GPT-5.4) | 37.18 | worse than baseline | | No Co-evolution | 42.88 | only +1.51 |

Learned routing and co-evolution are both necessary; a hand-written router actively hurts.

Token Cost and Generalization

• Eevee averages 4.32k tokens/example, close to GEPA's 3.47k and far below ACE's 21.30k (whose playbook-style context keeps growing)
• Cross-model: prompts learned on Qwen3-4B lift DeepSeek-V3.2 from 39.75 to 54.10 (+14.35); HumanEval +34.22
• Cross-task: +1.13 on MBPP; only -1.82 on unrelated MMLU-Pro

Case Study: What Does Prompt Learning Actually Learn?

Comparing empty prompts vs. learned results across 6 full runs:

• Positive: On Formula, the baseline mishandled unit scales; the learned prompt enforced dollar-based computation and output correct values. On HumanEval, the baseline emitted bare expressions without return; the learned prompt generated complete, executable function bodies. → Prompt learning excels at converting feedback into reusable procedures and rules.
• Negative: On GPQA Diamond, the baseline's physics prior picked the right answer; the learned prompt strengthened generic reasoning but applied a wrong physical assumption and chose wrong. → Prompt learning can underweight domain knowledge.

Core insight: prompt learning teaches agents *how to solve* (procedures, formats, strategies), not *what to know* (domain knowledge).

Limitations

1. Evolutionary search introduces run-to-run randomness in the learned routing and prompt sets 2. Still requires ground-truth or rule-based labels, not purely reflection-driven 3. Distribution shift risk: poor or mismatched adaptation data can degrade learned prompts

Conclusion

Eevee's contribution is architectural rather than algorithmic: it acknowledges the heterogeneity of real-world task streams, uses routed partitions to structurally eliminate cross-task interference, and co-evolves router and prompts. For agent developers, the design principle is clear: when your agent faces diverse task streams, let tasks find the right prompt rather than bloating a single one.

References:

• Xu, W., Liu, S., & Wang, M. (2026). Eevee: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents. *arXiv preprint arXiv:2606.11182*.
• Agrawal, L. A., et al. (2026). GEPA: Reflective prompt evolution can outperform reinforcement learning. *ICLR 2026*.