Overview
Chain-of-Experience (CoE) is a framework proposed by ByteDance Seed and UC Santa Cruz that turns inference into a learning loop. Rather than answering each question independently and forgetting the outcome, a model accumulates feedback signals across a sequence of problems and uses that experience to influence later responses.
Core Idea: Test-Time as Learning-Time
The setup is straightforward. A model is given a set of questions (math, code, knowledge QA). After each answer, it receives a feedback signal—either self-generated self-feedback, or environmental correctness feedback (e.g., test-case pass rate for code). That signal is stored in context and conditions subsequent answers.
This differs from standard in-context learning: ICL is passive and consumes whatever examples are provided; CoE is active, with the model generating its own experience during inference and letting that experience directly change behavior.
The paper explores a feedback spectrum:
- Self-feedback: the model judges its own answers.
- Correctness feedback: the environment reports pass/fail.
- Dual-channel: both combined.
- Feedback quality matters: open-ended tasks (e.g., creative writing) lack reliable ground-truth signals, making self-feedback less trustworthy.
- Accumulated experience consumes context window space; long task chains may hit token limits.
- Cross-domain transfer (e.g., math → code) is not evaluated; only within-task transfer is tested.
- The 5.6% average gain is meaningful but not transformative—more of a solid step than a leap.
- Paper: https://arxiv.org/abs/2608.18027
- HTML: https://arxiv.org/html/2608.18027v1
- CoE turns inference into a learning loop by accumulating feedback across questions
- 5.6% average accuracy gain and 19% API cost reduction across 8 models and 6 benchmarks
- Coding benchmarks see the largest gains: 66.4% → 75.0% on LiveCodeBench V6 with execution feedback
- Stronger models learn faster from accumulated experience
- Models partially resist deliberately wrong feedback, especially stronger ones
- Most performance gains come from the first few interaction rounds
- Limitations include feedback-quality dependence, context-window pressure, and untested cross-domain transfer
Across 8 models and 6 benchmarks, CoE delivers an average 5.6% accuracy gain and 19% API cost reduction.
Coding Benchmarks Show the Strongest Gains
On LiveCodeBench V6 and LiveBench Code, CoE's largest effects appear. With programmatic execution feedback, accuracy rises from 66.4% to 75.0%, an 8.6-point gain. Self-judgment alone contributes 7.0%.
This is intuitive: programming tasks provide unambiguous pass/fail signals, which are more informative than self-assessment. On math and knowledge QA (AIME 2025, OmniMath, GPQA Diamond), correctness feedback still defines an upper bound, while self-feedback yields meaningful but smaller improvements.
Three Key Findings
1. Stronger models learn faster. A model's base capability correlates with how much it improves from accumulated experience—you need a certain baseline to exploit feedback effectively. 2. Models resist junk feedback. When the researchers deliberately injected always-wrong or always-right feedback, degradation was limited (AIME 2025: −7.6%, GPQA Diamond: −2.6% on average). Stronger models like GPT-5 mini recovered better, suggesting the model does not blindly follow feedback but filters signal from noise. 3. Most gains come from the first few rounds. Returns diminish quickly after early iterations. A handful of interaction rounds suffices for practical use.
Why This Matters
Today's LLMs face a basic tension: training costs are enormous, yet models are frozen at deployment and cannot learn from use. RLHF and DPO inject preferences during training but the model reverts to one-shot behavior afterward.
CoE takes a different path: leave parameters alone and achieve continual improvement at inference through experience accumulation—closer to how humans learn without resetting between problems.
The 19% API cost reduction is also practical. For teams running large-scale evaluations, the same budget covers roughly 23% more queries. CoE's accuracy-per-token exceeds existing test-time strategies, meaning gains come from more efficient computation, not simply spending more tokens.
Honest Limitations
Broader Implications
CoE fits a wider research trend: moving from train-time optimization to inference-time optimization—from Chain-of-Thought to Self-Refine to Test-Time Training. CoE advances this trajectory to test-time continual learning, where experience accumulates across questions rather than within a single chain of thought.
If future LLM "intelligence" lives mostly at inference time rather than training time, a portion of today's training compute budgets may need to shift toward inference compute. That is not a conclusion the paper claims outright, but it is a direction the data points toward.