Ever wondered why a large language model that got a problem wrong will make the same mistake again on a similar problem next time?
Humans don't work that way. When you get stuck on a math problem, glance at the solution—"oh, so that's where substitution comes in"—and you won't fall into the same trap twice. This ability to "learn from experience" is one of the most fundamental features of intelligence. But today's large models—whether GPT-5, Gemini-2.5 Pro, or Claude-4.5 Sonnet—are "one-shot" in standard evaluations: solve a problem, right or wrong, then forget it and start from zero on the next one.
A team from ByteDance Seed and UC Santa Cruz has proposed a framework called Chain-of-Experience (CoE). What it does is simple: after the model completes a problem, it accumulates feedback signals into a "chain of experience" and keeps improving on subsequent problems.
Core Idea: Turning Test Time Into Learning Time
CoE works like this: instead of solving each problem independently, the model faces a set of problems (math, coding, knowledge QA). After each problem, it receives feedback—either self feedback (the model judges its own answer) or correctness feedback (a signal from the environment)—and accumulates these experiences into its context, influencing solutions to later problems.
This sounds like in-context learning (ICL), but there's a key difference: ICL is passive—you give it examples and it learns whatever you provide. CoE is active—the model generates its own experience during problem solving, and that experience directly changes its subsequent behavior.
The paper explores a "feedback spectrum":
- Self feedback: the model evaluates its own answer after solving, using the judgment as experience
- Correctness feedback: the environment tells the model whether it was right (e.g., test case pass rate on coding problems)
- Dual-channel combination: using both self and correctness feedback
Coding Tasks Show the Brightest Results
On coding tasks (LiveCodeBench V6 and LiveBench Code), CoE was most effective. With programmatic executor feedback (i.e., whether test cases pass), accuracy rose from 66.4% to 75.0%—a 8.6 percentage point gain. Even with only self-judgment feedback, the improvement was 7.0%.
This makes sense: coding problems have a clear right/wrong signal—code that runs is right, code that doesn't is wrong. This "hard feedback" carries more information than "feeling good about yourself."
But on math and knowledge QA (AIME 2025, OmniMath, GPQA Diamond), correctness feedback still sets an upper bound; self feedback can't reach that bound, though it still delivers substantial improvement.
Three Key Findings
Finding 1: Stronger models learn faster. A model's base capability correlates with its ability to improve—the smarter the model, the more it learns from experience. Consistent with intuition: you need a certain foundation to make effective use of feedback.
Finding 2: Models resist garbage feedback. When researchers deliberately fed the model all-wrong feedback (e.g., always saying "your answer is correct" or always "incorrect"), performance dropped only modestly—AIME 2025 dropped 7.6% on average, GPQA Diamond 2.6%. Stronger models (like GPT-5 mini) recovered better. This means models don't simply "believe" feedback—they have some ability to distinguish useful from useless signals.
Finding 3: Most gains come from the first few iterations. The marginal returns of experience accumulation diminish quickly—the earliest rounds of interaction contribute most of the improvement, then it plateaus. This matters practically: you don't need unlimited rounds of experience accumulation; a few rounds suffice.
Why This Matters
There's a fundamental contradiction in the current LLM landscape: training costs are astronomically high, but once a model is trained it's "frozen"—it doesn't learn from usage. RLHF and DPO try to inject human preferences during training, but after training the model returns to its "one-shot" state.
CoE represents another path: without changing model parameters, achieve continuous improvement at inference time through experience accumulation. This is closer to how humans learn—you don't need to "reset to factory settings" every time you learn something new.
From an engineering standpoint, the 19% API cost reduction is also practical. If you're a company running large-scale evaluations, this means the same budget covers 23% more evaluation volume. And CoE's "accuracy per token" is higher than existing test-time strategies—meaning it's not just buying higher accuracy with more tokens, it genuinely uses compute more efficiently.
Honest Limitations
CoE isn't a panacea. First, it depends on feedback signal quality—on tasks without a clear correct answer (e.g., open-ended writing), the reliability of self feedback is questionable. Second, experience accumulation consumes context window space; long tasks may hit token limits. Third, the paper only tested experience transfer within the same task category; cross-domain transfer (e.g., from math to coding) wasn't covered.
Also, the 5.6% average improvement, while significant, is small on some tasks. This isn't a "disruptive" result—it's a "solidly incremental" one.
My Take
CoE makes me think about a bigger question: our way of evaluating LLMs may be fundamentally wrong. Today's mainstream benchmarks—MMLU, HumanEval, MATH—are all "one-shot" tests, scoring each problem independently. But if we truly care about a model's "intelligence," we should measure its ability to "learn from experience," not its zero-shot performance.
This aligns with a recent wave of research: from Chain-of-Thought to Self-Refine to Test-Time Training, the field is shifting from "training-time optimization" to "inference-time optimization." CoE pushes this direction to "inference-time continuous learning"—not just within-problem reasoning optimization, but cross-problem experience accumulation.
If future LLM "intelligence" manifests mainly at inference time rather than training time, then the astronomical compute we spend on training today may need to be partially reallocated to inference-time computation. That's not the conclusion of the CoE paper alone, but CoE provides a solid data point for this direction.
---
Paper: https://arxiv.org/abs/2608.18027
HTML full text: https://arxiv.org/html/2608.18027v1