arXiv:2608.18027v1, posted 2026-08-18, a UC Santa Cruz × ByteDance Seed collaboration: Haoqin Tu and Yunhao Fang as co-first authors, with Cihang Xie and Shen Yan as senior authors. The paper is titled Chain-of-Experience, abbreviated CoE.
The one-line takeaway: Don't rush to fine-tune the model — send it in with a mistake notebook.
Mechanism: Don't Throw Away the Scratch Paper
Traditional test-time scaling throws the scratch work away — Self-Consistency compresses many reasoning chains into one answer, losing all the detours. Self-Refine and Reflexion internalize feedback within a single context, which the authors dismiss as "fragmented and shallow."
CoE instead records a binary tuple at each iteration, \(e = (\text{answer}, \text{feedback})\), and keeps the entire history in context. The next generation sees the full chain:
Zero weight updates — pure in-context reuse. It is orthogonal to cross-task "cheat sheets" like Dynamic CheatSheet / ACE: CoE is intra-task with full trajectories; DC/ACE are cross-task with lossy compression. Don't misread "Continual" in the title as continual learning — the parameters never move.
Four Feedback Tiers; the Ceiling Is Feedback Quality
1. None: no feedback, self-reflection only. Six-benchmark average 66.8%. 2. Model feedback: an LLM judge gives textual critiques (can be the same model). 71.0%, +5.6% over no feedback. 3. Executor feedback: run code, read errors, check test pass rates. Coding: 66.4 → 75.0. 4. Correctness feedback: told directly whether the answer is right, requires ground truth. 79.3%, +11.1% — but this is an oracle; the paper itself admits it is "often unavailable in reality" and positions it as an upper bound.
Experimental scope: six benchmarks — AIME 2025, OmniMath, LiveCodeBench V6, LiveBench Code, EvaLearn, GPQA Diamond — with closed-source models including GPT-5, GPT-5-mini, o4-mini, o3, o3-mini, Gemini-2.5 Pro, and Claude 4.5 Sonnet. Default 20 iterations with best-of selection, 3 runs per experiment.
Hard findings:
- Gains come early: on AIME, the first 20 iterations average +16.7%; iterations 20–50 add only +2.2%. Don't over-spend the iteration budget.
- Self-feedback saves money too: on AIME, self-feedback costs $4.6 vs $8.8 for no feedback, while scoring 4.4 points higher. Token counts across the three multi-round methods are nearly identical (gap under 2K), so gains come from reallocating compute into feedback iterations, not longer prompts.
- Robust to fake feedback: constant "all correct" or "all wrong" signals degrade AIME by only 7.6% on average; stronger models resist better (GPT-5 mini drops just 2.5%). All-wrong feedback even dips then rebounds — the paper calls it "constructive noise can stimulate deeper reasoning."
- Attribution: among 6,630 "wrong → right" samples, 47.7% were feedback-driven; coding tasks showed another 30.0% from spec retro-alignment (re-reading the problem requirements). Human-model agreement κ=0.768.
- Base ability ↔ improvement ability are positively correlated: average Pearson +0.50 across five benchmarks, up to 0.97 on LiveBench Code. The paper calls learning-from-experience "an emergent property that scales with model capability" — take that half-sentence with a discount (see below).
- ✅ Safe to cite: test-time iteration with self-feedback gains +5.6%; gains concentrate in the first 20 iterations; feedback quality ∝ verifier capability.
- ⚠️ Cite with caveats: the +11.1% must be labeled an oracle upper bound; "outperforms DC/ACE" must note the 10x token difference.
- ❌ Don't cite: SelMV's "universal improvement" (AIME actually drops under normal feedback); "effective across all tasks and models" (two built-in counterexamples).
The Accounting Needs Scrutiny: A Critical Audit
The paper is more honest than most in this genre — the oracle is labeled an upper bound and negative results are reported. But four accounts deserve discounts:
1. The +11.1% hangs on a perfect grader. Nearly half the gap between self-feedback's 5.6% and the oracle's 11.1% depends on a grader that doesn't exist in reality. Correctness feedback pushes GPQA to 99.52% — with 20 iterations of best-of plus per-step correctness checking, this approximates enumerating the answer space with the key. That isn't "learning from experience"; it's an open-book exam.
2. "Beats DC/ACE" is a victory at 10x compute. On AIME, Dynamic CheatSheet uses 11,233 tokens to score 74.7%; no-feedback CoE uses 106,825 tokens to score 74.1%. At matched compute, DC doesn't lose. Also, DC used ground-truth answers during curation and still underperformed, hinting the baseline's failure may be an adaptation issue — no ablation rules this out.
3. The "emergent property" evidence is thin. Each correlation has n≈8 models, 5 of them from the OpenAI family; points like r=0.33 and 0.24 are highly unstable at n=8, and there are no p-values or confidence intervals. Math-task correlations are actually weak; the pretty 0.97 comes from a single benchmark. Confounders — release date, reasoning-training intensity — go undiscussed.
4. A built-in counterexample is buried in the appendix. On BrowseComp-Plus (deep retrieval), most models score lower with self-feedback than without — self-critique can't fill knowledge gaps. Gemini-2.5 Pro is the only model that doesn't save money. The main text's "across all tasks and models" wording contradicts its own data.
Three minor errata: the abstract claims 8 models but the text lists 7; attribution agreement appears as 76.8% in text vs 76.0% in Table 4 (likely conflating κ with agreement); ACE's average is 64.0% in text but 63.0% when recomputed from the table.
How to Cite It
Closing Nail
Two gaps are most worth watching. First, open-source models' experience-utilization ability — "emergence" should be tested on Qwen / DeepSeek, but this paper is all closed-source APIs. Second, weight internalization — the paper itself admits in its limitations that distilling the experience chain back into parameters would be true continual learning, where it would meet training frameworks like AgentEvolver head-on.
Verification point, recorded here: if a CoE + open-source-model replication appears within three months (especially at the 7B scale) and the +5.6% figure holds, then "chains of experience" stand; if not, it's just a footnote to the context dividend of frontier LLMs.