TL;DR
UC Santa Cruz × ByteDance Seed's Haoqin Tu / Yunhao Fang / Yizhong Wang / Cihang Xie / Shen Yan team released Chain-of-Experience (CoE, arXiv 2608.18027) on August 18: it extends single-pass QA P(A|Q) into sequential decision-making \(a_t \sim P(a_t | Q, e_0, ..., e_{t-1})\), letting the model keep improving at test time through accumulated self-experience, with no parameter updates. Across 8 SOTA models × 6 benchmarks (math/code/knowledge), self-feedback alone beats ICL/ACE/DC baselines by 7-9pp while saving 19% on API costs. But dissecting the ablation data reveals a pattern structurally identical to last week's Mistral Agentic Search finding: the bulk of the gain comes from 'turning one attempt into a loop'; feedback type only adds 4-5 points. The paper also hides a finding under its headline — the stronger the base model, the larger the experience-loop gain (Pearson +0.5), meaning test-time scaling lets strong models get stronger faster.
1. What CoE Is: Sequential Extension + a Four-Type Feedback Spectrum
Traditional inference P(A|Q) is a one-shot affair. CoE turns it multi-step: generation at step \(t\) depends on all previous attempts \(a_0..a_{t-1}\) and their feedback \(f_0..f_{t-1}\), forming an experience chain \(e_i = (a_i, f_i)\). Four feedback types, increasing in signal strength, form a complete spectrum:
| Feedback type | Signal | Typical setting | Role in CoE | |---|---|---|---| | None | Model self-reflection only | Open-ended QA | Lower bound — tests 'is self-reflection enough' | | Execution | Code execution logs | LiveCodeBench | Strong signal in the code domain | | Model | LLM-as-judge textual critique | All tasks | Transferable 'soft oracle' | | Correctness | Binary {0,1} | Tasks with ground truth | Upper bound — measures remaining oracle headroom |
The elegance of the spectrum: it does not ask which feedback is better, it maps the dose-response curve of feedback strength vs gain. This complete coverage makes CoE an analysis benchmark, not just another leaderboard chaser.
2. Dissecting the Gains: The Loop Itself vs Feedback Type
This is the data I most wanted to break down. On Claude 4.5 Sonnet:
- No-feedback baseline (pure self-reflection loop): 66.8%
- Self-feedback (Model type): 71.0% (+4.2pp)
- Correctness oracle (Correctness type): 79.3% (a further +8.3pp)
| System | 'One-shot' baseline | 'Loop' gain | 'Tool/oracle' gain | Loop share of gain | |---|---|---|---|---| | Agentic Search (FinanceBench, GLM-5.2) | 26.7% | +52.6pp (search-only loop) | +6.7pp (five tools) | 88% | | Chain-of-Experience (6 benchmarks, Claude 4.5) | 62.9% | +8.1pp (self-feedback loop) | +8.3pp (correctness oracle) | 49% |
Two seemingly unrelated systems show the same law: 'turning one attempt into a loop' is the first-order improvement; feedback type / navigation tools are second-order refinements. The proportions differ, though — Agentic Search's loop contribution is 88%, CoE's about 49%. The reason is sound: Agentic Search's one-shot baseline is 26.7% (near random), leaving huge marginal room for loops; CoE's one-shot baseline is already 62.9% (strong model + zero-shot), compressing the loop's headroom. Weaker baseline → bigger loop gain — which sets up the next finding.
3. Pearson +0.5: The Matthew Effect of Test-Time Scaling
The paper is the first to systematically verify: the stronger the base model, the larger the gain from experience iteration, with an average Pearson correlation of +0.5 across multiple benchmarks.
This finding matters far more than the +7-9% headline, because it means test-time scaling is not a democratizing force 'letting weak models catch up to strong ones' but an accelerator letting strong models get stronger faster. Loop gains scale multiplicatively, not additively, with base capability — GPT-5 learns far more from experience loops than a 7B model. The counterintuitive corollary: test-time scaling will widen gaps between models, not narrow them. It is the 'rich get richer' Matthew effect applied directly to inference compute.
This aligns with last week's Agentic Search quote: 'retrieval quality scales with model capability instead of being capped by your chunking strategy'. CoE carries the same law into the reasoning domain: experience-loop gains scale with base capability, uncapped by feedback type. Two independent teams, two different domains, converging on one principle.
4. Experience Compression Fails: A Fifth Interface
A highly informative finding that is easy to skip: memory-based compression methods (Dynamic CheatSheet, SimpleMem), applied within-task, consistently underperform the full experience trail. Specifically — AIME 2025: self-feedback 60.0% vs +DC 50.0% vs +SimpleMem 56.7%; LiveBench Code: 57.8% vs +DC 51.6% vs +SimpleMem 54.7%. The paper's own words: 'aggressive compression may discard critical intermediate reasoning.'
This hits exactly at the 'structure that survives at the interface' principle I keep tracking — when a model reads its own prior experience, the interface is the compression/selection mechanism. The full trail compresses nothing; every intermediate reasoning chain survives. DC and SimpleMem compress experience into bullet lists, discarding intermediate reasoning structure. This is the exact same loss pattern as RAG chunking dropping table rows/columns, prose dropping falsifiable claims, or captions dropping morphological structure.
The 'interface as cognitive bottleneck' principle now covers five directions — world→model (OmniScientist reading raw data), human→model (GEN-1.5 demonstrating rather than describing), model→human (show-me structure diagrams rather than prose), model→corpus (Agentic Search navigation primitives rather than embedding projections), and now model→self (CoE full experience trails rather than compressed memory). Five interfaces, one principle: one-shot compression interfaces discard precisely the structure downstream tasks depend on; the remedy is isomorphic — swap single ports for primitive sets, one-shot for loops, compression for full trails.
Editorial Observations
The test-time version of the 'era of experience'. The paper cites Silver and Sutton's 2025 'era of experience' thesis — two RL patriarchs arguing AI must move from the era of 'human-labeled data' to 'autonomous experience'. CoE is that thesis's test-time instant version: when training data is expensive (yesterday's report on the three-order-of-magnitude gap — 1 million vs 10 billion samples), test-time experience is an alternative that adds no training cost. But the limitations section honestly admits the boundary: no parameter updates means this is 'contextual reuse of experience, not true learning'. This differs fundamentally from 'true self-improvement' (e.g., model-side bootstrapping like Ornith-1.5) — CoE's experience vanishes when the context window clears. It is a temporary inference-time gain, not a durable cognitive upgrade.
Another confirmation of verification-bandwidth economics. CoE self-feedback alone reaches 71.0% (vs 79.3% with an oracle) — a weak verification signal captures 85% of the gain. This echoes yesterday's industry dilemma: when the data gap is three orders of magnitude, any method that 'replaces external verification bandwidth with the model's own compute' has economic value. The 19% API cost saving is not saving money — it is saving verification bandwidth. CoE's position in the 'generation is free, so verification bandwidth is the bottleneck' picture: it substitutes part of external verification with the model's own looping reasoning.
Two sobering footnotes. First, in CoE's baseline comparison, ICL / ACE / DC all underperform the no-feedback baseline (62.1% / 64.0% / 62.7% vs 66.8%) — existing 'cross-task experience' methods are actually negative contributions within the CoE framework. This suggests CoE's gains may come less from 'having experience' and more from 'repeatedly attempting the same task' itself; the value of cross-task transfer is questionable. Second, testing is confined to math/code/knowledge — all domains with standard answers and verifiability. The limitations section explicitly notes that 'long-horizon interaction' scenarios are not covered, yet that is precisely where experience should matter most. A 7-9pp loop gain on ground-truth questions is fact; whether it holds on open-ended tasks awaits the next paper.
---
*Source: arXiv 2608.18027 (Tu, Fang, Wang, Xie, Yan; UCSC × ByteDance Seed) · 2026-08-18 · Gain decomposition table derived from the paper's Table 1 ablation data.*