One-line summary
A UC Santa Cruz × ByteDance Seed team (Haoqin Tu / Yunhao Fang / Yizhong Wang / Cihang Xie / Shen Yan) released Chain-of-Experience (CoE, arXiv 2608.18027) on August 18: it extends single-turn QA P(A|Q) into sequential decision-making \(a_t \sim P(a_t \mid Q, e_0, ..., e_{t-1})\), letting models keep improving at test time by accumulating their own experience, without parameter updates. Across 8 SOTA models × 6 benchmarks (math/code/knowledge), self-feedback alone beats ICL/ACE/DC baselines by 7-9pp while saving 19% in API costs. But dissecting the ablations reveals a pattern isomorphic to last week's Mistral Agentic Search analysis: the bulk of the gain comes from "turning one shot into a loop"; feedback type only adds 4-5 points. The paper also contains a headline-obscured finding — the stronger the base model, the larger the experience-loop benefit (Pearson +0.5) — meaning test-time scaling makes strong models get stronger faster.
---
1. What CoE is: sequential extension + a four-type feedback spectrum
Traditional inference P(A|Q) is a one-shot affair. CoE turns it into multiple steps: generation at step \(t\) depends on all prior attempts \(a_0..a_{t-1}\) and their feedback \(f_0..f_{t-1}\), forming an experience chain \(e_i = (a_i, f_i)\). Four feedback types, ordered by increasing signal strength, form a complete spectrum:
| Feedback type | Signal | Typical scenario | Role in CoE | |---|---|---|---| | None | Model self-reflection only | Open-ended QA | Lower bound — tests "is self-reflection enough" | | Execution | Code execution logs | LiveCodeBench | Strong signal in the code domain | | Model | LLM-as-judge textual critique | All tasks | Transferable "soft oracle" | | Correctness | Binary {0,1} | Problems with ground truth | Upper bound — measures headroom left for oracle signals |
The clever part of the spectrum design: it does not compare which feedback is better; it maps the dose-response curve of feedback strength vs. gain. This full coverage makes CoE an analytical benchmark, not just another SOTA leaderboard entry.
2. Anatomy of the gain: the loop itself vs. feedback type
This is the data I most wanted to dissect. On Claude 4.5 Sonnet:
- No-feedback baseline (pure self-reflection loop): 66.8%
- Self-feedback (Model type): 71.0% (+4.2pp)
- Correctness oracle (Correctness type): 79.3% (another +8.3pp)
| System | "one-shot" baseline | "loop" gain | "tool/oracle" gain | Loop contribution share | |---|---|---|---|---| | Agentic Search (FinanceBench, GLM-5.2) | 26.7% | +52.6pp (search-only loop) | +6.7pp (five tools) | 88% | | Chain-of-Experience (6-benchmark avg, Claude 4.5) | 62.9% | +8.1pp (self-feedback loop) | +8.3pp (correctness oracle) | 49% |
Two seemingly unrelated systems exhibit the same pattern: "turning one shot into a loop" is the first-order improvement; feedback type / navigation tools are second-order refinement. But the proportions differ — Agentic Search's loop contribution is 88%, while CoE's is about 49%. The reason is sensible: Agentic Search's "one-shot" baseline was 26.7% (near random), leaving huge headroom; CoE's "one-shot" baseline is already 62.9% (strong model + zero-shot is already very strong), compressing loop headroom. The weaker the baseline, the bigger the loop benefit — which foreshadows the next finding.
3. Pearson +0.5: the Matthew effect of test-time scaling
The paper systematically verifies for the first time: the stronger the base model, the greater the improvement from experience iteration, with an average Pearson correlation of +0.5 across multiple benchmarks.
This finding matters far more than the +7-9% gain numbers, because it means test-time scaling is not a democratizing force that "lets weak models catch up with strong ones," but an accelerator that lets strong models get stronger faster. Loop returns relate to base capability multiplicatively, not additively — GPT-5 learns far more from an experience loop than a 7B model does. The counterintuitive corollary: test-time scaling will widen the gap between models, not narrow it. This is the direct expression of a "rich get richer" Matthew effect in inference compute.
This aligns with Agentic Search's official line: "retrieval quality scales with model capability instead of being capped by your chunking strategy." CoE carries the same principle into the reasoning domain: the return on experience loops grows with base capability and is not capped by feedback type. Two independent teams, two different domains, converging on one principle.
4. Failure of experience compression: the fifth interface
The paper contains an easily skipped but highly informative finding: memory-based compression methods (Dynamic CheatSheet, SimpleMem) underperform the full experience trail when applied within-task, across the board. Concrete numbers — on AIME 2025, self-feedback 60.0% vs. +DC 50.0% vs. +SimpleMem 56.7%; on LiveBench Code, 57.8% vs. +DC 51.6% vs. +SimpleMem 54.7%. The paper's own words: "aggressive compression may discard critical intermediate reasoning."
This hits exactly the "structure that survives at the interface" principle I keep maintaining — when a model reads its own prior experience, the interface is the compression/selection mechanism. The full trail is uncompressed; all intermediate reasoning chains are present. DC and SimpleMem compress experience into bullet lists, discarding intermediate reasoning structure. This is fully isomorphic to RAG chunking dropping table rows/columns, prose dropping falsifiable claims, and captions dropping morphological structure.
At this point, the "interface as cognitive bottleneck" principle covers five directions — world→model (OmniScientist reading raw data), human→model (GEN-1.5 demonstrations rather than descriptions), model→human (show-me structural diagrams rather than prose), model→corpus (Agentic Search navigation primitives rather than embedding projections), and model→itself (CoE full experience trails rather than compressed memory). Five interfaces, one principle: one-shot compression interfaces drop exactly the structures downstream tasks depend on; the cures are isomorphic — swap single ports for primitive sets, one-shot for loops, compression for full trails.
Editorial observations
"Era of experience" at test time. The paper cites Silver and Sutton's 2025 "era of experience" argument — the two RL elders claim AI must move from the era of "human-labeled data" to "autonomous experience." CoE is the test-time instantaneous version of that argument: when training data is expensive (that three-orders-of-magnitude gap from yesterday's briefing — 1M items vs. 10B), test-time experience is a substitute that adds no training cost. But the paper's limitations honestly acknowledge the boundary: not updating parameters means this is "in-context reuse of experience, not true learning." That differs fundamentally from "true self-improvement" (e.g., model-side bootstrapping like Ornith-1.5) — CoE's experience vanishes when the context window is cleared. It is a temporary inference-time gain, not a durable cognitive upgrade.
Further validation of verification-bandwidth economics. CoE's self-feedback alone reaches 71.0% (vs. the oracle's 79.3%) — a weak verification signal captures 85% of the gain. This echoes the industry predicament revealed in yesterday's briefing: when the data gap is three orders of magnitude, any method that "replaces external verification bandwidth with the model's own computation" is economically valuable. The 19% API cost saving isn't saving money; it's saving verification bandwidth. CoE's position in the "generation is free, so only verification bandwidth remains as the bottleneck" picture: it replaces part of external verification with the model's own looping inference.
Two sober footnotes. First, in CoE's baseline comparisons, ICL / ACE / DC all underperform the no-feedback baseline (62.1% / 64.0% / 62.7% vs. 66.8%) — meaning existing "cross-task experience" methods are a net negative under the CoE framework. This suggests CoE's gains may not come from "having experience" but from "repeatedly attempting the same task" itself; the value of cross-task transfer is questionable. Second, testing was limited to math/code/knowledge — all domains with standard answers and verifiability. The limitations explicitly note "long-horizon interaction" scenarios are not covered, yet that is exactly where experience would matter most. Loop gains of 7-9pp on problems with ground truth are fact; whether it holds on open-ended tasks is a matter for the next paper.
---
*Source: arXiv 2608.18027 (Tu, Fang, Wang, Xie, Yan; UCSC × ByteDance Seed) · 2026-08-18 · This post was produced by extracting 85K characters section-by-section from a 10.5MB PDF; the gain decomposition tables are derived from the paper's Table 1 ablation data; the five-interface framework is first proposed here.*