Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost
> 📌 This is a GEO-optimized version of the original topic.
If a language model answers a math problem incorrectly, you have two options:
A. Make it reflect — "You got that wrong. Think about why and try again." B. Make it sample again — same problem, new random seed, take the majority answer.
Intuitively A seems smarter. But a July 2026 paper shows that at equal token budgets, B almost always beats A, often by a wide margin.
Paper: *Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B* (arXiv:2607.28576)
Method Dividend vs Budget Dividend
Nearly all reflection methods cause the model to generate more tokens: Self-Refine runs three passes (answer → critique → rewrite); Reflexion adds reflection and retry rounds; Best-of-N multiplies output N times.
But generating more tokens alone improves accuracy — Wang et al. (2024) showed majority voting over repeated samples beats a single sample. So if a reflection method beats single-pass answering, you cannot tell whether "reflection" or merely "more tokens" did the work.
The paper does something simple but previously undone: equalizes token budgets and compares methods head-to-head.
36 Comparisons, Zero Wins
Experimental setup:
- 7 methods: single-pass CoT, repeated sampling (majority vote), Self-Refine, Reflexion, Best-of-N (model-selected), Best-of-N (count-selected), debate
- 3 model scales: 1.5B, 3B, 7B
- 2 math benchmarks: GSM8K and MATH
- 150 problems, paired comparisons
- Rigorous statistics: bootstrap confidence intervals with multiple-comparison correction
- Repeated sampling + majority voting is almost always the better choice
- Model self-inspection may be a method-dividend illusion — the gains come from the mundane fact of generating more tokens
- The cost of "letting the model choose" declines with scale but never beats simple counting
- Method dividend vs budget dividend
- 36 comparisons, zero wins for self-inspection
- The cost of model-based answer selection
Result: no self-inspection method reliably beats repeated sampling at equal token budget. Of 36 comparisons, 0 were significantly positive, 10 significantly negative (all self-inspection methods), 26 indistinguishable from zero. All 18 self-inspection comparisons were negative: Self-Refine and forced Reflexion trail repeated sampling by 3.6–10.1 percentage points on 7B models.
The Cost of "Choosing"
Best-of-N has two selection modes: count selection (pick the majority answer among 8) vs model selection (let the model pick the best of 8). Intuitively the model should choose better — yet on 1.5B, count selection beats model selection by 8.0 and 11.3 percentage points. At 7B the gap shrinks to 2.0 and 1.3 points, no longer significant.
Small models are poor at judging their own answers; larger models improve slightly but still don't beat simple counting. Judging, comparing, and selecting all cost tokens that would yield more if spent on additional samples.
Reflexion's Embarrassment
On 1.5B models, Reflexion never triggered its own retry mechanism — it always judged itself correct, silently degrading to single-pass CoT. The "Reflexion effect" reported in the original paper, at least at small scale, wasn't reflection at all — just a single answer.
What This Means
If you care about per-token efficiency (as you should in production):
This mirrors a broader pattern in AI research: after equalizing conditions against simple baselines, complex methods often show no significant advantage — like DPO matching RLHF, or CoT's gains tracing back to longer generation.
A Deeper Insight
When a method claims to beat a baseline, first ask how many extra tokens it spent. If it burns 3× tokens for a 5% gain while a simple baseline gains 6% at the same cost, the former shouldn't be published. "Reflection" sounds sexy and feeds an "AI getting smarter" narrative; "sample more and vote" sounds too plain. This paper's value is executing a neglected controlled comparison rigorously. The conclusion is unsentimental — as science should be.
---
Paper: https://arxiv.org/abs/2607.28576
FAQ
Q1: Who is this for? Practitioners, researchers, and students in AI, machine learning, and deep learning.
Q2: Key takeaways?