A model with only seven million parameters achieves 97.9% fully-correct solutions on extremely hard Sudokus. On the same test, the 671-billion-parameter DeepSeek-R1 scores 0. So do o3-mini-high and Claude 3.7 Sonnet: 0.
This is not a new scaling myth. The paper (arXiv 2610.11570, October 8, 2026) does something far smaller — and far more fundamental: it identifies a class of "the-more-you-think-the-worse-it-gets" model pathology, and prescribes a cure that costs only about five hundred parameters.
The Disease: Thinking Itself Makes Things Worse
For puzzles like Sudoku, a natural approach is to let the model reason repeatedly: a two-layer Transformer network runs with the same weights over and over, revising the board each round on top of the previous round. Ten rounds means twenty layers of depth; ten thousand rounds means twenty thousand layers. In theory you can think as deep as you like — trading compute for depth, depth for reasoning.
This path rests on a default assumption: after each round, pass the final computed state to the next round. The paper names this convention carry-last — you only carry the last one.
Here is where it breaks. Suppose round five has already correctly deduced the top-right corner of the board. In round six, the model generates a new, possibly wrong intermediate idea on some still-empty cell, and this new state entirely replaces the old one. The correct parts, never being recorded, get overwritten. Round seven receives a board of mixed quality: it must not only keep solving, but first recover the correct answers that round six destroyed. If it can't, it breaks puzzles it had already solved.
This is not conjecture — it was measured. Using TRM (another 7M recursive reasoning model) as the patient: on Sudoku-Extreme, TRM's accuracy peaks at an execution depth of 468 layers (62.2%) and then declines (58.1% at 1,872 layers). Even more striking is its behavior: 36 to 72 rounds *after* solving a puzzle, TRM is still modifying an average of 77 board cells per puzzle, and after all that editing, the number of wrong cells increases — a net gain of 0.21 wrong cells per puzzle. It is not refining; it is demolishing its own work.
The Remedy: Don't Just Carry the Last Word — Carry a Discounted Ledger
The fix sounds plain: don't let each round's output directly overwrite history; keep a ledger of the past first. Deep learning already has a prototype of this idea — the residual connection. In ordinary networks, each layer doesn't replace the previous layer's output; it adds its own increment on top, so old information persists all the way through. Residual connections are the foundation that has allowed deep networks to grow deeper for the past decade.
But recurrent models had no equivalent. Naively porting "inter-layer residuals" over — letting round t see the outputs of all previous rounds — means stored history grows linearly with rounds and compute grows quadratically. Nobody can afford twenty thousand rounds that way.
The paper's approach has three steps, each cheap.
Step one: scoring. Each round, when a new state is produced, a 512-dimensional "inspection vector" compares against it and assigns a score: what is this proposal worth? The inspection vector is learned, shared across all rounds, and scores each new state exactly once — score it, record it, never recompute.
Step two: discounting. Older records are worth less: each record's weight decays exponentially over time, and the decay rate itself is learned. Fresh reasoning is most credible; stale drafts automatically step aside.
Step three: streaming bookkeeping. Because "score once + fixed-rate decay" forms a clean mathematical structure, the ledger doesn't need to store the full history — only a running weighted sum and a normalization coefficient, updated once per new record. No matter how long the history, what gets stored stays fixed: one d-dimensional vector plus one scalar per token position. The paper's comparison: naive approaches have peak memory growing linearly with rounds (O(Td)); this compresses it to O(d), and total aggregation compute from quadratic to linear.
The three steps together are called InfiLoop — the name is transparent: Infinite Loop. It wants the very act of "looping" to be able to run forever.
For lineage: the upstream of this idea is the Kimi team's March 2026 AttnRes (Attention Residuals), which did the same thing across layers — replacing fixed-weight accumulation of layer outputs with softmax attention. InfiLoop moves the same principle onto the recurrent-iteration axis, adding time decay and streaming. The entire scoring apparatus is 513 parameters (a 512-dim inspection vector plus 1 decay rate).
The Scorecard
| Method | Params | Sudoku | Maze | ARC-AGI-1 | ARC-AGI-2 | |---|---|---|---|---|---| | DeepSeek-R1 | 671B | 0.0 | 0.0 | 15.8 | 1.3 | | o3-mini-high | – | 0.0 | 0.0 | 34.5 | 3.0 | | Gemini 2.5 Pro | – | – | – | 37.0 | 4.9 | | HRM | 27M | 55.0 | 74.5 | 40.3 | 5.0 | | TRM | 7M | 74.7 | 85.3 | 44.6 | 7.8 | | FPRM | 7M | 94.2 | 87.0 | 47.5 | 6.2 | | InfiLoop | 7M | 97.9 | 87.4 | 49.1 | 13.6 |
Sudoku scores follow the strict all-81-cells-correct rule, on a test set of 420,000 brand-new puzzles. The ARC columns are pass@2: two guesses allowed, one correct counts.
One detail in the table is worth reading: URM (14M) has a footnote on its ARC scores — it reports pass@1, a different ruler from pass@2, so the paper excludes it from direct comparison. That kind of footnote is rare in recursive-model papers, and worth noting.
Efficiency: Half the Compute, One Twenty-Sixth the Depth
Accuracy is only half the story. The real leverage of this architecture is the compute curve.
On the same Sudoku test set, push all three models to 24,960 layers of execution depth: InfiLoop hits 90.9% at 1,872 layers, while FPRM needs 3,744 layers to reach 90.1% — half the compute for a tie. Given the same 3,744 layers, InfiLoop reaches 92.4%.
The starker comparison: InfiLoop at only 936 layers scores 86.5%, while TRM, dug down to 24,960 layers — 26 times the compute — manages only 83.65%. TRM's bottleneck is not insufficient depth; it is the carry rule. Add however many layers you like, old records still get overwritten. This is not the old story of "small models punching cleverly to save compute" — it is "fix the plumbing before talking about throughput."
The Most Interesting Finding: Step Sizes Spontaneously Spike
The scorer was never taught "what is correct" — training includes no extra supervision about correctness. Yet when you dissect its learned behavior, a beautiful pattern emerges.
Take a puzzle solved around round 13: before the solve, the average acceptance weight of new states is held near 0.40 — even below the mindless equal-weight baseline of 0.56 per record — effectively saying "I'm still exploring; new ideas stay on probation." At the round the answer is completed, the weight jumps to 0.91, committing nearly the entire ledger budget to that one complete solution. Afterward the state freezes; nothing moves.
No "self-check" module was explicitly designed. The scorer simply learned: during the guessing phase, discount the new; the moment the answer assembles, go all-in; after that, nobody touches anything. Self-verification grew out of the bookkeeping rule on its own.
One Sober Sentence
The "7M beats giant models" narrative needs to be pinned to a timeline. The frontier models in the paper's table (DeepSeek-R1, Claude 3.7 Sonnet, o3-mini, Gemini 2.5 Pro) cite scores from the October 2025 TRM paper — answer sheets from first-half-2025 models. By September–October 2026, the public ARC-AGI-2 leaderboard shows GPT-6 Astra at 95% and GPT-6.1 Sol at 94.2% — dropped into today's rankings, InfiLoop is not in the first tier.
So the paper's real contribution is not "small models beat big models," but a clean chain of evidence: swap the carry rule between iterations, and the same 7M network goes from peaking at 468 layers to still improving at twenty thousand. The depth was always there; what was wasted was the bookkeeping. How many points this bookkeeping rule is worth when moved onto large models is a question for the next exam.
---
*Paper: Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals (arXiv 2610.11570; Pengxiang Li et al.; Max Planck & Tübingen, Peking University, Surrey, and other institutions). Code: github.com/pixeli99/InfiLoop.*