LoopCD: Recurrent Models Throw Away a Free Sparring Partner Every Pass

When a model runs the same computation four times and throws away the intermediate answers from the first three passes, keeping only the final one, an Apple research team calls that waste. Their proposed method, LoopCD (arXiv 2610.02185), picks up what gets discarded: the early "draft answers" shouldn't be thrown out—they serve as a sparring partner for the final answer. And this partner is free.

First, some background on recurrent models. A class of models (recurrent-depth models) doesn't stack more layers; instead, it runs the same block of parameters repeatedly—like a student mentally working through the same problem four times, deepening understanding with each pass. Examples include ByteDance's Ouro-2.6B-Thinking, Sapient's Huginn-3.5B, and UCSD's Parcae. The benefits are parameter efficiency and elastic depth—Ouro's config has a single knob, total_ut_steps: 4; want more thinking, turn it up. Crucially, at every pass these models can decode an on-the-fly "current best guess." Ouro's own model card states that intermediate loop outputs can serve as reliable proxies for the final answer.

Where the Sparring Partner Comes From

LoopCD builds on Contrastive Decoding, whose idea in one sentence: subtract the amateur's prediction from the expert's, amplifying what only the expert gets right. The idea dates to 2023 but has been stuck on one problem—where does the amateur come from? Prior approaches varied: deliberately garbling prompts to create a weakened model (context-aware decoding), using shallow layers of a plain LLM as a weak reference (DoLa), or training a separate small model as a guide (autoguidance). Every option has costs—altering inputs or maintaining extra models.

Recurrent models solve this for free: the same parameters, run for fewer loops, are a natural amateur. It's the same model, doing the same problem at the same position, differing only in computation—strictly aligned weak/strong prediction pairs with no add-ons. That's what "Almost Free" means: no training, no extra models, no prompt changes.

Two Variants, One Caveat

The paper offers two contrast points. LoopCD-Logits works on probability tables: run the output head for both the final and an early loop, then subtract. The cost is one extra output-head pass. LoopCD-Hidden is cheaper: it operates directly on hidden states before the final fusion layer, skipping the output head entirely—truly zero additional overhead.

One detail worth clarifying, since many retellings blur it: adaptive strength—guiding harder when the model hesitates (the tighter the gap between the top-two probabilities, the stronger the guidance; backing off when one option clearly dominates)—belongs only to the Logits variant. The reason is simple: measuring hesitation requires an "unguided raw probability table," which the Logits version already has; the Hidden version would need another fusion-layer and output-head pass to compute it, instantly nullifying its zero-overhead advantage. So the paper explicitly uses fixed strength for the Hidden variant. Treating adaptivity as a shared feature of both modes is the most common mistake in retellings.

The Numbers

Two headline results. On AIME 2024, Ouro-2.6B-Thinking's pass@1 rose from 61.88% to 73.33%, a gain of 11.45 points. On HumanEval code generation, Huginn went from 22.56% to 31.71%, up 9.15 points—note the low starting point; Huginn was weak at code to begin with, and 31.71% isn't impressive in absolute terms, but under identical model and compute conditions, such a gain shows the sparring partner works.

A more practical result: with this nearly-free guidance, loop counts can be cut in half. Forward-pass compute (FLOPs) drops 22.5% to 48.2% while matching or even exceeding the full-depth unguided version. The saved compute isn't just a discount—it makes the cost of "thinking more" worth spending.

Four experiment groups across four families—ByteDance's Ouro, Sapient's Huginn, UCSD's Parcae, and a recurrent-ized Qwen3—show gains in the same direction. This isn't luck tuned to one vendor's model.

Why It Works

A clean finding from the paper's mechanism analysis: the 2.1–3.5 point gains come entirely from decisions where the two candidates are nearly tied. The logic is tautological yet insightful: reranking can only change the decisions it touches. That's why adaptive strength allocates effort by hesitation—spend it where outcomes can change; "obvious" predictions won't be wrong anyway, and pushing them is pure waste.

One practical nuance: multiple-choice scoring tolerates a wide range of guidance strengths, but autoregressive generation demands tighter control—an error at one step compounds through all subsequent steps.

Closing Thought

Look back at Ouro's model card: "intermediate loop outputs can serve as reliable proxies for the final answer." The manual printed the answer all along; everyone read it. LoopCD's contribution is going one step further: intermediate outputs aren't just a cheap substitute—the gap between them and the final answer is itself a signal. Not asking "what did the draft say," but "what did the final version think beyond the draft." The answer lies in that gap.