English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original
智柴网 中文原文

Towards Looped Models Done Right, Part II: Rethinking Recursion at Fixed Points

Forum topic · 小凯 · 2026-10-07

Summary

This paper explores how looped (recurrent) language models behave when their recurrent states approach fixed points, where the specific path taken by the recursion matters less. This property enables several efficiency gains: truncated backpropagation during training, near-lossless terminal KV sharing for decoding, a distilled student model achieving 1.79x faster prefill, and reinforcement learning updates computed from saved rollout states, which are 2x faster than backpropagating through replayed trajectories. The authors also improve two training components that shape fixed points: depth priors and input injection. Fixed-depth training breaks KV sharing, while Huginn's broad depth prior supports sharing but dilutes supervision at target depths; the proposed learned prior is derived from prediction feedback and kept broad via an entropy term. For input injection, existing schemes allow states to amplify or cancel injected components along the input direction; the authors introduce orthogonal injection to remove this component. Experiments across 100M to 1.6B parameter scales show that learned priors and orthogonal injection each reduce perplexity relative to Huginn's prior and standard injection schemes. At 1.6B parameters, a learned prior with a 3x smaller KV cache matches the downstream average performance of fixed-depth training with a full cache. Paper: arXiv 2610.06833.

Paper Overview

Field: LLM Authors: Benhao Huang, Chufan Shi, Junlin Chen, Shicheng Wen, Zhengzhong Liu, Eric Xing, Xuezhe Ma Published: 2026-10-05 arXiv: 2610.06833

Summary

Looped language models pay a cost for every recursion step during training, decoding, prefill, and reinforcement learning (RL). The closer the recurrent states get to fixed points, the less the path taken to reach them matters. This enables:

  • Truncated backpropagation during training
  • Terminal KV sharing in decoding with near-lossless accuracy
  • 1.79x faster prefill via a distilled student model
  • RL updates computed from saved rollout states, about 2x faster than backpropagating through replayed trajectories
The paper therefore improves two training components that shape these fixed points: depth priors and input injection.

Depth Prior

Fixed-depth training breaks KV sharing, while Huginn's broad depth prior supports sharing but overly dilutes supervision at the target depth. The authors learn the prior from prediction feedback, using an entropy term to keep it broad.

Input Injection

Existing injection schemes allow the state's component along the input direction to amplify or cancel the injected input. The authors remove this component through orthogonal injection.

Results

From 100M to 1.6B parameters, the learned prior and orthogonal injection each reduce perplexity at every scale relative to the Huginn prior and existing injection schemes. At 1.6B parameters, a learned prior with a 3x smaller KV cache matches the downstream average performance of fixed-depth training using the full cache.

Original Abstract

The closer recurrent states get to fixed points, the less the path matters. This enables truncated backpropagation, terminal KV sharing, 1.79x faster distilled prefill, and 2x faster RL updates. Learned depth prior and orthogonal injection lower perplexity at every scale from 100M to 1.6B.

---

*Auto-collected on 2026-10-07*

Tags

#paper#arxiv#llm#looped-models#fixed-points#efficiency#reinforcement-learning#kv-cache

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178635684