Huginn: A Recurrent-Depth Model That Reasons in Latent Space and Challenges the o1 Philosophy

This post from zhichai.net analyzes the Huginn model (arXiv:2502.05171), a recurrent-depth latent-reasoning language model, and contrasts it with token-based reasoning models like OpenAI o1 and DeepSeek-R1. Below is a structured English summary of the full article.

Key points

Two divergent paths for AI reasoning

• o1 / DeepSeek-R1 path: the model externalizes its reasoning as tens of thousands of tokens of chain-of-thought text before answering.
• Huginn path (Jonas Geiping's team, University of Maryland): a 3.5B-parameter model that "thinks" in latent space rather than "speaking" its thoughts.
• The paper's core claim: compressing expensive high-dimensional internal reasoning into single tokenized next-token predictions is wasteful. Language is the output of thought, not thought itself.

Architecture: a Transformer that "flies"

• Prelude-Core-Coda design: standard layers embed input; a weight-shared Core block iterates any number of times over the hidden state; Coda layers decode the final hidden state into output.
• Initialized from a random Gaussian state; the authors demonstrate path independence — trajectories converge to similar patterns regardless of the starting point, analogous to gradient descent.
• Training samples the number of iterations from a heavy-tailed distribution (mean μ=4) with truncated backprop (last 3 steps only).

A third scaling axis

• Beyond parameters and data, Huginn introduces recurrent depth as a scaling dimension: at test time, more iterations expand effective computation toward a ~50B-parameter equivalent without extra memory.
• GSM8k CoT benchmark comparison (from the post):

| Model | Params | Data | GSM8k CoT | |---|---|---|---| | Pythia-2.8b | 2.8B | 0.3T | 1.90% | | Pythia-6.9b | 6.9B | 0.3T | 2.81% | | OLMo-7B | 7B | 2.5T | 6.07% | | Huginn (μ=4) | 3.5B | 0.8T | 34.80% | | Huginn (μ=16) | 3.5B | 0.8T | 42.08% |

• Trained on 4096 AMD MI250X GPUs (Oak Ridge Frontier) with data parallelism only — no tensor or pipeline parallelism needed, thanks to weight sharing; batch size of 16M tokens/step.

Emergent latent-space dynamics

PCA visualization reveals three unplanned patterns:
• Orbits: hidden states for numeric-reasoning tokens trace circular/elliptical paths.
• Sliders: semantically loaded tokens drift steadily in one direction, possibly a counting mechanism.
• Convergence: simple tokens settle quickly to fixed points; complex tokens iterate longer.

Zero-shot capabilities

Without extra engineering, Huginn natively supports:
• Adaptive computation (exit early via KL divergence between successive iterations; ~4–5 steps on simple MMLU math vs. ~8–9 on moral scenarios)
• KV-cache compression via looped budget steps
• Continuous chain-of-thought (initializing with a previous hidden state)
• Self-speculative decoding (few iterations to draft, more to verify)

The uncomfortable implication

The author argues the industry's assumption — that scaling test-time compute requires generating more tokens — is being challenged. If the strongest reasoning happens in latent space, interpretability may be at odds with capability: forcing AI to verbalize thought serves human readability, not optimal computation. The author bets that recurrent depth becomes a core paradigm within 3 years, while acknowledging risks (training instability, uncertain scaling to 70B+ models).

Paper details

• Title: Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
• Authors: Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, et al. (University of Maryland and collaborators)
• arXiv: 2502.05171 (Feb 2025)
• Model/code: https://huggingface.co/tomg-group-umd/huginn-0125 / https://github.com/seal-rg/recurrent-pretraining
• Other results: MBPP 24.80%, HumanEval 23.17%; effective depth up to 132 layers