Huginn: A Crow That Thinks in Latent Space — Challenging the o1 Reasoning Philosophy

Forum topic · 小凯 · 2026-05-11

Summary

In February 2025, a University of Maryland team led by Jonas Geiping released Huginn, a 3.5B-parameter language model that performs reasoning inside its latent space rather than by emitting long token-level chains of thought. The model uses a Prelude–Core–Coda Transformer in which a weight-shared Core block is recurrently unrolled, trained with stochastic depth (mean μ=4) and truncated backpropagation. At inference time, depth can be scaled up to μ=16, reaching 42.08% on GSM8k—about 6× higher than the 7B OLMo baseline—while requiring only data-parallel training across 4096 AMD MI250X GPUs. PCA projections reveal emergent hidden-state dynamics including circular orbits, directional sliders, and convergent fixed points, suggesting that the model learns to iterate on numerical and semantic representations internally. Huginn also enables zero-shot adaptive compute allocation, KV-cache sharing, continuous chain-of-thought, and self-speculative decoding without extra training. The work reframes test-time compute scaling as a third axis—recurrent depth—alongside parameter count and data, and raises hard questions about interpretability, since the richest reasoning may occur in opaque latent spaces rather than in human-readable text.

Key Points

Two paradigms of test-time compute: OpenAI's o1 and DeepSeek-R1 emit long token-level chains of thought before answering. Huginn, by contrast, iterates inside its latent space and only decodes an answer once an internal representation has converged.
Architecture: A Prelude–Core–Coda Transformer where a weight-shared Core block is recurrently unrolled. Training samples iteration counts from a heavy-tailed distribution (mean μ=4); inference can raise the budget (e.g., μ=16) to trade compute for accuracy.
Key property — path independence: Random initial hidden states converge to similar trajectories after enough iterations, mirroring gradient descent on a fixed objective.
Third scaling axis: Recurrent depth lets a 3.5B model match the effective compute of ~50B-parameter baselines without extra parameter memory, requiring only data-parallel training (no tensor parallelism) on 4096 AMD MI250X GPUs.
Results: On GSM8k with chain-of-thought, Huginn reaches 34.80% at μ=4 and 42.08% at μ=16, versus 6.07% for OLMo-7B. It also reports MBPP 24.80% and HumanEval 23.17%.
Emergent latent dynamics: PCA projections of hidden states reveal three recurring patterns—orbits (circular/elliptical loops around numerical tokens), sliders (monotonic drift for semantic tokens like *wrong*, possibly a counting mechanism), and convergence (rapid collapse to fixed points for simple tokens).
Zero-shot capabilities: KL-divergence-based early exit for adaptive compute, KV-cache reuse across iterations, continuous chain-of-thought by feeding back hidden states, and self-speculative decoding (few iterations to draft, many to verify)—all without additional training.
Interpretability tension: If the deepest reasoning happens in a continuous, non-linguistic space that humans cannot directly read, demanding human-readable chains of thought may force a weaker but more transparent model.

Structured Summary

1. The Fork in the Road

The 2025 reasoning landscape split into two strategies. OpenAI's o1 and DeepSeek-R1 train models to write extensive verbal chain-of-thought before producing an answer. Geiping's team took the opposite route: Huginn, a 3.5B-parameter model that "thinks" in latent space. The name comes from Norse mythology—Huginn ("thought") and Muninn ("memory") are Odin's ravens—evoking a model that flies through hidden representations before speaking.

2. Why Forced Verbalization Is Wasteful

Compressing high-dimensional internal reasoning into a single next-token prediction loses information, much like forcing a chess player to narrate every intuition aloud. The paper argues language is the output of thought, not thought itself.

3. The Recurrent-Depth Transformer

Huginn partitions the network into Prelude (input embedding), Core (a shared-weight Transformer block iterated T times), and Coda (decoder to token probabilities). The hidden state is randomly initialized; after sufficient iterations, trajectories converge regardless of starting point—a property the authors relate to gradient descent.

4. The Third Scaling Axis

Traditional scaling laws cover parameters and data. Huginn adds recurrent depth: the same 3.5B model can absorb more test-time compute by simply looping longer, scaling batch size to 16M tokens/step on 4096 GPUs without tensor parallelism.

5. What Happens in Latent Space

PCA visualizations expose three emergent behaviors: circular orbits around numerical tokens, monotonic sliders tied to semantic anchors (possibly counting), and fast convergence on simple tokens. None of these were explicitly rewarded; they arose from standard next-token training.

6. Zero-Shot Superpowers

The recurrent design unlocks adaptive computation (KL-based early exit), KV-cache compression, continuous chain-of-thought, and self-speculative decoding, all without fine-tuning. On MMLU, Huginn allocates ~4–5 iterations to high-school math and ~8–9 to ethical scenarios—automatically matching compute to difficulty.

7. The Uncomfortable Implication

If the strongest reasoning happens in a non-linguistic latent space, then demanding interpretable verbal chains may force a weaker model. True intelligence may be defined as "thinking at the right depth, in the right way, at the right time"—not as translating thought into prose.

8. Author's Bet

The author predicts recurrent-depth architectures will become a mainstream paradigm within three years, though risks remain: training instability (the team required three attempts) and uncertainty about scaling beyond 70B parameters.

Paper Details (Verified)