Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning (arXiv 2505.15988)

Forum topic · 小凯 · 2026-05-22

Summary

Equilibrium Reasoners (EqR), a paper by Benhao Huang, Zhengyang Geng, and Zico Kolter (arXiv:2505.15988, May 2025), proposes that generalizable reasoning in iterative latent-state models emerges from learning task-conditioned attractors—stable fixed points of latent dynamical systems that correspond to valid solutions. EqR enables test-time scaling without external verifiers or task-specific priors by scaling internal dynamics along two axes: depth (running more iterations) and breadth (aggregating stochastic trajectories from multiple initializations). Empirically, test-time scaling gains are tightly coupled with stronger convergence toward solution-aligned attractors. The attractor perspective lets networks allocate test-time compute adaptively by task difficulty: easy cases converge within 1-5 steps, while hard cases benefit from large-scale scaling. By unrolling up to the equivalent of 40,000 layers, scalable latent reasoning improves a feedforward model's accuracy on Sudoku-Extreme from 2.6% to over 99%. The authors argue the learned attractor landscape provides a useful mechanistic lens for understanding scalable reasoning in iterative implicit models.

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

Paper Overview

Field: Machine Learning Authors: Benhao Huang, Zhengyang Geng, Zico Kolter Published: 2025-05-20 arXiv: 2505.15988

Abstract

Scaling test-time compute by iteratively updating a latent state has emerged as a powerful paradigm for reasoning. Yet the internal mechanisms that enable these iterative models to generalize beyond memorized patterns remain unclear. We hypothesize that generalizable reasoning arises from learning task-conditioned attractors: latent dynamical systems whose stable fixed points correspond to valid solutions. We formalize this process through Equilibrium Reasoners (EqR), which enable test-time scaling without external verifiers or task-specific priors.

EqR scales internal dynamics along two axes:

Depth: running more iterations
Breadth: aggregating stochastic trajectories from multiple initializations

Empirically, gains from test-time scaling are tightly coupled with stronger convergence toward solution-aligned attractors. This attractor perspective allows neural networks to adaptively allocate test-time compute based on task difficulty: simple cases converge within 1-5 steps, while hard cases benefit from large-scale test-time scaling. By unrolling up to the equivalent of 40,000 layers, scalable latent reasoning improves a feedforward model's accuracy on Sudoku-Extreme from 2.6% to over 99%.

These results suggest that learned attractor landscapes offer a useful mechanistic lens for understanding scalable reasoning in iterative implicit models.

--- *Auto-collected on 2026-05-22*