The "click" moment of understanding
You've probably had this experience: stuck on a hard exam problem, your first answer feels off. So you restart—new angle, new starting point, compute again. Second pass, third pass... gradually the answer firms up until something clicks: yes, that's it.
That click has a mathematical name—convergence to an attractor.
CMU Locus Lab's (Zico Kolter's group) latest paper Equilibrium Reasoners (EqR) shows neural networks can have the same experience. The key isn't making the network bigger, but shaping its internal dynamics into a "good" landscape—so the correct answer becomes a stable attractor, and reasoning naturally slides toward the right solution.
The result? A tiny 5M-parameter model jumps from 2.6% to 99.8% accuracy on extreme Sudoku, effectively unrolling over 40,000 layers.
Why does "thinking more" sometimes help and sometimes hurt?
Test-time compute scaling is one of today's hottest AI directions: spend more inference compute, get higher accuracy. But reality is messier—often more iterations make things worse, as the model spirals into confused loops.
Why? Because "thinking more" only works if your reasoning process has a correct "gravitational center"—an attractor. Without one, extra iterations are wasted noise.
Attractors: the gravity of thought
Imagine a ball rolling on bumpy terrain toward the nearest valley—that's an attractor. EqR's core insight: an iterative reasoning model's internal state space is exactly such a terrain. Each iteration is the ball taking a step. If the terrain is shaped well, the correct answer is the widest, deepest valley—no matter where the ball starts, it rolls toward the right solution.
The authors classify reasoning models' state spaces into four terrain modes:
| Mode | Terrain | Outcome | Fix | |------|---------|---------|-----| | (a) No correct attractor | All wrong valleys | Always wrong | Retrain | | (b) Correct and wrong attractors coexist | Good and bad valleys | Maybe right, maybe wrong | More restarts (breadth scaling) | | (c) Correct attractor too narrow | Good valley too small to find | Occasionally right | Restarts + more steps | | (d) Ideal terrain | Correct valley wide and deep | Always right | EqR's goal |
Key insight: depth (more steps) and breadth (more restarts) are complementary. Depth stabilizes the ball in its current valley; breadth gives more chances to find the right valley. But breadth only helps once depth is sufficient.
Five steps from feedforward to iterative reasoner
1. Weight tying — 42 independent layers become 2 shared layers reused iteratively. Parameters drop from 105M to 5M, yet accuracy jumps from 2.6% to 32.6%. Shared weights create iterative dynamics—like revisiting the same problem. 2. Segmented online training (SOT) — update parameters every few iterations rather than after all of them, like checking answers after a few problems. Accuracy: 74.7%. 3. Hierarchical iteration — fast and slow state variables updated at different rates, like the brain's fast/slow thinking. Accuracy: 76.5%. 4. Adaptive computation time (ACT) — the model decides when it's "thought enough": early exit on easy problems, more steps on hard ones. Accuracy: 84.8%. 5. Attractor landscape shaping (EqR's core innovation) — the killer technique.
Two lightweight tricks to reshape the landscape
1. Randomized State Initialization (RI)
Prior iterative models always trained from the same fixed initial state—like always starting problems the same way. EqR starts every training run from a random position, forcing the model to find correct answers from any starting point, vastly widening the correct attractor's basin. On Maze, this alone lifts accuracy from 44.9% to 68.6%.
2. Noise Injection (NI)
Tiny Gaussian noise at each iteration—like letting your mind wander slightly, which can help escape local optima. With noise strength β=0.01 plus light damping (λ=0.05), Maze accuracy jumps from 68.6% to 82.2%.
The two tricks are complementary: RI widens "where you start," NI widens "how you get there."
The numbers: 2.6% to 99.8%
| Method | Sudoku Extreme | Maze Unique | |--------|---------------|-------------| | Feedforward (64 layers) | 2.6% | 0.0% | | HRM | 55.0% | 0.3% | | TRM | 84.8% | 44.9% | | URM | 77.6% | 51.4% | | EqR (D=16, B=1) | 86.4% | 82.2% | | EqR (D=64, B=1) | 93.0% | 88.9% | | EqR (D=64, B=128) | 99.8% | 93.0% |
Striking: trained with only 16 iterations, EqR extrapolates to 1024+ iterations at inference (equivalent to 40,000+ layers), climbing from 86.4% to 99.8%. Attractor dynamics learned at training depth safely extrapolate far beyond it.
Convergence: the cheapest answer quality checker
Once the attractor landscape is shaped, the residual (distance from current state to the fixed point) becomes a reliable quality metric. No external verifier needed—just pick the trajectory that converges best. During breadth scaling, Top-1 Converged selection beats majority voting in both efficiency and accuracy.
ACT lets easy problems stop after ~5.4 iterations on average. At D=1024, average NFE drops from 1024 to 58.7—a 17.4× efficiency gain with under 1% accuracy loss.
Engineering takeaways
1. Small models can do big reasoning. EqR's core model is only 5M parameters—what matters is shaped dynamics, not size. Big implications for edge deployment. 2. Train-inference extrapolation works. Train 16 steps, run 1024+, and performance keeps improving—genuine generalization, not overfitting to training depth. 3. Convergence signals beat voting. A mathematical metric (residual) selects better answers with less compute than majority voting. 4. Two universal, lightweight tricks. RI and NI require no architecture changes, extra data, or external verifiers—any iterative reasoner can try them. 5. Sequence depth before breadth. First ensure single trajectories converge (depth), then add restarts (breadth). Reversing the order hurts.
Reflections
The deeper question: what are we actually teaching neural networks?
The traditional view: networks learn input-to-output mappings. EqR suggests iterative models learn a dynamical system—a force field from which correct answers naturally emerge. This mirrors energy-landscape ideas in physics: protein folding, spin glasses, phase transitions—nature's most elegant computation happens by sliding down energy landscapes. EqR essentially replicates that principle in neural networks.
The concept of "attractor alignment" hints at a new training paradigm: not teaching the network what the answer is, but teaching it to construct a force field where answers naturally surface. That could be a key step toward truly scalable reasoning.
Open question: current experiments are limited to structured tasks like Sudoku and mazes. Whether attractor landscapes can be so clearly defined and shaped for open-domain reasoning (math proofs, code generation) remains to be seen—but viewing reasoning as dynamical-system evolution opens a new door.
---
Paper | arxiv.org/abs/2605.21488 Code | github.com/locuslab/EqR (upcoming) Authors | Benhao Huang, Zhengyang Geng, Zico Kolter (CMU Locus Lab) Venue | ICML 2026