[论文] Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

论文概要

研究领域: ML 作者: Benhao Huang, Zhengyang Geng, Zico Kolter 发布时间: 2025-05-20 arXiv: 2505.15988

中文摘要

通过迭代更新隐状态来扩展测试时计算已成为强大的推理范式。然而，使这些迭代模型能够泛化超出记忆模式的内部机制仍不清楚。我们假设可泛化推理源于学习任务条件吸引子：稳定不动点对应有效解的潜在动力系统。我们通过平衡推理器(EqR)将这一过程形式化，它无需外部验证器或任务特定先验即可实现测试时扩展。EqR沿两个轴扩展内部动态：深度（运行更多迭代）和广度（聚合多初始化的随机轨迹）。经验上，测试时扩展的收益与向解对齐吸引子的更强收敛紧密耦合。这种吸引子视角使神经网络能够根据任务难度自适应分配测试时计算。简单情况在1-5步内收敛，困难情况则从大规模测试时扩展中获益。通过展开多达等效40,000层，可扩展隐推理将前馈模型在Sudoku-Extreme上的准确率从2.6%提升到超过99%。这些结果表明，学习到的吸引子景观为理解迭代隐模型中的可扩展推理提供了有用的机制透镜。

原文摘要

Scaling test-time compute by iteratively updating a latent state has emerged as a powerful paradigm for reasoning. Yet the internal mechanisms that enable these iterative models to generalize beyond memorized patterns remain unclear. We hypothesize that generalizable reasoning arises from learning task-conditioned attractors: latent dynamical systems whose stable fixed points correspond to valid solutions. We formalize this process through Equilibrium Reasoners (EqR), which enable test-time scaling without external verifiers or task-specific priors. EqR scales internal dynamics along two axes: depth, by running more iterations, and breadth, by aggregating stochastic trajectories from multiple initializations. Empirically, gains from test-time scaling are tightly coupled with stronger conve...

自动采集于 2026-05-22

#论文 #arXiv #ML #小凯