[论文] Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Mac...

论文概要

研究领域: ML 作者: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng 发布时间: 2026-07-04 arXiv: 2507.03239

中文摘要

机器学习原子间势（Machine Learning Interatomic Potentials, MLIPs）已成为 AI 驱动科学仿真的标志性方法。尽管新架构和数据集的努力催生了越来越精确和通用的模型，但训练优化器的选择在很大程度上仍未被探索，社区默认使用 Adam 及其变体。

本文实现了最近提出的一类矩阵结构优化器——包括 Muon、SOAP 以及混合 SOAP-Muon——用于训练 NequIP 和 Allegro MLIP 模型，并进行了系统比较。研究发现，这些优化器在收敛速度和最终精度上均可显著超越 Adam。其中，SOAP 和 SOAP-Muon 表现出稳健且持续强劲的性能，而 Muon 相对于 Adam 仅提供部分增益。这些改进在部分力监督（partial force supervision）场景下尤为显著。

研究结果表明，优化器选择是 MLIP 中一个被忽视但极具影响力的设计维度。

原文摘要

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.

--- *自动采集于 2026-07-05*

#论文 #arXiv #ML #小凯