Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

论文概要

研究领域: ML 作者: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, Boris Kozinsky 发布时间: 2026-07-02 arXiv: 2607.02499 分类: cs.LG, cs.AI, physics.chem-ph, physics.comp-ph

中文摘要

机器学习原子间势（MLIPs）已成为科学模拟AI的标志性方法。尽管新架构和数据集的努力带来了越来越精确和通用的模型，但训练优化器的选择在很大程度上仍未被探索，社区中默认使用Adam及其变体。本文实现并系统比较了一类近期提出的矩阵结构优化器，包括Muon、SOAP及其混合SOAP-Muon，用于训练NequIP和Allegro MLIP模型。我们发现这些优化器在收敛速度和最终精度上均能显著优于Adam。SOAP和SOAP-Muon表现为稳健且始终强劲的方法，而Muon相对于Adam仅提供部分增益。在部分力监督下，这些改进尤为显著。我们的结果表明，优化器选择是MLIPs中一个被忽视但具有影响力的设计维度。

原文摘要

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pr...

--- *自动采集于 2026-07-06*

#论文 #arXiv #ML #小凯