Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

Forum topic · 小凯 · 2026-07-06

Summary

Machine learning interatomic potentials (MLIPs) are a hallmark of AI for scientific simulation, yet optimizer choice for training has been largely unexplored, with the community defaulting to Adam and its variants. This arXiv paper (2607.02499) by Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, and Boris Kozinsky implements and systematically compares a class of recently proposed matrix-structured optimizers—Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models. The authors find these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon provides only partial gains over Adam. The improvements are particularly pronounced under partial force supervision, indicating better label efficiency. The results show that optimizer choice is an overlooked but impactful design dimension for MLIP training.

Overview

Field: Machine Learning (cs.LG, cs.AI, physics.chem-ph, physics.comp-ph)
Authors: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, Boris Kozinsky
Published: 2026-07-02
arXiv: 2607.02499

Abstract

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design dimension for MLIPs.

Key Takeaways

Matrix-structured optimizers (Muon, SOAP, SOAP-Muon) substantially outperform Adam for MLIP training in both convergence speed and final accuracy.
SOAP and the hybrid SOAP-Muon are the most robust and consistently strong choices; Muon alone offers only partial gains over Adam.
Gains are especially significant in label-efficient settings with partial force supervision.
Optimizer selection should be treated as a first-class design decision when training NequIP, Allegro, and similar MLIP architectures.

--- *Auto-collected on 2026-07-06*