Beyond Adam: SOAP and Muon Optimizers for Faster, Label-Efficient Training of ML Interatomic Potentials

Forum topic · 小凯 · 2026-07-05

Summary

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI-driven scientific simulation, yet the community has largely defaulted to Adam and its variants for training. This paper (arXiv:2507.03239) by Gil Harari, Yoel Zimmermann, and Ola Tangen Kulseng implements and systematically compares recently proposed matrix-structured optimizers—Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models. The results show these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon provides only partial gains relative to Adam. The improvements are especially pronounced under partial force supervision, a label-efficient training regime where force labels are limited. The authors conclude that optimizer choice is an overlooked yet impactful design axis for MLIP development, suggesting that simply swapping the optimizer can yield significant gains without changing architectures or datasets.

Paper Overview

Field: Machine Learning Authors: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng Published: 2026-07-04 arXiv: 2507.03239

Abstract

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community.

Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision.

Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.

Key Findings

Matrix-structured optimizers (Muon, SOAP, SOAP-Muon) can significantly beat Adam in MLIP training, in both convergence speed and final accuracy.
SOAP and SOAP-Muon are the most robust and consistently strong performers.
Muon delivers only partial gains compared to Adam.
Gains are especially large under partial force supervision — i.e., label-efficient settings with limited force labels.
Optimizer choice is a neglected but impactful design dimension for MLIPs, independent of architecture and dataset improvements.

---

*Auto-collected on 2026-07-05.*