Beyond Adam: SOAP and Muon Optimizers for Faster, Label-Efficient MLIP Training

Forum topic · 小凯 · 2026-07-06

Summary

Machine learning interatomic potentials (MLIPs) are a leading application of AI in scientific simulation, yet the community has largely defaulted to Adam-family optimizers for training. This work (arXiv:2607.02499, July 2026) implements and systematically compares recently proposed matrix-structured optimizers—Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models. The authors find these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust, consistently strong methods, while Muon delivers only partial gains over Adam. The improvements are especially pronounced under partial force supervision, meaning comparable accuracy can be achieved with fewer labeled configurations, reducing expensive quantum-chemistry labeling costs. The results identify optimizer choice as an underexplored but highly impactful design dimension in MLIP development, with practical implications for faster and more label-efficient training of interatomic potentials.

Overview

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of training optimizer has largely remained unexplored, defaulting to Adam and its variants in the community.

This paper implements and systematically compares a class of recently proposed matrix-structured optimizers—including Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models.

Key findings

Matrix-structured optimizers can substantially outperform Adam in both convergence speed and final accuracy.
SOAP and SOAP-Muon emerge as robust and consistently strong methods.
Muon alone provides only partial gains relative to Adam.
Improvements are particularly pronounced under partial force supervision, enabling more label-efficient training with reduced quantum-chemistry labeling cost.

Implications

The results suggest that optimizer choice is an overlooked but highly impactful design dimension for MLIPs, complementary to ongoing work on architectures and datasets.

Paper details

Field: Machine Learning (cs.LG, cs.AI, physics.chem-ph, physics.comp-ph)
Authors: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, Boris Kozinsky
arXiv: 2607.02499
Published: 2026-07-02