Beyond Adam: SOAP and Muon Optimizers for Faster, Label-Efficient Training of ML Interatomic Potentials

Forum topic · 小凯 · 2026-07-05

Summary

A new paper (arXiv:2507.03239) by Gil Harari, Yoel Zimmermann, and Ola Tangen Kulseng examines an overlooked design choice in machine learning interatomic potentials (MLIPs): the training optimizer. While the MLIP community has defaulted to Adam and its variants, the authors implement and systematically compare recently proposed matrix-structured optimizers—Muon, SOAP, and a hybrid SOAP-Muon—for training NequIP and Allegro models. Results show these optimizers substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon prove robust and consistently strong, whereas Muon delivers only partial gains over Adam. The improvements are especially pronounced under partial force supervision, a label-efficient training regime where force labels are available for only a subset of atoms or configurations. The study concludes that optimizer selection is an impactful, underexplored design axis for MLIPs and AI-driven scientific simulation more broadly.

Paper Overview

Field: Machine Learning Authors: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng Published: 2026-07-04 arXiv: 2507.03239

Abstract

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community.

The authors implement and systematically compare a class of recently proposed matrix-structured optimizers—Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models.

Key Findings

Matrix-structured optimizers can substantially outperform Adam in both convergence speed and final accuracy.
SOAP and SOAP-Muon emerge as robust and consistently strong methods across experiments.
Muon alone provides only partial gains relative to Adam.
The improvements are particularly pronounced under partial force supervision, where training uses force labels only on a subset of the data—making these optimizers valuable for label-efficient training.

Conclusion

Optimizer choice is an overlooked yet impactful design axis for MLIPs, suggesting that simply swapping Adam for SOAP or SOAP-Muon can yield faster and more accurate potential models without architectural changes.

--- *Auto-collected on 2026-07-05*