Beyond Adam: SOAP and Muon Optimizers for Faster, Label-Efficient Training of ML Interatomic Potentials

Forum topic · 小凯 · 2026-07-06

Summary

A new study (arXiv:2607.02499) examines an overlooked design choice in training machine learning interatomic potentials (MLIPs): the optimizer. While the community defaults to Adam and its variants, the authors implement and systematically compare recently proposed matrix-structured optimizers—Muon, SOAP, and the hybrid SOAP-Muon—when training NequIP and Allegro MLIP models. Results show these optimizers substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon prove robust and consistently strong, whereas Muon delivers only partial gains relative to Adam. Improvements are especially pronounced under partial force supervision, meaning comparable accuracy can be reached with fewer labeled training examples. The findings suggest optimizer selection is a high-impact, underexplored dimension in MLIP development, offering a simple drop-in improvement for AI-driven scientific simulation workflows.

Overview

Field: Machine Learning / AI for Science
Authors: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, Boris Kozinsky
Published: 2026-07-02
arXiv: 2607.02499
Categories: cs.LG, cs.AI, physics.chem-ph, physics.comp-ph

Summary (translated)

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design dimension for MLIPs.

Key Takeaways

Optimizer choice for MLIP training is underexplored; Adam has been the default.
Matrix-structured optimizers (Muon, SOAP, SOAP-Muon) beat Adam in convergence speed and final accuracy.
SOAP and SOAP-Muon are the most robust and consistently strong choices; Muon alone gives only partial gains.
Gains are especially significant with partial force supervision, enabling more label-efficient training.

--- *Auto-collected on 2026-07-06*