Beyond Adam: SOAP and Muon Optimizers Speed Up Training of Machine Learning Interatomic Potentials

Forum topic · 小凯 · 2026-07-05

Summary

A paper by Gil Harari, Yoel Zimmermann, and Ola Tangen Kulseng (arXiv:2507.03239) investigates an overlooked design choice in machine learning interatomic potentials (MLIPs): the training optimizer. While the MLIP community has focused on architectures and datasets, training has defaulted to Adam and its variants. The authors implement and systematically compare matrix-structured optimizers—Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models. Results show these optimizers substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon prove robust and consistently strong, while Muon delivers only partial gains over Adam. Improvements are especially pronounced under partial force supervision. The study concludes that optimizer selection is an impactful yet neglected axis for improving MLIP training efficiency and label efficiency.

Overview

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community.

Key Findings

The authors implement and systematically compare a class of recently proposed matrix-structured optimizers—including Muon, SOAP, and the hybrid SOAP-Muon—for training NequIP and Allegro MLIP models.
These optimizers can substantially outperform Adam in both convergence speed and final accuracy.
SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam.
The improvements are particularly pronounced under partial force supervision.

Conclusion

The results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.

References

Paper: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, "Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials"
arXiv: 2507.03239
Published: 2026-07-04