Mirror Descent on Riemannian Manifolds: RMD Framework with Non-Asymptotic Convergence Guarantees

Forum topic · 小凯 · 2026-03-19

Summary

A paper by Jiaxin Jiang, Lei Shi, and Jiyuan Tan (arXiv:2503.13851, March 2025) generalizes Mirror Descent (MD), a scalable first-order optimization method widely used in image processing, policy optimization, and neural network training, to optimization over Riemannian manifolds. The authors develop a Riemannian Mirror Descent (RMD) framework via reparameterization and propose a stochastic variant of RMD. Non-asymptotic convergence guarantees are established for both RMD and its stochastic counterpart. As an application to the Stiefel manifold, the RMD framework reduces to the Curvilinear Gradient Descent (CGD) method. Furthermore, specializing the stochastic RMD framework to the Stiefel setting yields a stochastic extension of CGD, effectively addressing large-scale manifold optimization problems. The work bridges classical mirror descent theory with Riemannian optimization, providing both theoretical convergence analysis and practical algorithms for constrained optimization on manifolds.

Paper Overview

Field: Machine Learning
Authors: Jiaxin Jiang, Lei Shi, Jiyuan Tan
Published: 2025-03-18
arXiv: 2503.13851

Abstract

Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, the authors develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. They also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD.

As an application to the Stiefel manifold, the RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, the result is a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.

Key Contributions

A Riemannian Mirror Descent (RMD) framework derived via reparameterization
A stochastic variant of RMD for large-scale problems
Non-asymptotic convergence guarantees for both RMD and stochastic RMD
Application to the Stiefel manifold, recovering CGD and extending it to the stochastic setting

---

*Auto-collected on 2026-03-19*