no. 14 · Friday 13 November 2026
Nonlinear Eigen-approach ADMM for Sparse Optimization on Stiefel Manifold
Abstract
With growing interest in machine learning and data science, creating efficient algorithms to optimize various dimension reduction objectives has become particularly important. This paper is concerned with optimization problems on the Stiefel manifold whose objective functions take the form as the sum of a smooth function and a nonsmooth one. A common practice is to split such a problem into a smooth optimization subproblem on the Stiefel manifold and a nonsmooth one and solve them by some version of the alternating direction method of multipliers (ADMM). Usually the smooth subproblem is solved by standard optimization techniques skillfully adapting to the manifold. However, often the smooth subproblem has strong linear algebra characteristics which, if carefully exploited, can lead to much more efficient solvers than the standard optimization techniques that ignore the characteristics. In this paper, it is proposed to tackle the smooth subproblem through a nonlinear eigen-approach which can fully utilize rich linear algebra structures that dominantly present in objective functions from today's data science applications. We conduct convergence analysis and analyze computational complexity for obtaining an $ε$-stationary point. Numerical experiments demonstrate the advantages of the proposed algorithm compared to the state-of-the-art approaches.
- comments
- 36 pages, 5 figures, 1 table
- fields
- Optimization and Control