Paperlane

no. 14 · Friday 13 November 2026

Nonlinear Eigen-approach ADMM for Sparse Optimization on Stiefel Manifold

Jiawei Wang, Li Wang, Richard Yi Da Xu and Ren-Cang Li

math.OC · v2 · 36 pp

Abstract

With growing interest in machine learning and data science, creating efficient algorithms to optimize various dimension reduction objectives has become particularly important. This paper is concerned with optimization problems on the Stiefel manifold whose objective functions take the form as the sum of a smooth function and a nonsmooth one. A common practice is to split such a problem into a smooth optimization subproblem on the Stiefel manifold and a nonsmooth one and solve them by some version of the alternating direction method of multipliers (ADMM). Usually the smooth subproblem is solved by standard optimization techniques skillfully adapting to the manifold. However, often the smooth subproblem has strong linear algebra characteristics which, if carefully exploited, can lead to much more efficient solvers than the standard optimization techniques that ignore the characteristics. In this paper, it is proposed to tackle the smooth subproblem through a nonlinear eigen-approach which can fully utilize rich linear algebra structures that dominantly present in objective functions from today's data science applications. We conduct convergence analysis and analyze computational complexity for obtaining an $ε$-stationary point. Numerical experiments demonstrate the advantages of the proposed algorithm compared to the state-of-the-art approaches.

comments
36 pages, 5 figures, 1 table
fields
Optimization and Control
pdf ↗ page arXiv:2406.01885 ↗