Nonconvex regularizer and homotopy-based sparse optimization: Convergent algorithms and applications
Lists
Only you see this
Abstract
Non-convex optimization regularized with a sparsity function has many applications in machine learning and other fields. Non-convex relaxation of the sparsity function often leads to more effective sparse optimization algorithms. In this paper, we propose a new weighted regularizer to approximate the sparsity function, and derive a weighted thresholding operator for the sparse optimization problem with the regularizer. Then, iterative weighted thresholding algorithms are designed, followed with an acceleration by using Nesterov's acceleration method and non-monotone line search. Under the Kurdyka-Łojasiewicz (KL) property, the smoothness and the appropriate convexity assumptions, we prove that the two algorithms are convergent and the convergence rates are $ O({1}/{k}) $ and $ O({1}/{k^2}) $ respectively, where $ k $ is the iteration counter. Moreover, we develop convergent practical homotopy algorithms by invoking the two iterative weighted thresholding algorithms as subroutines respectively. A series of numerical experiments demonstrate that our algorithms are superior in both average recovery rate and average running time for the sparse recovery problem, and are competitive in solution quality and average running time for the logistic regression problem, compared to state-of-the-art algorithms.
BibTeX
@article{huang2025nonconvex,
author = {Zilin Huang and Jiang Lan-fan and Weiwei Cao and Wenxing Zhu},
title = {{Nonconvex regularizer and homotopy-based sparse optimization: Convergent algorithms and applications}},
journal = {Journal of Industrial and Management Optimization},
year = {2025},
doi = {10.3934/jimo.2025022},
}Loading related content...