Geometric Analysis of Nonconvex Optimization Landscapes for Overcomplete Learning
Lists
Only you see this
Abstract
Learning overcomplete representations finds many applications in machine learning and data analytics. In the past decade, despite the empirical success of heuristic methods, theoretical understandings and explanations of these algorithms are still far from satisfactory. In this work, we provide new theoretical insights for several important representation learning problems: learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictionaries. We formulate these problems as $\ell^4$-norm optimization problems over the sphere and study the geometric properties of their nonconvex optimization landscapes. For both problems, we show the nonconvex objective has benign (global) geometric structures, which enable the development of efficient optimization methods finding the target solutions. Finally, our theoretical results are justified by numerical simulations.
BibTeX
@inproceedings{qu2020geometric,
author = {Qing Qu and Yuexiang Zhai and Xiao Li and Yuqian Zhang and Zhihui Zhu},
title = {{Geometric Analysis of Nonconvex Optimization Landscapes for Overcomplete Learning}},
booktitle = {International Conference on Learning Representations},
year = {2020},
}| Analysis of the Optimization Landscapes for Overcomplete Representation Learning | arXiv · 2019 |
| Complete Dictionary Recovery over the Sphere | arXiv · 2015 |
| Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picture | IEEE Trans. Information Theory · 2017 |
| Complete Dictionary Recovery Using Nonconvex Optimization | ICML · 2015 |
| Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Method | IEEE Trans. Information Theory · 2017 |
| Global Identifiability of Overcomplete Dictionary Learning via L1 and Volume Minimization | ICLR · 2025 |