Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Youngsuk Park, Sauptik Dhar, Stephen P. Boyd, Mohak Shah

Abstract

This paper proposes an adaptive metric selection strategy called diagonal Barzilai-Borwein (DBB) stepsize for the popular Variable Metric Proximal Gradient (VM-PG) algorithm [1], [2]. The proposed approach better captures the local geometry of the problem while keeping the per-step computation cost similar to the widely used scalar Barzilai-Borwein (BB) stepsize. We provide the theoretical convergence analysis for VM-PG using DBB stepsize. Finally, our empirical results show ~10 - 40 % improvement in convergence times for the VM-PG using DBB compared to the BB stepsize for different machine learning problems on several datasets.
BibTeX
@inproceedings{park2019variable,
  author = {Youngsuk Park and Sauptik Dhar and Stephen P. Boyd and Mohak Shah},
  title = {{Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  doi = {10.1109/icassp40776.2020.9054193},
}
Figure