Non-Euclidean High-Order Smooth Convex Optimization

Juan Pablo Contreras, Cristóbal Guzmán, David Martínez-Rubio

Abstract

We develop algorithms for the optimization of convex objectives that have Hölder continuous $q$-th derivatives by using a $q$-th order oracle, for any $q \geq 1$. Our algorithms work for general norms under mild conditions, including the $\ell_p$-settings for $1\leq p\leq \infty$. We can also optimize structured functions that allow for inexactly implementing a non-Euclidean ball optimization oracle. We do this by developing a non-Euclidean inexact accelerated proximal point method that makes use of an \emph{inexact uniformly convex regularizer}. We show a lower bound for general norms that demonstrates our algorithms are nearly optimal in high-dimensions in the black-box oracle model for $\ell_p$-settings and all $q \geq 1$, even in randomized and parallel settings. This new lower bound, when applied to the first-order smooth case, resolves an open question in parallel convex optimization.
BibTeX
@misc{contreras2024non,
  author = {Juan Pablo Contreras and Cristóbal Guzmán and David Martínez-Rubio},
  title = {{Non-Euclidean High-Order Smooth Convex Optimization}},
  year = {2024},
  eprint = {2411.08987},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.08987},
}

Loading related content...

Figure