AdaGrad Converges in a Robust Sense: Almost Sure Last-Iterate Rates under Any Stopping Time
Lists
Only you see this
Abstract
AdaGrad has become a widely used algorithm for training deep models. Recently, the study of almost sure last-iterate convergence rates in stochastic optimization has attracted increasing attention, as it provides the guarantee of stability and **robustness** for arbitrary single trajectory. While such results are well understood for stochastic gradient descent (SGD), the corresponding analysis for AdaGrad remains limited. In this paper, we establish **almost sure** convergence rates of AdaGrad for the **last-iterate** in the (strongly) convex setting and for the best-iterate in the non-convex setting, both valid under **arbitrary** stopping times and with a flexible dependence on gradient history.
BibTeX
@misc{ou2026adagrad,
author = {Weiming Ou and Zhihao Liu and Xiao Wang},
title = {{AdaGrad Converges in a Robust Sense: Almost Sure Last-Iterate Rates under Any Stopping Time}},
year = {2026},
url = {https://openreview.net/forum?id=DwWorqSjwv},
howpublished = {OpenReview},
note = {Submitted to ICLR 2026},
}Loading related content...