TRSVR: An Adaptive Stochastic Trust-Region Method with Variance Reduction
Lists
Only you see this
Abstract
We propose a stochastic trust-region method for unconstrained nonconvex optimization that incorporates stochastic variance-reduced gradients (SVRG) to accelerate convergence. Unlike classical trust-region methods, the proposed algorithm relies solely on stochastic gradient information and does not require function value evaluations. The trust-region radius is adaptively adjusted based on a radius-control parameter and the stochastic gradient estimate. Under mild assumptions, we establish that the algorithm converges in expectation to a first-order stationary point. Moreover, the method achieves iteration and sample complexity bounds that match those of SVRG-based first-order methods, while allowing stochastic and potentially gradient-dependent second-order information. Extensive numerical experiments demonstrate that incorporating SVRG accelerates convergence, and that the use of trust-region methods and Hessian information further improves performance. We also highlight the impact of batch size and inner-loop length on efficiency, and show that the proposed method outperforms SGD and Adam on several machine learning tasks.
BibTeX
@misc{fang2026trsvr,
author = {Yuchen Fang and Xinshou Zheng and Javad Lavaei},
title = {{TRSVR: An Adaptive Stochastic Trust-Region Method with Variance Reduction}},
year = {2026},
eprint = {2601.14647},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2601.14647},
}Loading related content...