No article text was recorded for this version.
non-convex convergence theory
Version 1 of 1190 papers192 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
- Blog: Direct Nonlinear Acceleration
- Blog: Optimistic Online-to-Batch Conversions for Accelerated Convergence and Universality
- Blog: Improved Penalty Method via Doubly Stochastic Gradients for Bilevel Hyperparameter Optimization
- Blog: Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
- Blog: Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients
- Blog: Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization
- Blog: The Road Less Scheduled
- Blog: Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers
- Blog: Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent
- Blog: Provable Acceleration of Wide Neural Net Training via Polyak's Momentum
- Blog: GFedCL: Graph-Based Federated Continual Learning with Spatial and Temporal Awareness
- Blog: Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
- Blog: Adaptive Proximal Gradient Method for Convex Optimization
- Blog: Optimistic Online-to-Batch Conversions for Accelerated Convergence and Universality
- Blog: On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization
- Blog: AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods
- Blog: Robust Learning Rate Selection for Stochastic Optimization via Splitting Diagnostic
- Blog: Differentially Private Adaptive Optimization with Delayed Preconditioners
- Blog: Parallel training of DNNs with Natural Gradient and Parameter Averaging
- Blog: An online passive-aggressive algorithm for difference-of-squares classification
- Blog: Implicit differentiation of Lasso-type models for hyperparameter optimization
- Blog: Adaptive Conditional Gradient Sliding: Projection-Free and Line-Search-Free Acceleration
- Blog: Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
- Blog: Towards Constituting Mathematical Structures for Learning to Optimize
- Blog: Taming GANs with Lookahead-Minmax
- Blog: RSPO: Regularized Self-Play Alignment of Large Language Models
- Blog: A Computationally Efficient Sparsified Online Newton Method
- Blog: The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure
- Blog: Agnostic Active Learning Is Always Better Than Passive Learning
- Blog: Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
- Blog: A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization
- Blog: Convergence Analysis of Two-layer Neural Networks with ReLU Activation
- Blog: Learning-Rate-Free Learning by D-Adaptation
- Blog: Memory Augmented Optimizers for Deep Learning
- Blog: Adaptive Gradient Methods for Constrained Convex Optimization
- Blog: Convergence of a Stochastic Gradient Method with Momentum for Nonsmooth Nonconvex Optimization
- Blog: Revisiting Stochastic Extragradient
- Blog: Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity
- Blog: On Stochastic Gradient and Subgradient Methods with Adaptive Steplength Sequences
- Blog: Learning Robust Neural Processes with Risk-Averse Stochastic Optimization
- Blog: Exploring Fast and Communication-Efficient Algorithms in Large-Scale Distributed Networks
- Blog: Boosting Data-Driven Mirror Descent with Randomization, Equivariance, and Acceleration
- Blog: MoMo: Momentum Models for Adaptive Learning Rates
- Blog: Value Function Gradient Learning for Large-Scale Multistage Stochastic Programming Problems
- Blog: Optimal Adaptive and Accelerated Stochastic Gradient Descent
- Blog: Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
- Blog: Heavy-ball Algorithms Always Escape Saddle Points
- Blog: Practical Schemes for Finding Near-Stationary Points of Convex Finite-Sums
- Blog: Improved Algorithms for Convex-Concave Minimax Optimization
- Blog: DeltaGrad: Rapid retraining of machine learning models
- Blog: Accelerated, Optimal and Parallel: Some results on model-based stochastic optimization
- Blog: Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits
- Blog: Linear Last-iterate Convergence in Constrained Saddle-point Optimization
- Blog: ADAM Optimization with Adaptive Batch Selection
- Blog: Accelerated Convergence of Nesterov's Momentum for Deep Neural Networks under Partial Strong Convexity
- Blog: Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
- Blog: Extreme Tensoring for Low-Memory Preconditioning
- Blog: FOAM: Frequency and Operator-Error Based Adaptive Damping Method for Reducing Staleness-Oriented Error for Shampoo
- Blog: Comment on Stochastic Polyak Step-Size: Performance of ALI-G
Showing 60 of 190 papers.
History & citation
Current · Permanent link
- This direction split from Reconciling Optimization Theory with Non-Convex Loss Landscapes (83% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_non_convex_convergence_theory_762bdbbe,
title = {non-convex convergence theory},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/non-convex-convergence-theory/rdr_762bdbbe1fa146a786961343c9fa0e11},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_non_convex_convergence_theory_762bdbbe_v1,
title = {non-convex convergence theory},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/non-convex-convergence-theory/rdr_762bdbbe1fa146a786961343c9fa0e11/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}