No article text was recorded for this version.
premature convergence (learning rate decay threshold)
Version 1 of 127 papers39 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Why ADAGRAD Fails for Online Topic Modeling
- Blog: On the Limits of Momentum in Decentralized and Federated Optimization
- Blog: Appropriate Learning Rates of Adaptive Learning Rate Optimization Algorithms for Training Deep Neural Networks
- Blog: Improved End-to-End Speech Recognition Using Adaptive Per-Dimensional Learning Rate Methods
- Blog: Note on Learning Rate Schedules for Stochastic Optimization
- Blog: Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning rates
- Blog: On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms
- Blog: On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems
- Blog: Understanding Short-Horizon Bias in Stochastic Meta-Optimization
- Blog: On the Almost Sure Convergence of the Stochastic Three Points Algorithm
- Blog: Fundamental Convergence Analysis of Sharpness-Aware Minimization
- Blog: Bootstrap Generalization Ability from Loss Landscape Perspective
- Blog: Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
- Blog: On the resolution of misspecified convex optimization and monotone\n variational inequality problems
- Blog: Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
- Blog: Towards Flatter Loss Surface via Nonmonotonic Learning Rate Scheduling
- Blog: Stronger Baselines for Trustable Results in Neural Machine Translation
- Blog: Training Deep Neural Networks Using Conjugate Gradient-like Methods
- Blog: Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
- Blog: Towards Faster Stochastic Gradient Search
- Blog: Decoupling Learning and Decision-Making: Breaking the $\mathcal{O}(\sqrt{T})$ Barrier in Online Resource Allocation with First-Order Methods
- Blog: Selecting and Composing Learning Rate Policies for Deep Neural Networks
- Blog: How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
- Blog: On the Convergence of Adam and Beyond
- Blog: Follow the Moving Leader in Deep Learning
- Blog: AMH: AdaGrad-Momentum Hybrid for Robust Training on Small Coresets
- Blog: Escaping the Gravitational Pull of Softmax
History & citation
Current · Permanent link
- This direction split from The Search Then Converge Paradigm for Stochastic Optimization (78% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_premature_convergence_learning_rate_deca_69ba1cfb,
title = {premature convergence (learning rate decay threshold)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/premature-convergence-learning-rate-decay-threshold/rdr_69ba1cfb92794e369c4a401219a65ec5},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_premature_convergence_learning_rate_deca_69ba1cfb_v1,
title = {premature convergence (learning rate decay threshold)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/premature-convergence-learning-rate-decay-threshold/rdr_69ba1cfb92794e369c4a401219a65ec5/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}