No article text was recorded for this version.
staleness-aware step-size scaling
Version 2 of 2130 papers201 conceptsfrozen 2026-10-04
Papers in this version
- Blog: ASAP.SGD: Instance-based Adaptiveness to Staleness in Asynchronous SGD
- Blog: Delay-Tolerant Algorithms for Asynchronous Distributed Online Learning
- Blog: At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?
- Blog: FADAS: Towards Federated Adaptive Asynchronous Optimization
- Blog: Delay-Adaptive Step-sizes for Asynchronous Learning
- Blog: Distributed SGD Generalizes Well Under Asynchrony
- Blog: Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problems
- Blog: Asynchronous Byzantine Machine Learning (the case of SGD)
- Blog: Momentum Centering and Asynchronous Update for Adaptive Gradient Methods
- Blog: Staleness-Aware Async-SGD for Distributed Deep Learning
- Blog: FADAS: Towards Federated Adaptive Asynchronous Optimization
- Blog: Delay-Adaptive Step-sizes for Asynchronous Learning
- Blog: Distributed Asynchronous Optimization with Unbounded Delays: How Slow Can You Go?
- Blog: Asynchronous Stochastic Gradient Descent for Extreme-Scale Recommender Systems
- Blog: AdaDelay: Delay Adaptive Distributed Stochastic Optimization
- Blog: Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
- Blog: A Distributed, Asynchronous and Incremental Algorithm for Nonconvex Optimization: An ADMM Based Approach
- Blog: Model Accuracy and Runtime Tradeoff in Distributed Deep Learning: A Systematic Study
- Blog: The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory
- Blog: Asynchronous Distributed ADMM for Large-Scale Optimization- Part II: Linear Convergence Analysis and Numerical Performance
- Blog: Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD
- Blog: Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling
- Blog: Stability-Based Generalization Analysis of the Asynchronous Decentralized SGD
- Blog: Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Sampling
- Blog: Toward Understanding the Impact of Staleness in Distributed Machine Learning
- Blog: Asynchronous Stochastic Optimization Robust to Arbitrary Delays
- Blog: Amortized Analysis of Asynchronous Price Dynamics
- Blog: Distributed stochastic optimization with large delays
- Blog: Delay-Adaptive Distributed Stochastic Optimization
- Blog: AdaDelay: Delay Adaptive Distributed Stochastic Convex Optimization
- Blog: Asynchronous Heterogeneous Linear Quadratic Regulator Design
- Blog: Asynchronous Multi-Task Learning
- Blog: SLAMB: Accelerated Large Batch Training with Sparse Communication
- Blog: Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays
- Blog: PipeMare: Asynchronous Pipeline Parallel DNN Training
- Blog: Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis
- Blog: Asynchronous Stochastic Optimization Robust to Arbitrary Delays
- Blog: Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD
- Blog: Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning
- Blog: Delay-Tolerant Online Convex Optimization: Unified Analysis and Adaptive-Gradient Algorithms
- Blog: Asynchronous Stochastic Gradient Descent with Delay Compensation
- Blog: Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning
- Blog: Delay-adaptive step-sizes for asynchronous learning
- Blog: Optimal convergence rates of totally asynchronous optimization
- Blog: Distributed SGD Generalizes Well Under Asynchrony
- Blog: PipeMare: Asynchronous Pipeline Parallel DNN Training
- Blog: Ordered Momentum for Asynchronous SGD
- Blog: Distributed Proximal Gradient Algorithm for Partially Asynchronous Computer Clusters
- Blog: Distributed deep learning on edge-devices: feasibility via adaptive compression
- Blog: Developing a Loss Prediction-based Asynchronous Stochastic Gradient Descent Algorithm for Distributed Training of Deep Neural Networks
- Blog: FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction
- Blog: A Block-wise, Asynchronous and Distributed ADMM Algorithm for General Form Consensus Optimization
- Blog: Minimum latency training of sequence transducers for streaming end-to-end speech recognition
- Blog: Large Scale Distributed Deep Networks
- Blog: Towards Understanding Acceleration Tradeoff between Momentum and Asynchrony in Nonconvex Stochastic Optimization
- Blog: AsySPA: An Exact Asynchronous Algorithm for Convex Optimization Over Digraphs
- Blog: Asynchronous Coordinate Descent under More Realistic Assumptions
- Blog: FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction
- Blog: One-Step Gradient Delay is Not a Barrier for Large-Scale Asynchronous Pipeline Parallel LLM Pretraining
- Blog: Stochastic Gradient Descent under Markovian Sampling Schemes
Showing 60 of 130 papers.
History & citation
Current · Permanent link
- delay-variable dependency split from this direction (50% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_staleness_aware_step_size_scaling_10d105ee,
title = {staleness-aware step-size scaling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/staleness-aware-step-size-scaling/rdr_10d105ee1a6b40ee9e5e41512d56f3f5},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-18},
}This exact version (version 2, frozen text):
@misc{lacuna_staleness_aware_step_size_scaling_10d105ee_v2,
title = {staleness-aware step-size scaling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/staleness-aware-step-size-scaling/rdr_10d105ee1a6b40ee9e5e41512d56f3f5/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-18},
}