No article text was recorded for this version.
gradient normalization
Version 1 of 1191 papers231 conceptsfrozen 2026-10-04
Papers in this version
- Blog: On Robustness of the Normalized Subgradient Method with Randomly Corrupted Subgradients
- Blog: Delving into the Convergence of Generalized Smooth Minimax Optimization
- Blog: AdaNorm: Adaptive Gradient Norm Correction Based Optimizer for CNNs
- Blog: Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization
- Blog: Stochastic normalized gradient descent with momentum for large-batch training
- Blog: Stochastic Normalized Gradient Descent with Momentum for Large Batch Training
- Blog: Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization
- Blog: Robust Training of Neural Networks Using Scale Invariant Architectures
- Blog: Robustness to Unbounded Smoothness of Generalized SignSGD
- Blog: A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
- Blog: Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
- Blog: Beyond Convexity: Stochastic Quasi-Convex Optimization
- Blog: Beyond Convexity: Stochastic Quasi-Convex Optimization
- Blog: ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate
- Blog: Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
- Blog: Generalized Smooth Bilevel Optimization with Nonconvex Lower-Level
- Blog: Optimizing Shallow Networks for Binary Classification
- Blog: Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks
- Blog: Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum
- Blog: Robust Distributed Optimization With Randomly Corrupted Gradients
- Blog: Online Learning with Non-Convex Losses and Non-Stationary Regret
- Blog: Complexities in Projection-Free Stochastic Non-convex Minimization
- Blog: High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
- Blog: Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization
- Blog: Decentralized Stochastic Nonconvex Optimization under the Relaxed Smoothness
- Blog: High-Performance Large-Scale Image Recognition Without Normalization
- Blog: Parameter-Agnostic Optimization under Relaxed Smoothness
- Blog: Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization
- Blog: High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
- Blog: Error Feedback under (L0,L1)-Smoothness: Normalization and Momentum
- Blog: Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
- Blog: AMH: AdaGrad-Momentum Hybrid for Robust Training on Small Coresets
- Blog: Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods
- Blog: Robust Training of Neural Networks Using Scale Invariant Architectures
- Blog: Learning Set Functions with Implicit Differentiation
- Blog: Distributed Online Non-convex Optimization with Composite Regret
- Blog: Diffusion Visual Counterfactual Explanations
- Blog: Extending AdamW by Leveraging Its Second Moment and Magnitude
- Blog: Improving Explicit Dynamic Gaussian Splatting Optimization via Update Mixture
- Blog: From Gradient Clipping to Normalization for Heavy Tailed SGD
- Blog: Learning Gradient Descent: Better Generalization and Longer Horizons
- Blog: On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
- Blog: Scalable Planning with Tensorflow for Hybrid Nonlinear Domains
- Blog: Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
- Blog: Second-order Information in First-order Optimization Methods
- Blog: Variants of SGD for Lipschitz Continuous Loss Functions in Low-Precision Environments
- Blog: SFace: Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition
- Blog: Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
- Blog: On the difficulty of training Recurrent Neural Networks
- Blog: Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation
- Blog: Self-supervised Learning of Latent Space Dynamics
- Blog: A Stable Whitening Optimizer for Efficient Neural Network Training
- Blog: Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima
- Blog: Training Neural Networks for and by Interpolation
- Blog: Robustness to Unbounded Smoothness of Generalized SignSGD
- Blog: LUSD: Localized Update Score Distillation for Text-Guided Image Editing
- Blog: GeoGuide: Geometric guidance of diffusion models
- Blog: Tuning-Free Decentralized Nonconvex Stochastic Optimization
- Blog: Removing Batch Normalization Boosts Adversarial Training
- Blog: Cakewalk Sampling
Showing 60 of 191 papers.
History & citation
Current · Permanent link
- Optimization via Gradient Direction Normalization merged into this direction (55% overlap)
- Gradient Normalization for Heavy-Tailed and Non-Smooth Optimization merged into this direction (63% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_gradient_normalization_6f94fc95,
title = {gradient normalization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/gradient-normalization/rdr_6f94fc955636431f930984e254ca5d9e},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 1, frozen text):
@misc{lacuna_gradient_normalization_6f94fc95_v1,
title = {gradient normalization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/gradient-normalization/rdr_6f94fc955636431f930984e254ca5d9e/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}