No article text was recorded for this version.
layer-wise adaptive optimization
Version 1 of 121 papers24 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Distributed adversarial training to robustify deep neural networks at scale
- Blog: Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
- Blog: ImageNet Training in Minutes
- Blog: Physics-inspired deep learning to characterize the signal manifold of quasi-circular, spinning, non-precessing binary black hole mergers
- Blog: On Large-Cohort Training for Federated Learning
- Blog: Enabling Differentially Private Federated Learning for Speech Recognition: Benchmarks, Adaptive Optimizers, and Gradient Clipping
- Blog: SpeedyZero: Mastering Atari with Limited Data and Time
- Blog: FORGE: Pre-Training Open Foundation Models for Science
- Blog: Comparative Study of Large Language Model Architectures on Frontier
- Blog: Deconstructing What Makes a Good Optimizer for Language Models
- Blog: Large Batch Optimization for Object Detection: Training COCO in 12 minutes
- Blog: Findings of the Association for Computational Linguistics: EMNLP 2020
- Blog: DeiT III: Revenge of the ViT
- Blog: Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
- Blog: Federated Learning with Differential Privacy for End-to-End Speech Recognition
- Blog: Importance of Smoothness Induced by Optimizers in FL4ASR: Towards Understanding Federated Learning for End-to-End ASR
- Blog: ResNet strikes back: An improved training procedure in timm
- Blog: Exascale deep learning for climate analytics
- Blog: CosmoFlow: Using Deep Learning to Learn the Universe at Scale
- Blog: The Computation of Generalized Embeddings for Underwater Acoustic Target Recognition using Contrastive Learning
- Blog: On the Local Hessian in Back-propagation
History & citation
Current · Permanent link
- This direction split from Layer-wise Adaptive Rate Scaling for Large Batch Training (81% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_layer_wise_adaptive_optimization_1f807a51,
title = {layer-wise adaptive optimization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/layer-wise-adaptive-optimization/rdr_1f807a511a3447369153c2d4f9f1f4c4},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 1, frozen text):
@misc{lacuna_layer_wise_adaptive_optimization_1f807a51_v1,
title = {layer-wise adaptive optimization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/layer-wise-adaptive-optimization/rdr_1f807a511a3447369153c2d4f9f1f4c4/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}