No article text was recorded for this version.
large-batch generalization gap
Version 2 of 2223 papers279 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Hybrid Dual-Batch and Cyclic Progressive Learning for Efficient Distributed Training
- Blog: Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Blog: Distributed Training of Deep Neuronal Networks: Theoretical and Practical Limits of Parallel Scalability
- Blog: Train longer, generalize better: closing the generalization gap in large batch training of neural networks
- Blog: Understanding Training Efficiency of Deep Learning Recommendation Models at Scale
- Blog: Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Blog: Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
- Blog: Large Batch Training of Convolutional Networks
- Blog: Contrastive Weight Regularization for Large Minibatch SGD
- Blog: AAAI New Faculty Highlights: General and Scalable Optimization for Robust AI
- Blog: A New Perspective for Understanding Generalization Gap of Deep Neural Networks Trained with Large Batch Sizes
- Blog: Implicit Gradient Alignment in Distributed and Federated Learning
- Blog: On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
- Blog: Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training
- Blog: ImageNet Training in Minutes
- Blog: Extrapolation for Large-batch Training in Deep Learning
- Blog: Extrapolation for Large-batch Training in Deep Learning
- Blog: Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
- Blog: Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks.
- Blog: FPDeep: Scalable Acceleration of CNN Training on Deeply-Pipelined FPGA Clusters
- Blog: Don't Decay the Learning Rate, Increase the Batch Size
- Blog: Stable Anderson Acceleration for Deep Learning
- Blog: CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers
- Blog: The Impact of the Mini-batch Size on the Variance of Gradients in Stochastic Gradient Descent
- Blog: Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training
- Blog: Scalable and Practical Natural Gradient for Large-Scale Deep Learning
- Blog: Cyclic Block Coordinate Descent With Variance Reduction for Composite Nonconvex Optimization
- Blog: Revisiting Small Batch Training for Deep Neural Networks
- Blog: Distributed Training of Deep Neural Networks: Theoretical and Practical Limits of Parallel Scalability
- Blog: Achieving Small-Batch Accuracy with Large-Batch Scalability via Adaptive Learning Rate Adjustment
- Blog: Accelerated Parallelizable Neural Network Learning Algorithm for Speech Recognition
- Blog: Stochastic normalized gradient descent with momentum for large-batch training
- Blog: Distributed Training and Optimization Of Neural Networks
- Blog: Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
- Blog: Accumulated Decoupled Learning with Gradient Staleness Mitigation for Convolutional Neural Networks
- Blog: Optimizing Deeper Transformers on Small Datasets
- Blog: Democratizing Production-Scale Distributed Deep Learning
- Blog: Scaling SGD Batch Size to 32K for ImageNet Training
- Blog: MegDet: A Large Mini-Batch Object Detector
- Blog: Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
- Blog: Gradient Diversity: a Key Ingredient for Scalable Distributed Learning
- Blog: BN-invariant Sharpness Regularizes the Training Model to Better Generalization
- Blog: Towards Theoretical Understanding of Large Batch Training in Stochastic Gradient Descent
- Blog: Two Tiered Distributed Training Algorithm for Acoustic Modeling
- Blog: Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model
- Blog: Train longer, generalize better: closing the generalization gap in large batch training of neural networks
- Blog: Backdrop: Stochastic Backpropagation
- Blog: Local SGD Converges Fast and Communicates Little
- Blog: Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
- Blog: CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU
- Blog: Compressed Communication for Distributed Training: Adaptive Methods and System
- Blog: Three Mechanisms of Weight Decay Regularization
- Blog: The Effect of Network Width on the Performance of Large-batch Training
- Blog: Sparse Communication for Training Deep Networks
- Blog: On the impact of larger batch size in the training of Physics Informed Neural Networks
- Blog: AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
- Blog: Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism
- Blog: Stability and Generalization of Learning Algorithms that Converge to Global Optima
- Blog: Decentralized SGD and Average-direction SAM are Asymptotically Equivalent
- Blog: Understanding the effects of data parallelism and sparsity on neural network training
Showing 60 of 223 papers.
History & citation
Current · Permanent link
- Flatness Regularization for Large Batch Training merged into this direction (67% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_large_batch_generalization_gap_ca5eddb5,
title = {large-batch generalization gap},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/large-batch-generalization-gap/rdr_ca5eddb58cd04b299c3e7c511642cd4b},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_large_batch_generalization_gap_ca5eddb5_v2,
title = {large-batch generalization gap},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/large-batch-generalization-gap/rdr_ca5eddb58cd04b299c3e7c511642cd4b/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}