No article text was recorded for this version.
critical batch size
Version 1 of 131 papers43 conceptsfrozen 2026-10-04
Papers in this version
- Blog: The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning
- Blog: Parallelizing Stochastic Gradient Descent for Least Squares Regression: Mini-batching, Averaging, and Model Misspecification
- Blog: Online Learning and Information Exponents: The Importance of Batch size & Time/Complexity Tradeoffs
- Blog: Model Accuracy and Runtime Tradeoff in Distributed Deep Learning: A Systematic Study
- Blog: Don't Decay the Learning Rate, Increase the Batch Size
- Blog: Measuring the Effects of Data Parallelism on Neural Network Training
- Blog: Seesaw: Accelerating Training by Balancing Batch Size and Learning Rate Scheduling
- Blog: Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
- Blog: Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates
- Blog: Deep Learning at Scale and at Ease
- Blog: Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization
- Blog: Large Scale Language Modeling: Converging on 40GB of Text in Four Hours
- Blog: The Effect of Network Width on the Performance of Large-batch Training
- Blog: Understanding the effects of data parallelism and sparsity on neural network training
- Blog: Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model
- Blog: Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
- Blog: Ellipsoid method for convex stochastic optimization in small dimension
- Blog: Deep, Deep Learning with BART
- Blog: Large scale distributed neural network training through online distillation
- Blog: Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training
- Blog: Method with batching for stochastic finite-sum variational inequalities in non-Euclidean setting
- Blog: Optimal Mini-Batch Size Selection for Fast Gradient Descent
- Blog: MuLoCo: Muon is a Practical Inner Optimizer for DiLoCo
- Blog: Better Mini-Batch Algorithms via Accelerated Gradient Methods
- Blog: A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting
- Blog: An Even More Optimal Stochastic Optimization Algorithm: Minibatching and Interpolation Learning
- Blog: Stochastic Nonconvex Optimization with Large Minibatches
- Blog: 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs
- Blog: Distributed Second-Order Optimization using Kronecker-Factored Approximations
- Blog: Papaya: Practical, Private, and Scalable Federated Learning
- Blog: A Simple Practical Accelerated Method for Finite Sums
History & citation
Current · Permanent link
- This direction split from The Efficiency Wall of Large Batch Training (87% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_critical_batch_size_265eb6d4,
title = {critical batch size},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/critical-batch-size/rdr_265eb6d480ed40f182a4fe5ba2f3089d},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 1, frozen text):
@misc{lacuna_critical_batch_size_265eb6d4_v1,
title = {critical batch size},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/critical-batch-size/rdr_265eb6d480ed40f182a4fe5ba2f3089d/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}