No article text was recorded for this version.
generalization-optimization tradeoff
Version 1 of 131 papers40 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Kronecker-factored Curvature Approximations for Recurrent Neural Networks
- Blog: Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
- Blog: Small Steps and Giant Leaps: Minimal Newton Solvers for Deep Learning
- Blog: Accelerating Gradient Boosting Machines
- Blog: Investigations on hessian-free optimization for cross-entropy training of deep neural networks
- Blog: The Three Stages of Learning Dynamics in High-dimensional Kernel Methods
- Blog: Deep learning via Hessian-free optimization
- Blog: Scaling up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix
- Blog: Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization
- Blog: Surrogate maximization/minimization algorithms and extensions
- Blog: Margin Maximizing Loss Functions
- Blog: Tighter bounds lead to improved classifiers
- Blog: Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth
- Blog: Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization
- Blog: Speedy Performance Estimation for Neural Architecture Search
- Blog: Risk Bounds of Accelerated SGD for Overparameterized Linear Regression
- Blog: Rapid Overfitting of Multi-Pass SGD in Stochastic Convex Optimization
- Blog: Boosting Data-Driven Mirror Descent with Randomization, Equivariance, and Acceleration
- Blog: ADLER - An efficient Hessian-based strategy for adaptive learning rate
- Blog: Stability and Generalization for Stochastic (Compositional) Optimizations
- Blog: A Convergence Rate Analysis for LogitBoost, MART and Their Variant
- Blog: Understanding the Role of Momentum in Stochastic Gradient Methods
- Blog: Bayesian Model Selection, the Marginal Likelihood, and Generalization
- Blog: Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
- Blog: Using Embedding Masks for Word Categorization
- Blog: Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks
- Blog: Backpropagation through the Void: Optimizing control variates for black-box gradient estimation
- Blog: Towards Generalizable 3D Human Pose Estimation via Ensembles on Flat Loss Landscapes
- Blog: Globally aware optimization with resurgence
- Blog: Deep Active Learning by Leveraging Training Dynamics
- Blog: Hot Swapping for Online Adaptation of Optimization Hyperparameters
History & citation
Current · Permanent link
- This direction split from The Generalization Cost of Efficient Optimization (55% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_generalization_optimization_tradeoff_18495949,
title = {generalization-optimization tradeoff},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/generalization-optimization-tradeoff/rdr_18495949bafc4d41b45b74b095d5f4a6},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_generalization_optimization_tradeoff_18495949_v1,
title = {generalization-optimization tradeoff},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/generalization-optimization-tradeoff/rdr_18495949bafc4d41b45b74b095d5f4a6/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}