No article text was recorded for this version.
hyperparameter sensitivity (second-order hyperparameter sensitivity)
Version 2 of 282 papers82 conceptsfrozen 2026-10-04
Papers in this version
- Blog: LEAD: Least-Action Dynamics for Min-Max Optimization
- Blog: Practical Quasi-Newton Methods for Training Deep Neural Networks
- Blog: Second-Order Optimization with Lazy Hessians
- Blog: A Decentralized Second-Order Method with Exact Linear Convergence Rate for Consensus Optimization
- Blog: mL-BFGS: A Momentum-based L-BFGS for Distributed Large-scale Neural Network Optimization
- Blog: Modular Block-diagonal Curvature Approximations for Feedforward Architectures
- Blog: Topmoumoute Online Natural Gradient Algorithm
- Blog: Graph-Coupled Oscillator Networks
- Blog: Do Parameters Reveal More than Loss for Membership Inference?
- Blog: Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
- Blog: An Improved Empirical Fisher Approximation for Natural Gradient Descent
- Blog: A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks
- Blog: Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis
- Blog: On the importance of initialization and momentum in deep learning
- Blog: On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime
- Blog: An Evaluation of Fisher Approximations Beyond Kronecker Factorization
- Blog: Momentum Residual Neural Networks
- Blog: Inner Classifier-Free Guidance and Its Taylor Expansion for Diffusion Models
- Blog: Sequence training of DNN acoustic models with natural gradient
- Blog: SPAN: A Stochastic Projected Approximate Newton Method
- Blog: Heavy Ball Neural Ordinary Differential Equations
- Blog: HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance
- Blog: Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization
- Blog: Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
- Blog: EA-CG: An Approximate Second-Order Method for Training Fully-Connected Neural Networks
- Blog: Large Scale Empirical Risk Minimization via Truncated Adaptive Newton Method
- Blog: LEAD: Min-Max Optimization from a Physical Perspective
- Blog: Combining Natural Gradient with Hessian Free Methods for Sequence Training
- Blog: Fast Deep Hedging with Second-Order Optimization
- Blog: A Self-Correcting Variable-Metric Algorithm for Stochastic Optimization
- Blog: Why Do Attributes Propagate in Graph Convolutional Neural Networks?
- Blog: Scalable Minimum Bayes Risk Training of Deep Neural Network Acoustic Models Using Distributed Hessian-free Optimization
- Blog: Shampoo: Preconditioned Stochastic Tensor Optimization
- Blog: Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
- Blog: Fast Black-box Variational Inference through Stochastic Trust-Region Optimization
- Blog: mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization
- Blog: M-FAC: Efficient Matrix-Free Approximations of Second-Order Information
- Blog: A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning
- Blog: SKFAC: Training Neural Networks With Faster Kronecker-Factored Approximate Curvature
- Blog: Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
- Blog: Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
- Blog: Extragradient Method in Optimization: Convergence and Complexity
- Blog: Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC
- Blog: DiSCO: Distributed Optimization for Self-Concordant Empirical Loss
- Blog: BAMSProd: A Step towards Generalizing the Adaptive Optimization Methods to Deep Binary Model
- Blog: Understanding and improving Shampoo and SOAP via Kullback-Leibler Minimization
- Blog: A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract)
- Blog: Preconditioner on Matrix Lie Group for SGD
- Blog: Stochastic Cubic Regularization for Fast Nonconvex Optimization
- Blog: KrADagrad: Kronecker Approximation-Domination Gradient Preconditioned Stochastic Optimization
- Blog: Preconditioners for the Stochastic Training of Neural Fields
- Blog: Efficient Optimization Methods for Extreme Similarity Learning with Nonlinear Embeddings
- Blog: Asynchronous Training Schemes in Distributed Learning with Time Delay
- Blog: Distributed Second-Order Optimization using Kronecker-Factored Approximations
- Blog: Semistochastic Quadratic Bound Methods
- Blog: Convergence rates for the Heavy-Ball continuous dynamics for non-convex optimization, under Polyak-Łojasiewicz condition
- Blog: Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
- Blog: Deep learning via Hessian-free optimization
- Blog: A Block Coordinate Descent Optimizer for Classification Problems Exploiting Convexity
- Blog: Splitting of Composite Neural Networks via Proximal Operator With Information Bottleneck
Showing 60 of 82 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_hyperparameter_sensitivity_second_order__dbeb611f,
title = {hyperparameter sensitivity (second-order hyperparameter sensitivity)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/hyperparameter-sensitivity-second-order-hyperparameter-sensitivity/rdr_dbeb611f7444481c87279de645c564b4},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 2, frozen text):
@misc{lacuna_hyperparameter_sensitivity_second_order__dbeb611f_v2,
title = {hyperparameter sensitivity (second-order hyperparameter sensitivity)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/hyperparameter-sensitivity-second-order-hyperparameter-sensitivity/rdr_dbeb611f7444481c87279de645c564b4/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}