Robust Curvature Estimation for Second Order Optimization

This direction is split into other directions: inversion-free preconditioning, numerical instability in second-order optimization. Its last version stays readable and citable here.

Version 1 of 167 papers71 conceptsfrozen 2026-10-04

First-order optimizers like SGD and Adam are the workhorses of modern machine learning because they are cheap and reliable. However, they are "blind" to the landscape's curvature, often oscillating in steep ravines or crawling through flat plateaus. Second-order methods, which use the Hessian matrix (the second derivative of the loss), promise to find the optimum in fewer steps by warping the search space to be more spherical.

Despite this promise, second-order methods are notoriously fragile. In practice, the curvature information we rely on is often "broken"—mathematically singular, numerically unstable, or computationally ruinous. Recent research focuses on rehabilitating these methods by making curvature estimation robust enough for the rigors of deep learning and distributed systems.

The Singularity Bottleneck

The fundamental operation in second-order optimization is preconditioning: multiplying the gradient by the inverse of a curvature matrix (like the Hessian or the Fisher Information Matrix). For this to work, the matrix must be invertible and positive-definite.

In complex models, this is rarely guaranteed. As Christopher Nemeth notes in Parameter estimation for state space models using sequential Monte Carlo algorithms, estimating the information matrix for second-order steps often yields results that aren't positive-definite, causing the optimization to stall or diverge. This "singularity" problem is particularly acute in distributed settings. Emil Klintberg and Attila Kozma have shown that when local inequality constraints are active, the dual Hessian becomes non-invertible, leading to numerical failure in distributed quadratic programming.

To survive these "mathematical dead-ends," researchers like Zeyu Fang suggest incorporating slack variables or error margins, while Kozma proposes $L_2$ relaxation strategies to prevent the dual Hessian from collapsing into a singular state.

The Precision Gap

Even when a Hessian is theoretically invertible, it may be "ill-conditioned"—meaning its eigenvalues span many orders of magnitude. In the world of modern hardware, this is a death sentence.

Deep learning has shifted toward low-precision arithmetic (FP16 or BF16) to increase throughput. However, second-order statistics are hypersensitive to rounding errors. Rohan Anil and colleagues, in their work on the Shampoo optimizer, observed that curvature statistics often require 64-bit double precision to prevent model divergence. This creates a "precision tax" that negates the speed benefits of second-order methods.

To bridge this gap, Wu Lin and Felix Dangel introduced SINGD (Structured Inverse-Free Natural Gradient Descent). Instead of performing unstable inversions in low precision, SINGD performs updates in a "matrix logarithm space." This transformation maps the complex geometry of positive-definite matrices onto a flat Euclidean space where standard arithmetic is much safer. Similarly, Giuseppe Carrino and Theo Mary have developed "frugal" Newton methods that strategically mix precisions—using high precision for gradients but allowing the linear solvers to operate in low precision.

Inverse-Free and Structured Approximations

If inverting a matrix is the source of instability, the most robust solution is to stop inverting it.

Recent work has moved toward "inverse-free" updates. Jonathan Mei and Luke Walters developed KrADagrad, which uses a Kronecker factorization to update the inverse of the preconditioner directly. By avoiding the inversion step entirely, the algorithm remains stable even in standard 32-bit precision.

Another path to robustness is through dimensionality reduction. The Nyström method, which approximates a large matrix using a small subset of its columns, has emerged as a powerful tool for scaling curvature. Hyunsuk Ko introduced NysAct, which uses this approximation to estimate the "shape" of the loss landscape for Transformers. This provides the benefits of second-order convergence without the exploding gradients typically caused by ill-conditioned full-rank matrices.

Stabilizing Hierarchical Problems

The challenge of unstable curvature is most visible in Bilevel Optimization—problems where one optimization is nested inside another (e.g., meta-learning or hyperparameter tuning). Here, the "hypergradient" of the outer loop depends on the inverse Hessian of the inner loop.

If the inner problem is non-convex or over-parameterized, that Hessian is almost certainly singular. Feihu Huang points out that this makes standard gradient-based approaches mathematically undefined. To solve this, Michael Arbel proposed using "selection maps" from Morse theory to handle degenerate critical points, while Arjun Prakash and Denizalp Goktas utilize Nyström hypergradients to stabilize the actor-critic training process in reinforcement learning.

Why It Works: The Intuition of Damping

Across all these papers, a recurring theme is damping or regularization. When we encounter a singular Hessian, we are essentially seeing a direction in the loss landscape that is perfectly flat. Trying to "correct" for flatness by inverting it leads to an infinite step size (an exploding gradient).

By adding a small constant to the diagonal of the curvature matrix—a technique discussed by Katharina Ott and Felix Petersen—we effectively tell the optimizer: "If a direction looks perfectly flat, treat it as slightly curved." This simple nudge ensures the matrix remains invertible and the updates remain bounded, turning a fragile mathematical ideal into a robust numerical tool.

Papers in this version

Showing 60 of 67 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_curvature_estimation_for_second_o_ffc6c971,
  title = {Robust Curvature Estimation for Second Order Optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-curvature-estimation-for-second-order-optimization/rdr_ffc6c971420c4893b8a9f212355ff2dc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_curvature_estimation_for_second_o_ffc6c971_v1,
  title = {Robust Curvature Estimation for Second Order Optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-curvature-estimation-for-second-order-optimization/rdr_ffc6c971420c4893b8a9f212355ff2dc/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}