Shoot from the HIP: Hessian Interatomic Potentials without derivatives

TL;DR

Calculating molecular Hessians (second derivatives of energy) usually requires computationally brutal automatic differentiation (AD) or quantum chemical methods. Hessian Interatomic Potentials (HIP) bypasses derivatives entirely by directly predicting $\mathrm{SE}(3)$-equivariant, symmetric molecular Hessians via spherical tensor readouts in graph neural networks. This makes Hessian computation 10–70× faster, significantly more memory-efficient, and more accurate across geometry optimization, zero-point energy estimation, and transition-state search.

What Problem Does This Paper Address?

In computational chemistry, the Hessian matrix—a $3N \times 3N$ matrix of mixed second-order energy derivatives with respect to nuclear coordinates—is vital for:

• Accelerating geometry optimization via second-order Newton/RFO methods.
• Finding transition states (first-order saddle points) to determine reaction barriers and pathways.
• Vibrational frequency analysis and zero-point energy (ZPE) thermochemical corrections.

Despite its importance, computing the Hessian is a notorious computational bottleneck. Traditional ab initio Density Functional Theory (DFT) approaches (like CPKS) scale as $\mathcal{O}(N^5)$. While machine-learned interatomic potentials (MLIPs) have drastically sped up energy and force predictions, calculating Hessians through MLIPs currently relies on automatic differentiation (AD) or finite differences. Because AD requires $3N$ separate Hessian-vector products (HVPs) to reconstruct the full matrix, it scales quadratically $\mathcal{O}(N^2)$ in cost and memory, degrades heavily under batching, and makes training models on Hessian labels roughly $25\times$ slower.

Methods

Instead of differentiating an energy function twice, HIP treats the Hessian as a direct target within an equivariant graph neural network (using EquiformerV2 as the backbone):

• Symmetry by Design: A physical Cartesian Hessian must satisfy rotation equivariance ($\mathbf{H}_{I,J} \to \mathbf{Q} \mathbf{H}_{I,J} \mathbf{Q}^\top$) and interchange symmetry ($\mathbf{H} = \mathbf{H}^\top$). HIP uses irreducible representations (irreps) up to degree $l=2$ generated during message passing.
• Hessian Prediction Head: The model takes atom-pair message features and projects them into $0e \oplus 1e \oplus 2e$ irreps via a linear layer. It then reconstructs the $3 \times 3$ Cartesian sub-blocks using Clebsch-Gordan tensor product expansion (the inverse of irreducible tensor decomposition) before applying final matrix symmetrization ($\mathbf{H} = \mathbf{H}' + \mathbf{H}'^\top$).
• Subspace Loss Function: Downstream vibrational and transition-state tasks depend predominantly on the lowest-frequency modes. To prioritize these without suffering from the numerical instabilities of backpropagating through full eigenspectrum decompositions, the authors introduce a targeted subspace loss ($\mathcal{L}_{\text{sub}}$) that aligns the predicted Hessian against the $k$ lowest ground-truth eigenvectors and eigenvalues (setting $k=8$ to cover 6 rigid-body degrees of freedom plus the two softest vibrational modes).

Datasets

The authors benchmark HIP across multiple established molecular datasets:

1. HORM (T1x and RGD1): A large-scale reactive dataset computed at the $\omega\text{B97X/6-31G}^*$ DFT level. Training used ~1.725M configurations from Transition1x (T1x), evaluated on 50,844 T1x geometries and 60,000 out-of-distribution molecules from RGD1.
2. QM9 Hessian: 41,645 equilibrium configurations across four solvent environments (vacuum, water, THF, toluene) at the $\omega\text{B97X/6-31G}^*$ level.
3. ReactBench: Downstream benchmark assessing transition state discovery (combining growing string methods, rational function optimization, and intrinsic reaction coordinate verification).

Key Insights and Ideas

• Direct Prediction is Safe for Hessians: Direct (non-conservative) force prediction is often avoided in molecular dynamics because non-integrable errors cause artificial energy drift over long trajectories. However, no mainstream chemical workflow integrates Hessians over long trajectories. For single-point tasks (vibrations, ZPE) or optimization preconditioning, integrability errors act like ordinary approximation errors, making direct regression safe and practical.
• Dramatic Sparsity and Parallelization Gains: Because HIP predicts Hessians directly through graph message passing using a local distance cutoff, compute scales as $\mathcal{O}(N)$ for large systems, completely dodging the sequential $\mathcal{O}(N^2)$ HVP loops required by AD.
• End-to-End Regularization: Training a model end-to-end on energies, forces, and direct Hessians does not degrade performance; rather, direct Hessian supervision acts as a strong regularizer that improves energy and force accuracy.

Key Results

• Inference Speed & Memory: HIP is 10–70× faster than AD for single-molecule evaluation, achieves >78× speedup under batched inference, and cuts peak memory usage by 2–3× on small molecules.
• Hessian Accuracy: On the HORM-T1x validation set, HIP-EquiformerV2 reduces Hessian MAE from $0.074$ to $0.020\ \text{eV/\AA}^2$ (over $3.5\times$ lower) and cuts eigenvalue MAE from $0.242$ to $0.041\ \text{eV/\AA}^2$ compared to AD EquiformerV2. The cosine similarity of the softest mode ($v_1$) reaches 0.982 (vs. 0.915 for AD).
• Zero-Point Energy (ZPE): Reactant ZPE error drops to $0.0004\ \text{eV}$—two orders of magnitude more accurate than AD EquiformerV2 ($0.0600\ \text{eV}$).
• Transition-State Classification: HIP correctly classifies stationary points via frequency analysis with 92% accuracy, outperforming AD baselines (71–82%).
• Optimization Efficiency: Rational function optimization (RFO) driven by HIP Hessians converges in the fewest steps and lowest total wall-clock time compared to finite-difference, AD, and BFGS baselines.

Limitations and Future Work

• Supervision Requirement: Direct regression requires explicit DFT Hessian labels for training, which are expensive to generate compared to simple energies and forces.
• Domain Scope: Experiments focused exclusively on small organic molecules (up to ~30 atoms). Future work remains to validate HIP on periodic systems, extended condensed-phase materials, and biological macromolecules like proteins.

@inproceedings{burger2025shoot,
  author = {Andreas Burger and Luca Thiede and Nikolaj Rønne and Nandita Vijaykumar and Tejs Vegge and Arghya Bhowmik and Alan Aspuru-Guzik},
  title = {{Shoot from the HIP: Hessian Interatomic Potentials without derivatives}},
  booktitle = {SIMBIOCHEM},
  year = {2025},
  url = {https://openreview.net/forum?id=VbAGJ6vh3k},
}