Exploring the Design Space of Machine Learning Models for Quantum Chemistry with a Fully Differentiable Framework

Divya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin‐Lic Chan, Michele CeriottiJournal of Chemical Theory and Computation 2025 · DOI

TL;DR

This paper introduces a novel, fully differentiable framework that integrates machine learning (ML) models with quantum chemistry (QM) calculations. This allows ML models to learn effective Hamiltonians, which can then be used to predict a wide range of molecular properties, offering improved accuracy and transferability compared to traditional property-specific ML models.

What Problem Does This Paper Address?

Traditional machine learning models in quantum chemistry often act as direct "surrogates" for specific properties like dipole moments or energies. While effective for the properties they're trained on, these models struggle to generalize to new situations or predict properties they weren't explicitly trained for. This limitation arises because they don't capture the underlying physics of the electronic structure. To overcome this, researchers are exploring ML models that predict more fundamental "ingredients" of QM calculations, such as the electron density or the effective single-particle Hamiltonian. The challenge lies in effectively training these models and integrating them into existing QM workflows to leverage their predictive power for multiple properties. This paper addresses this by developing a framework that allows ML models to predict an effective Hamiltonian, which is then used within a differentiable QM workflow (PySCFAD) to compute various properties.

Methods

The core of this work is a framework that seamlessly integrates ML models with the PySCFAD (a differentiable QM workflow) library. Instead of directly predicting properties, the ML model learns to predict the matrix elements of an effective single-particle Hamiltonian. This Hamiltonian is then fed into PySCFAD, which performs the necessary QM operations (like diagonalization) to compute target properties. This "indirect" training approach allows the ML model to be optimized based on the accuracy of these derived properties, rather than just the Hamiltonian itself.

Key methodological aspects include:

  • Differentiable QM Workflow (PySCFAD): This is crucial as it allows for backpropagation of gradients from the target properties all the way back to the ML model's parameters. This enables efficient optimization of the ML-predicted Hamiltonian.
  • Hybrid ML/QM Models: The approach combines ML models with QM calculations, where the ML model provides an intermediate representation (the Hamiltonian) that is then processed by QM methods.
  • Exploration of Design Space: The paper systematically investigates how different choices in the ML model's design, particularly the inclusion of various QM properties as training targets (e.g., molecular orbital energies, dipole moments, polarizabilities, Mayer bond order), influence model performance and transferability.
  • Minimalistic Hamiltonian Parameterization: The ML model's Hamiltonian is parameterized in a symmetry-adapted, minimal basis (STO-3G), which is then used to predict properties that might have been computed using larger, more accurate basis sets (def2-TZVP). This explores the possibility of achieving high accuracy with computationally cheaper models.

Datasets

  • QM7b dataset: A subset of 700 molecules composed of C, H, N, and O atoms, selected for structural diversity using farthest point sampling on three-body atom-centered descriptors.
  • QM7 and QM9 datasets: Used for evaluating model performance on both training and test sets. QM9 is a larger dataset that spans a similar compositional space.
  • Graphene dataset (from Ref. 32): Used for condensed-phase Hamiltonian predictions, specifically for band structure calculations.

Key Insights and Ideas

  • Hamiltonian as a Central Intermediate: Learning the effective Hamiltonian, rather than just individual properties, provides a physically interpretable and versatile representation that can unlock access to multiple observables.
  • Indirect Training is Powerful: Training ML models indirectly on properties derived from an ML-predicted Hamiltonian, especially when targeting properties from larger basis sets, can lead to more accurate and transferable models than direct property prediction.
  • Basis Set Mismatch Can Be Managed: Even when the ML model uses a minimal basis (STO-3G) to represent the Hamiltonian, it can learn to predict properties computed with a larger basis (def2-TZVP) with remarkable accuracy, suggesting the ML model implicitly learns to compensate for basis set incompleteness.
  • Physics-Based Constraints Improve Transferability: Incorporating more QM-derived properties as training targets generally improves model accuracy and, crucially, enhances the model's ability to extrapolate to larger or more complex molecules where traditional property-specific models fail.
  • Differentiable QM is Key: The seamless integration with differentiable QM workflows like PySCFAD is essential for enabling this indirect training paradigm and exploring the ML/QM design space effectively.

Key Results

  • Improved Accuracy for Response Properties: The hybrid ML/QM models, especially when trained on multiple targets (ε, μ, α, B), significantly improve the prediction accuracy for response properties like polarizability, often by orders of magnitude compared to the basis set error.
  • Comparable Accuracy to Larger Basis Sets: Upscaled indirect models, which predict properties from larger basis sets using a minimal basis Hamiltonian, achieve accuracy comparable to that of their large-basis counterparts, while remaining computationally more efficient.
  • Superior Transferability: The Hamiltonian-based models demonstrate superior transferability to larger molecules and complex systems (e.g., polyenes, polyacenes, polyenoic amino acids) where traditional property-specific models struggle due to their local nature and inability to capture non-local effects like electron delocalization.
  • Condensed-Phase Success: The framework successfully predicts band structures for graphene using a minimal basis representation, achieving an MAE of 252 meV on occupied band energies, demonstrating its applicability beyond molecular systems.

Limitations and Future Work

  • Model Basis Size: While the models can compensate for basis set differences, the size of the ML model's Hamiltonian basis is still a factor that influences accuracy, particularly for properties sensitive to excited states. The authors suggest this should be considered a hyperparameter.
  • ML Model Architecture: The paper primarily uses simple linear models to focus on the QM design choices. More sophisticated ML architectures (e.g., graph neural networks) could further enhance accuracy and expressivity.
  • Holistic Optimization: The authors emphasize the need for a holistic approach, optimizing geometric features, ML architecture, and QM approximations together.
  • Future Work: The framework is extensible to more complex ML architectures and can be applied to a wider range of QM approximations and physical observables. The authors suggest further exploration of the design space, including different QM levels of theory and basis sets.

Summary updated

Abstract

Traditional atomistic machine learning (ML) models serve as surrogates for quantum mechanical (QM) properties, predicting quantities such as dipole moments and polarizabilities directly from compositions and geometries of atomic configurations. With the emergence of ML approaches to predict the "ingredients" of a QM calculation, such as the ground-state charge density or the effective single-particle Hamiltonian, it has become possible to obtain multiple properties through analytical physics-based operations on these intermediate ML predictions. We present a framework that seamlessly integrates the prediction of an effective electronic Hamiltonian, for both molecular and condensed-phase systems, with PySCFAD, a differentiable QM workflow. This integration facilitates training models indirectly against functions of the Hamiltonian, such as electronic energy levels, dipole moments, polarizability, etc. We then use this framework to explore various possible choices within the design space of hybrid ML/QM models, examining the influence of incorporating multiple targets on model performance and learning a reduced-basis ML Hamiltonian that can reproduce targets computed on a much larger basis. Our benchmarks evaluate the accuracy and transferability of these hybrid models, compare them against predictions of atomic properties from their surrogate models, and provide indications to guide the design of the interface between the ML and QM components of the model.
BibTeX
@article{suman2025exploring,
  author = {Divya Suman and Jigyasa Nigam and Sandra Saade and Paolo Pegolo and Hanna Türk and Xing Zhang and Garnet Kin‐Lic Chan and Michele Ceriotti},
  title = {{Exploring the Design Space of Machine Learning Models for Quantum Chemistry with a Fully Differentiable Framework}},
  journal = {Journal of Chemical Theory and Computation},
  year = {2025},
  doi = {10.1021/acs.jctc.5c00522},
}
Figure