Algorithmic Differentiation for Automatized Modelling of Machine Learned Force Fields

Niklas Frederik Schmitz, Klaus-Robert Müller, Stefan ChmielaarXiv 2022 · DOI · OpenReview

Abstract

Reconstructing force fields (FFs) from atomistic simulation data is a challenge since accurate data can be highly expensive. Here, machine learning (ML) models can help to be data economic as they can be successfully constrained using the underlying symmetry and conservation laws of physics. However, so far, every descriptor newly proposed for an ML model has required a cumbersome and mathematically tedious remodeling. We therefore propose using modern techniques from algorithmic differentiation within the ML modeling process -- effectively enabling the usage of novel descriptors or models fully automatically at an order of magnitude higher computational efficiency. This paradigmatic approach enables not only a versatile usage of novel representations and the efficient computation of larger systems -- all of high value to the FF community -- but also the simple inclusion of further physical knowledge such as higher-order information (e.g. Hessians, more complex partial differential equations constraints etc.), even beyond the presented FF domain.

Introduction: The Mathematical Tax on Physics-Informed ML

In the fields of chemistry and physics, Force Fields (FFs) are essential for describing atomic interactions. To achieve high accuracy and data efficiency, researchers often incorporate physical laws—such as energy conservation—directly into machine learning models via differential equation (DE) constraints. For instance, a conservative force field is defined as the negative gradient of a scalar energy potential.

However, implementing these constraints typically imposes a significant "mathematical tax." First, researchers must manually derive complex multi-variable calculus expressions whenever a new molecular descriptor is introduced, a process that is both tedious and prone to error. Second, naive implementations of these constraints often lead to a "computational inflation," where memory requirements and processing times explode, making the models impractical for large-scale use. Consequently, many sophisticated physics-informed models remain theoretical or are limited to a few hand-derived cases.

Methods: Automating Constraints with Algorithmic Differentiation

The proposed framework addresses these challenges by leveraging Algorithmic Differentiation (AD)—the core technology behind backpropagation in deep learning—to automate the construction of physics-constrained models. Instead of manually deriving derivatives, the authors use AD to define the model's structure itself.

The key technical innovation lies in the use of Jacobian-vector products (JVPs) and Vector-Jacobian products (VJPs) to manage the tensor-product structure of constrained Gaussian Process (GP) kernels. By employing "early contraction" of derivatives, the framework avoids the creation of massive, dense derivative matrices. This allows for the automatic application of linear differential operators, such as Gradients, Hessians, and Laplacians, while maintaining computational efficiency comparable to unconstrained models.

Efficient contraction of derivatives using AD.

This modular approach allows researchers to "mix and match" state-of-the-art molecular descriptors (e.g., from Neural Networks like SchNet or PaiNN) with rigorous physical constraints. Furthermore, the use of AD enables gradient-based optimization of internal model hyperparameters, a task previously considered too complex for such constrained systems.

Key Insights and Results

The framework's efficiency was evaluated using standard benchmarks like the MD17 dataset and various toy partial differential equation (PDE) problems.

Massive Computational Speedups

In benchmarks involving molecules such as Aspirin and Ethanol, the AD-based contraction method achieved speedups of 30x to 60x compared to standard "dense" implementations. This efficiency allows high-quality force field modeling to be performed on a single GPU, tasks that previously required large CPU clusters.

The Power of Higher-Order Information

A major insight of this work is that AD makes the inclusion of second-order information, such as Hessians (curvature), computationally feasible. The results demonstrate that incorporating Hessians can improve reconstruction accuracy by an order of magnitude, even when working with extremely small datasets.

Impact of adding differential constraints on accuracy.

By recombining existing models—for example, pairing the sGDML model with FCHL19 kernels—the authors identified new model variants that outperformed original versions across the entire MD17 benchmark suite.

Conclusion and Future Directions

While the framework provides a significant $3N$ improvement in efficiency (where $N$ is the number of atoms), it still faces the cubic scaling challenges inherent to standard Gaussian Processes for very large systems. Future work will focus on extending this methodology to more complex PDEs and exploring hybrid models that further integrate deep learning descriptors with GP-based physical constraints. By turning physics into a "plug-and-play" component, this approach paves the way for more accessible and powerful automated modeling in the physical sciences.

Summary updated

Abstract

Reconstructing force fields (FFs) from atomistic simulation data is a challenge since accurate data can be highly expensive. Here, machine learning (ML) models can help to be data economic as they can be successfully constrained using the underlying symmetry and conservation laws of physics. However, so far, every descriptor newly proposed for an ML model has required a cumbersome and mathematically tedious remodeling. We therefore propose using modern techniques from algorithmic differentiation within the ML modeling process -- effectively enabling the usage of novel descriptors or models fully automatically at an order of magnitude higher computational efficiency. This paradigmatic approach enables not only a versatile usage of novel representations and the efficient computation of larger systems -- all of high value to the FF community -- but also the simple inclusion of further physical knowledge such as higher-order information (e.g. Hessians, more complex partial differential equations constraints etc.), even beyond the presented FF domain.
BibTeX
@misc{schmitz2022algorithmic,
  author = {Niklas Frederik Schmitz and Klaus-Robert Müller and Stefan Chmiela},
  title = {{Algorithmic Differentiation for Automatized Modelling of Machine Learned Force Fields}},
  year = {2022},
  eprint = {2208.12104},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2208.12104},
}
Figure