A smooth basis for atomistic machine learning

Filippo Bigi, Kevin Huguenin-Dumittan, Michele Ceriotti, David E. ManolopoulosThe Journal of Chemical Physics 2022 · DOI · OpenReview

TL;DR

This paper introduces a new "Laplacian eigenstate" (LE) basis for representing atomic environments in machine learning. This LE basis, derived from solving a specific mathematical problem, leads to smoother representations and demonstrably improves the accuracy of machine learning models for predicting material properties, outperforming existing methods.

What Problem Does This Paper Address?

Machine learning models for materials science often rely on describing the local atomic environment around a central atom. This involves representing the density of neighboring atoms using a set of mathematical functions, called a basis set. While the angular part of this representation is well-handled by spherical harmonics, choosing the best radial basis functions has been an open question. Existing approaches often prioritize simplicity or computational efficiency, but this paper argues that the smoothness of the radial basis functions is a crucial, often overlooked, factor for building accurate and robust machine learning models. The challenge is to find a principled way to construct such a smooth basis that also performs well across various metrics and downstream tasks.

Methods

The core contribution is the introduction of the Laplacian eigenstate (LE) basis. This basis is derived by solving the Laplacian eigenvalue problem within a sphere around the atom of interest, subject to boundary conditions. This mathematical approach naturally yields basis functions with controllable smoothness. The authors demonstrate that this LE basis, when truncated based on its eigenvalues, provides a smoother representation than many existing methods. They also extend this concept to higher-order correlations of atomic densities. Furthermore, they propose unsupervised metrics (residual variance, Jacobian variance, and Jacobian condition number) to quantitatively assess the quality of different basis sets.

Datasets

The authors evaluated their methods on:

  • Methane datasets: Both a "random methane" dataset and a "low-energy methane" subset were used. The low-energy subset was specifically curated to represent more structured configurations.
  • AIRSS carbon dataset: This dataset represents condensed-phase carbon structures.
  • rMD17 dataset: This dataset, commonly used for benchmarking, was employed for learning interatomic potentials.

No new benchmarks were introduced; instead, existing datasets were used to compare the performance of the proposed LE basis against established methods.

Key Insights and Ideas

  • Smoothness is Key: The paper strongly advocates for "smoothness" as a critical property of basis functions for atomistic machine learning. This smoothness leads to better interpolation and extrapolation capabilities for ML models.
  • Principled Basis Construction: The LE basis offers a mathematically grounded approach to constructing smooth radial basis functions, moving beyond heuristic choices.
  • Laplacian Eigenstates as Optimal Smoothness: Solving the Laplacian eigenvalue problem within a sphere provides a natural way to generate basis functions that are maximally smooth in that region.
  • Trade-offs in Optimization: Data-driven bases optimized for specific metrics (like residual variance) can sometimes lead to poorer performance on other metrics (like Jacobian condition number), suggesting that a balance is needed.
  • Smoothness vs. Data-Driven Optimization: While data-driven bases can achieve excellent performance on specific metrics, the LE basis, due to its inherent smoothness, often performs comparably or better, especially when considering robustness and generalization.
  • Higher-Order Correlations Benefit from Smoothness: The advantages of the LE basis extend to higher-order atomic density correlations, crucial for accurate potential energy models.

Key Results

  • Improved Performance on Metrics: The LE basis consistently performed well on unsupervised metrics, often outperforming widely used basis sets and being competitive with data-driven ones. Notably, it showed superior performance on the Jacobian condition number test, indicating a more uniform and less distorted representation.
  • Competitive Regression Performance: When used to build models for predicting potential energies, the LE basis, especially when combined with target-oriented heuristics, achieved equal or improved regression performance compared to data-driven and heuristic bases.
  • Smoothness Aids Generalization: The LE basis demonstrated better performance on learning tasks, particularly for the methane datasets, suggesting that its inherent smoothness aids in generalizing to unseen configurations.
  • Efficiency Gains: The LE basis, with its eigenvalue-based truncation, offers a more efficient way to control the complexity of the representation compared to some data-driven approaches.

Limitations and Future Work

  • Data-Driven Optimization Trade-offs: The paper highlights that bases optimized for specific metrics (like residual variance) might not perform as well on others (like Jacobian condition number), suggesting that a balance is needed.
  • Generalization to Higher Orders: While the LE basis concept is extended to higher-order correlations, the computational cost of these higher-order features can still be a challenge. The authors suggest that aggressive truncation strategies might be necessary for larger v.
  • Parameter Optimization: For specific applications like learning interatomic potentials, the optimal choice of truncation parameters (e.g., $E_{max}(v)$) might need to be adapted to the training set size and the specific chemical system. The paper proposes a scaling argument for this adaptation.
  • Future Work: The authors suggest that incorporating the principles of the LE basis into existing equivariant neural network architectures could lead to further improvements. They also point to the potential of adapting truncation parameters based on the training data size for improved performance.

Summary updated

Abstract

Machine learning frameworks based on correlations of interatomic positions begin with a discretized description of the density of other atoms in the neighbourhood of each atom in the system. Symmetry considerations support the use of spherical harmonics to expand the angular dependence of this density, but there is as yet no clear rationale to choose one radial basis over another. Here we investigate the basis that results from the solution of the Laplacian eigenvalue problem within a sphere around the atom of interest. We show that this generates the smoothest possible basis of a given size within the sphere, and that a tensor product of Laplacian eigenstates also provides the smoothest possible basis for expanding any higher-order correlation of the atomic density within the appropriate hypersphere. We consider several unsupervised metrics of the quality of a basis for a given dataset, and show that the Laplacian eigenstate basis has a performance that is much better than some widely used basis sets and is competitive with data-driven bases that numerically optimize each metric. In supervised machine learning tests, we find that the optimal function smoothness of the Laplacian eigenstates leads to comparable or better performance than can be obtained from a data-driven basis of a similar size that has been optimized to describe the atom-density correlation for the specific dataset. We conclude that the smoothness of the basis functions is a key and hitherto largely overlooked aspect of successful atomic density representations.
BibTeX
@article{bigi2022smooth,
  author = {Filippo Bigi and Kevin Huguenin-Dumittan and Michele Ceriotti and David E. Manolopoulos},
  title = {{A smooth basis for atomistic machine learning}},
  journal = {The Journal of Chemical Physics},
  year = {2022},
  doi = {10.1063/5.0124363},
}

Loading related content...

Figure