Recursive evaluation and iterative contraction of N-body equivariant features

Jigyasa Nigam, Sergey N. Pozdnyakov, Michele CeriottiThe Journal of Chemical Physics 2020 · DOI

TL;DR

This paper introduces a new framework, NICE, for building powerful, symmetry-aware representations of atomic structures for machine learning. By recursively constructing and iteratively contracting "equivariant features," NICE efficiently generates systematically improvable representations that capture complex atomic interactions without an exponential explosion in complexity.

What Problem Does This Paper Address?

Machine learning models for materials science and chemistry often need to understand the structure of molecules and materials. A common approach is to represent these structures using features that respect fundamental symmetries like translation, rotation, and permutation of identical atoms. While low-order "correlation functions" (like pairwise interactions) are easy to compute, they are often insufficient to capture the full complexity of atomic environments. Higher-order correlations are more descriptive but lead to an exponential increase in the number of features, making them computationally intractable. This paper tackles the challenge of creating representations that are both highly descriptive (capturing high-order interactions) and computationally manageable, allowing for systematically improvable models.

Methods

The core of the proposed method is the N-body iterative contraction of equivariants (NICE) framework. It builds upon the idea of "equivariant features," which are generalizations of N-body invariants that transform predictably under symmetry operations.

Here's a breakdown of the key techniques:

  • Recursive Construction of Equivariant Features: The paper defines a recursive formula to build higher-order equivariant features from lower-order ones. This is analogous to how angular momenta are combined in quantum mechanics. This allows for a systematic generation of features up to any desired "body order" (representing interactions involving that many atoms).
  • Iterative Contraction: To combat the exponential growth in the number of features, NICE employs an iterative contraction step. At each body order, a correlation matrix is computed between the generated features and the atomic density. This matrix is then diagonalized, and the most significant features (principal components) are selected. This effectively "contracts" the feature space by keeping only the most informative components.
  • Symmetry Preservation: The framework explicitly incorporates symmetry by using "equivariant" features, which transform predictably under rotations and inversions. This ensures that the learned properties are independent of the structure's orientation or position.
  • Parity Consideration: The method also tracks the parity (even or odd behavior under inversion) of features, recognizing that different parities can decay at different rates and can be treated separately for efficiency.

Datasets

The primary dataset used for evaluation is a collection of 3 million CH4 quasi-random configurations. This dataset is noted for its large size and random nature, which is considered a "worst-case scenario" for testing the framework's ability to handle complex, potentially less structured data without benefiting from inherent dimensionality reductions due to specific atomic arrangements.

Key Insights and Ideas

  • Bridging the Gap: NICE provides a practical bridge between simple, low-order features and the exponentially complex, high-order features needed for accurate atomic property prediction.
  • Systematic Improvability: The recursive and iterative nature of NICE means that representations can be systematically improved by increasing the body order or the number of selected features, offering a clear path to better models.
  • Feature Space Compression: The iterative contraction step is crucial. It demonstrates that even with high body orders, a relatively small number of selected features can capture a very high percentage of the dataset's variance, making high-order representations computationally feasible.
  • Parity Matters: Separating features by parity can lead to significant improvements in efficiency and memory usage, as different parity components can behave differently.

Key Results

  • Efficient Feature Generation: The NICE framework successfully generates high-order equivariant features without an exponential explosion in the number of terms.
  • High Variance Capture: Even at higher body orders, a few hundred selected features (principal components) can represent over 99% of the dataset's variance.
  • Linear Model Performance: Linear models trained on NICE features show significant improvement with increasing body order, and the learning curves saturate at a much higher accuracy compared to models using only lower-order features.
  • Neural Network Potential Advantage: While linear models show promise, a neural network potential built on NICE features significantly outperforms linear models, especially in data-rich regimes, indicating the power of combining these features with more flexible regression models.

Limitations and Future Work

  • Proof of Principle: The paper presents NICE as a proof of principle, and the contraction step is a simple application of Principal Component Analysis (PCA). More sophisticated feature selection methods, informed by supervised learning criteria or using sparse techniques, could further enhance the framework.
  • Non-Asymptotic Optimizations: The current work focuses on the core recursive and contraction steps. Future work could explore non-asymptotic optimizations for even greater efficiency.
  • Diverse Datasets: The authors suggest systematic benchmarking on more diverse datasets (beyond random CH4 configurations) is needed to fully assess the framework's robustness.
  • Long-Range Physics: Incorporating features designed to capture long-range physics is another avenue for future research.
  • Comparison with Other Methods: Further comparisons with other advanced methods, including kernel methods and more general non-linear regression strategies, are warranted.

Summary updated

Abstract

Mapping an atomistic configuration to a symmetrized N-point correlation of a field associated with the atomic positions (e.g., an atomic density) has emerged as an elegant and effective solution to represent structures as the input of machine-learning algorithms. While it has become clear that low-order density correlations do not provide a complete representation of an atomic environment, the exponential increase in the number of possible N-body invariants makes it difficult to design a concise and effective representation. We discuss how to exploit recursion relations between equivariant features of different order (generalizations of N-body invariants that provide a complete representation of the symmetries of improper rotations) to compute high-order terms efficiently. In combination with the automatic selection of the most expressive combination of features at each order, this approach provides a conceptual and practical framework to generate systematically improvable, symmetry adapted representations for atomistic machine learning.
BibTeX
@article{nigam2020recursive,
  author = {Jigyasa Nigam and Sergey N. Pozdnyakov and Michele Ceriotti},
  title = {{Recursive evaluation and iterative contraction of <i>N</i>-body equivariant features}},
  journal = {The Journal of Chemical Physics},
  year = {2020},
  doi = {10.1063/5.0021116},
}
Figure