Unsupervised machine learning in atomistic simulations, between predictions and understanding

Michele CeriottiThe Journal of Chemical Physics 2019 · DOI · arXiv

Abstract

Automated analyses of the outcome of a simulation have been an important part of atomistic modeling since the early days, addressing the need of linking the behavior of individual atoms and the collective properties that are usually the final quantity of interest. Methods such as clustering and dimensionality reduction have been used to provide a simplified, coarse-grained representation of the structure and dynamics of complex systems from proteins to nanoparticles. In recent years, the rise of machine learning has led to an even more widespread use of these algorithms in atomistic modeling and to consider different classification and inference techniques as part of a coherent toolbox of data-driven approaches. This perspective briefly reviews some of the unsupervised machine-learning methods—that are geared toward classification and coarse-graining of molecular simulations—seen in relation to the fundamental mathematical concepts that underlie all machine-learning techniques. It discusses the importance of using concise yet complete representations of atomic structures as the starting point of the analyses and highlights the risk of introducing preconceived biases when using machine learning to rationalize and understand structure-property relations. Supervised machine-learning techniques that explicitly attempt to predict the properties of a material given its structure are less susceptible to such biases. Current developments in the field suggest that using these two classes of approaches side-by-side and in a fully integrated mode, while keeping in mind the relations between the data analysis framework and the fundamental physical principles, will be key to realizing the full potential of machine learning to help understand the behavior of complex molecules and materials.

Introduction: The Challenge of Coarse-Graining

Modern atomistic simulations, ranging from protein folding to the behavior of catalysts and battery materials, generate immense datasets tracking millions or billions of atoms. The central challenge is coarse-graining: translating this chaotic list of coordinates into a high-level understanding of structural motifs, metastable states, or reaction pathways. While human intuition is powerful, it is prone to bias. Machine learning (ML) offers a data-centric alternative, yet the common belief that unsupervised learning is entirely "agnostic" is a pitfall. This perspective explores how unsupervised ML acts as a bridge between raw data and physical insight, emphasizing the need for robust representations and the integration of supervised methods.

The Foundation: Representations and Symmetries

Before applying any ML algorithm, atomic structures must be converted into mathematical vectors. A critical requirement for these representations is that they respect fundamental physical symmetries: translation, rotation, and permutation of identical atoms.

Representations must be invariant under physical symmetries: (b) permutation of identical atoms, (c) translation, and (d) rotation.

The paper highlights that "representation is everything." If a representation is incomplete—for instance, if it ignores three-body correlations—the resulting unsupervised analysis will be blind to those physical features. Common frameworks like SOAP (Smooth Overlap of Atomic Positions) and Behler-Parrinello symmetry functions are discussed as ways to encode these local environments into a format suitable for ML.

The Unsupervised Toolbox: Clustering and Dimensionality Reduction

The core of the unsupervised approach involves identifying patterns within the high-dimensional feature space of atomic configurations.

Overview of the unsupervised ML workflow: from local environments to kernel-based distances, leading to classification, inference, and dimensionality reduction.
  1. Clustering: Techniques such as k-means, Gaussian Mixture Models (GMM), and DBSCAN are employed to identify structural motifs or metastable states, such as the distinct folding states of a protein.
  2. Dimensionality Reduction: Methods like Principal Component Analysis (PCA), Sketch-map, and t-SNE project high-dimensional data onto 2D or 3D "maps." This "materials cartography" allows researchers to visualize transition pathways and the overall landscape of chemical space.
  3. The Kernel Trick: This mathematical technique allows for computing similarities between complex structures without explicitly defining every feature, effectively unifying various representation methods.

Insights: The Myth of the Agnostic Algorithm

A key insight is the warning against the term "unbiased." Human intervention—choosing the representation, selecting hyperparameters, and defining the scope of the data—effectively "bakes in" prior expectations. To validate unsupervised findings, the paper suggests using supervised learning as a check: if a low-dimensional map cannot accurately predict a material's energy or conductivity, it has likely discarded essential physical information.

Results and Applications

Unsupervised ML has demonstrated significant qualitative successes:

  • Identifying Polymorphs: Discovering "exotic" phases by grouping locally stable atomic arrangements.
  • Accelerated Sampling: Using dimensionality reduction to identify "collective variables" in metadynamics, allowing simulations to escape local energy minima.
  • Structural Diversity: Guiding experimental synthesis by identifying unique and stable structures within large databases using Generalized Convex Hulls (GCH).

Conclusion: Toward an Integrated Approach

As simulations move toward higher dimensions, the "curse of dimensionality" makes distance-based metrics less meaningful. The future of the field lies in a fully integrated mode where supervised and unsupervised methods are used side-by-side. By combining the predictive power of supervised learning with the interpretability of unsupervised "cartography," researchers can move beyond black-box predictions toward a deeper rationalization of structure-property relations in complex materials.

Summary updated

Abstract

Automated analyses of the outcome of a simulation have been an important part of atomistic modeling since the early days, addressing the need of linking the behavior of individual atoms and the collective properties that are usually the final quantity of interest. Methods such as clustering and dimensionality reduction have been used to provide a simplified, coarse-grained representation of the structure and dynamics of complex systems from proteins to nanoparticles. In recent years, the rise of machine learning has led to an even more widespread use of these algorithms in atomistic modeling and to consider different classification and inference techniques as part of a coherent toolbox of data-driven approaches. This perspective briefly reviews some of the unsupervised machine-learning methods-that are geared toward classification and coarse-graining of molecular simulations-seen in relation to the fundamental mathematical concepts that underlie all machine-learning techniques. It discusses the importance of using concise yet complete representations of atomic structures as the starting point of the analyses and highlights the risk of introducing preconceived biases when using machine learning to rationalize and understand structure-property relations. Supervised machine-learning techniques that explicitly attempt to predict the properties of a material given its structure are less susceptible to such biases. Current developments in the field suggest that using these two classes of approaches side-by-side and in a fully integrated mode, while keeping in mind the relations between the data analysis framework and the fundamental physical principles, will be key to realizing the full potential of machine learning to help understand the behavior of complex molecules and materials.
BibTeX
@article{ceriotti2019unsupervised,
  author = {Michele Ceriotti},
  title = {{Unsupervised machine learning in atomistic simulations, between predictions and understanding}},
  journal = {The Journal of Chemical Physics},
  year = {2019},
  doi = {10.1063/1.5091842},
}
Figure