Deep integration of machine learning into computational chemistry and materials science

Julia Westermayr, Michael Gastegger, Kristof T. Schütt, Reinhard J. MaurerarXiv 2021 · arXiv

TL;DR

Machine learning (ML) is rapidly transforming computational chemistry and materials science by enabling faster and more accurate simulations of molecular and material properties. This paper provides a comprehensive overview of how ML is being integrated into existing workflows, from model building and electronic structure calculations to dynamics simulations and connecting theory with experiment, ultimately paving the way for more efficient discovery and design.

What Problem Does This Paper Address?

Computational chemistry and materials science rely heavily on electronic structure theory and molecular simulations to predict properties and understand behavior. However, these methods are often computationally expensive, limiting the scale and complexity of systems that can be studied. This creates a bottleneck for scientific discovery, particularly in areas like drug design, materials discovery, and understanding complex chemical reactions. The paper addresses the challenge of making these powerful computational tools more accessible, efficient, and broadly applicable by exploring the deep integration of machine learning.

Methods

The paper discusses a wide range of ML techniques being applied, including:

  • Supervised Learning: Used for regression (e.g., predicting energies, forces, properties) and classification (e.g., identifying multi-reference character).
  • Unsupervised Learning: Applied for dimensionality reduction, clustering, and identifying collective variables in molecular dynamics.
  • Deep Learning (Neural Networks): Particularly convolutional neural networks (CNNs) and graph neural networks (GNNs) are used for learning representations of molecular structures and electronic properties.
  • Kernel Methods (e.g., Gaussian Processes): Employed for tasks like uncertainty quantification and building surrogate models.
  • Active Learning and Bayesian Optimization: Strategies to efficiently select training data and explore complex search spaces.
  • Transfer Learning and $\Delta$-ML: Techniques to leverage knowledge from existing models and improve data efficiency by learning differences between methods.
  • Physically Informed ML: Designing ML models that incorporate physical symmetries and constraints (e.g., roto-translational invariance, permutational invariance) to improve accuracy and data efficiency.

Datasets

The paper references numerous existing datasets used for training and evaluation, including:

  • QM9: A dataset of small organic molecules with computed quantum chemical properties.
  • QM7-X: A dataset for atomization energies of organic molecules.
  • Various surface science and materials databases: Used for training interatomic potentials and predicting material properties.

The paper does not introduce a new dataset but emphasizes the importance of data availability and sharing for the advancement of ML in the field.

Key Insights and Ideas

  • ML as a "Swiss Army Knife": ML is not a single solution but a versatile toolkit that can enhance virtually every step of the computational workflow, from initial model building to final data analysis.
  • Bridging the Accuracy-Efficiency Gap: ML excels at learning complex relationships from data, allowing it to create surrogate models that can predict properties with near ab initio accuracy but at a fraction of the computational cost.
  • Beyond Energy and Forces: While ML has been highly successful in developing interatomic potentials for molecular dynamics, its application is expanding to predict a wider range of properties, including electronic structure quantities, excited states, and spectroscopic data.
  • Democratizing Complex Methods: ML has the potential to automate complex decision-making processes (e.g., active space selection in multi-reference methods) and make advanced computational techniques more accessible to non-experts.
  • The Rise of Modular Software: The trend towards modular software design in computational chemistry facilitates the integration of ML libraries and tools, creating more flexible and powerful workflows.
  • Connecting Theory and Experiment: ML can act as a crucial bridge by enabling more efficient and accurate theoretical predictions that can be directly compared to experimental results, and by extracting valuable information from experimental data.

Key Results

  • Accelerated Simulations: ML-based interatomic potentials can speed up molecular dynamics simulations by orders of magnitude, enabling the study of larger systems and longer timescales.
  • High Accuracy: ML models can achieve accuracy comparable to high-level quantum chemistry methods (e.g., Coupled Cluster) for various properties, often with significantly less computational cost.
  • Improved Data Efficiency: Techniques like $\Delta$-ML and physically informed ML architectures can drastically reduce the amount of training data required for accurate models.
  • Automated Workflows: ML is enabling more automated approaches for tasks like geometry optimization, transition state search, and active space selection.
  • Enhanced Spectroscopic Predictions: ML models can accurately predict vibrational, UV-Vis, and NMR spectra, facilitating the interpretation of experimental measurements.

Limitations and Future Work

  • Data Requirements and Transferability: While ML improves data efficiency, generating sufficient and representative training data remains a challenge, especially for complex systems or rare events. Ensuring the transferability of ML models to new chemical spaces is also an ongoing area of research.
  • Uncertainty Quantification: Developing robust methods for quantifying the uncertainty of ML predictions is crucial for reliable scientific applications.
  • Integration into Software: Seamless integration of ML tools into existing electronic structure and simulation software packages requires standardization and community effort.
  • Education and Training: Computational scientists will need to adapt their skill sets to incorporate ML concepts and understand the capabilities and limitations of these new tools.
  • Future Directions: The authors highlight the potential for ML in inverse design, automated alchemical transformations for materials discovery, and further development of ML-driven quantum dynamics simulations. They also emphasize the need for open data sharing and collaborative development to foster the sustainable integration of ML.

Summary updated

Abstract

Machine learning (ML) methods are being used in almost every conceivable area of electronic structure theory and molecular simulation. In particular, ML has become firmly established in the construction of high-dimensional interatomic potentials. Not a day goes by without another proof of principle being published on how ML methods can represent and predict quantum mechanical properties -- be they observable, such as molecular polarizabilities, or not, such as atomic charges. As ML is becoming pervasive in electronic structure theory and molecular simulation, we provide an overview of how various aspects of atomistic computational modelling are being transformed by the incorporation of ML approaches. From the perspective of the practitioner in the field, we assess how common workflows to predict structure, dynamics, and spectroscopy are affected by ML. Finally, we discuss how a deeper and lasting integration of ML methods with computational chemistry and materials science can be achieved and what it will mean for research practice, software development, and postgraduate training.
BibTeX
@misc{westermayr2021deep,
  author = {Julia Westermayr and Michael Gastegger and Kristof T. Schütt and Reinhard J. Maurer},
  title = {{Deep integration of machine learning into computational chemistry and materials science}},
  year = {2021},
  eprint = {2102.08435},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.08435},
}
Figure