Element selection for functional materials discovery by integrated machine learning of atomic contributions to properties

Andrij Vasylenko, Dmytro Antypov, Vladimir V. Gusev, Michael W. Gaultois, Matthew S. Dyer, Matthew J. Rosseinsky

TL;DR

This paper introduces PhaseSelect, a novel end-to-end machine learning framework that predicts material properties and chemical novelty at the "phase field" level (i.e., the set of constituent elements). By focusing on elemental combinations rather than specific compositions, PhaseSelect significantly accelerates materials discovery by reducing combinatorial complexity and enabling more effective exploration of uncharted chemical spaces.

What Problem Does This Paper Address?

Discovering new functional materials is a monumental task due to the vast combinatorial space of possible elemental combinations and compositions. Traditional approaches often rely on extensive screening of specific compounds, which can be computationally expensive and prone to missing novel materials due to historical biases in research data. Furthermore, extrapolating properties from known compositions to entirely new ones is challenging. This paper addresses the need for a higher-level, more efficient approach to guide materials discovery by focusing on the fundamental building blocks: the elements themselves. By considering "phase fields" (sets of elements) instead of individual compounds, the authors aim to reduce the search space and prioritize promising new areas of chemistry for experimental investigation.

Methods

The core of this work is the PhaseSelect model, an end-to-end machine learning framework. It integrates several key components:

  • Elemental Representation Learning: PhaseSelect first learns representations for individual chemical elements. This is achieved using a shallow autoencoder trained on a large materials database (Materials Project). The autoencoder learns to compress sparse binary representations of elemental co-occurrence into a lower-dimensional latent space, capturing essential chemical information.
  • Phase Field Representation: These learned elemental representations are then used to construct representations for phase fields. This involves an "attention mechanism" that weights the contributions of constituent elements to the overall properties of the phase field. This allows the model to understand which elements are most influential for a given property.
  • Supervised Property Prediction: PhaseSelect employs two types of supervised learning:
  • Regression: To predict the maximum expected value of a target property within a phase field.
  • Binary Classification: To assign phase fields to "high-performing" or "low-performing" classes based on predefined property thresholds.
  • Unsupervised Ranking (Chemical Novelty): An autoencoder is used to learn chemical similarity between phase fields. By analyzing reconstruction errors, PhaseSelect can rank phase fields by their novelty, identifying combinations that deviate from known, synthetically accessible materials.

The entire model is trained end-to-end, optimizing all components simultaneously.

Datasets

The study utilizes several large materials databases:

  • Materials Project (MPDS): Used for training the elemental representation learning module.
  • SuperCon: Used for data on superconducting transition temperatures.
  • MPDS: Used for data on Curie temperatures and energy band gaps.
  • ICSD: Used to identify phase fields with reported stable compounds but no known properties in MPDS or SuperCon, serving as a validation set for novel predictions.

The authors aggregated data from these sources into three main datasets for superconducting transition temperature, Curie temperature, and energy band gap.

Key Insights and Ideas

  • Phase Fields as a Powerful Abstraction: Shifting the focus from specific compositions to "phase fields" (sets of elements) dramatically reduces the search space and allows for more efficient exploration of novel chemistries.
  • Attention for Interpretability: The use of attention mechanisms provides insights into which elemental contributions are most critical for predicting specific material properties, aligning with human chemical intuition.
  • Bridging Supervised and Unsupervised Learning: PhaseSelect effectively combines supervised learning for property prediction with unsupervised learning for novelty assessment, offering a comprehensive approach to materials discovery.
  • Predicting Novelty and Performance Simultaneously: The model not only predicts the likelihood of a phase field exhibiting desirable properties but also its novelty, guiding researchers towards unexplored but potentially promising areas.

Key Results

PhaseSelect demonstrated significant improvements in predicting material properties compared to baseline models (random forest with Magpie descriptors).

  • Regression Performance: Achieved MAE values of 8.0 K for superconducting transition temperature, 127.0 K for Curie temperature, and 44.1 eV for energy band gap.
  • Classification Performance: Achieved average accuracies of 80.4% for superconductivity, 86.2% for magnetism, and 75.6% for energy gap.
  • Novelty Ranking: The model successfully identified novel ternary phase fields with high probabilities of exhibiting desired properties and low synthetic uncertainty, including combinations not previously explored for these applications. For example, it highlighted Fe-based ternary materials for magnetism and Mg, Fe, Nb for superconductivity.

Limitations and Future Work

  • Dataset Size and Bias: While the approach mitigates some biases, the performance is still dependent on the available training data. The authors acknowledge that the relatively small size of some datasets limits the possibility of extensive train-test splits and can affect model performance.
  • Extrapolation Challenges: While PhaseSelect aims to improve extrapolation, predicting properties for entirely new elemental combinations remains a challenge.
  • Focus on Maximum Values: The current model focuses on the maximum reported property value within a phase field, potentially overlooking phase fields with interesting property distributions or lower but still useful values.
  • Future Work: The authors suggest that the PhaseSelect framework can be extended to other material properties and that the identified promising phase fields can serve as starting points for more computationally intensive investigations and experimental synthesis. The interpretability offered by the attention mechanism also opens avenues for deeper understanding of structure-property relationships.

Summary updated

Abstract

Fundamental differences between materials originate from the unique nature of their constituent chemical elements. Before specific differences emerge according to the precise ratios of elements in a given crystal structure, a material can be represented by the set of its constituent chemical elements. By working at the level of the periodic table, assessment of materials at the level of their phase fields reduces the combinatorial complexity to accelerate screening, and circumvents the challenges associated with composition-level approaches such as poor extrapolation within phase fields, and the impossibility of exhaustive sampling. This early stage discrimination combined with evaluation of novelty of phase fields aligns with the outstanding experimental challenge of identifying new areas of chemistry to investigate, by prioritising which elements to combine in a reaction. Here, we demonstrate that phase fields can be assessed with respect to the maximum expected value of a target functional property and ranked according to chemical novelty. We develop and present PhaseSelect, an end-to-end machine learning model that combines the representation, classification, regression and ranking of phase fields. First, PhaseSelect constructs elemental characteristics from the co-occurrence of chemical elements in computationally and experimentally reported materials, then it employs attention mechanisms to learn representation for phase fields and assess their functional performance. At the level of the periodic table, PhaseSelect quantifies the probability of observing a functional property, estimates its value within a phase field and also ranks a phase field novelty, which we demonstrate with significant accuracy for three avenues of materials applications for high-temperature superconductivity, high-temperature magnetism, and targeted bandgap energy.
BibTeX
@inproceedings{vasylenko2023element,
  author = {Andrij Vasylenko and Dmytro Antypov and Vladimir V. Gusev and Michael W. Gaultois and Matthew S. Dyer and Matthew J. Rosseinsky},
  title = {{Element selection for functional materials discovery by integrated machine learning of atomic contributions to properties}},
  booktitle = {Npj},
  year = {2023},
  doi = {10.1038/s41524-023-01072-x},
}
Figure