Applying Large Graph Neural Networks to Predict Transition Metal Complex Energies Using the tmQM_wB97MV Data Set

Aaron G. Garrison, Javier Heras-Domingo, John R. Kitchin, Gabriel dos Passos Gomes, Zachary W. Ulissi, Samuel M. BlauJournal of Chemical Information and Modeling 2023 · DOI · OpenReview

TL;DR

This paper introduces the tmQM_wB97MV dataset, a recomputed and filtered version of the tmQM dataset, to address systematic underprediction issues observed when training graph neural networks (GNNs) on homogeneous transition metal complexes. By using a higher-level DFT functional and addressing structural inconsistencies, they demonstrate significantly improved GNN performance and highlight the importance of data quality for generalizable ML models in catalysis.

What Problem Does This Paper Address?

Discovering new catalysts is crucial for advancing technologies like green hydrogen production. While machine learning (ML) offers a powerful way to accelerate this discovery process by acting as surrogates for expensive quantum mechanical calculations (like DFT), current ML models often struggle with generalizability. Specifically, models trained on existing datasets for homogeneous catalysis, like tmQM, exhibit systematic underprediction for certain subsets of transition metal complexes. This indicates a potential issue with the underlying DFT data quality or its suitability for ML training, hindering the development of reliable ML models for high-throughput screening in this domain. The paper aims to resolve this by creating a more robust dataset and evaluating GNN performance on it.

Methods

The core of the methodology involves:

  • Dataset Curation: The original tmQM dataset was filtered to remove structures with implicit hydrogens, which can lead to anomalous geometries.
  • High-Level DFT Recomputation: Energies for the remaining structures were recomputed using a more accurate DFT functional (wB97M-V) and a better basis set (def2-SVPD) compared to the original tmQM dataset's TPSSh/def2-SVP. This aims to improve the accuracy and consistency of the target energies.
  • Graph Neural Network (GNN) Training: Several state-of-the-art GNN architectures (SchNet, SpinConv, PaiNN, GemNet-T) were trained on both the original tmQM and the new tmQM_wB97MV datasets.
  • Reference Correction: A linear regression strategy was employed to subtract element-specific average energies, yielding a surrogate for formation energy and helping to normalize the target energy distributions.
  • Transfer Learning (Fine-tuning): Pre-trained models from the heterogeneous catalysis dataset OC20 were used to initialize weights for models trained on tmQM_wB97MV, exploring the benefits of transfer learning.

Datasets

  • tmQM: The original dataset of 86,665 transition metal complexes, calculated at the TPSSh/def2-SVP level of DFT.
  • tmQM_wB97MV: The newly introduced dataset, containing 86,507 structures from tmQM (after filtering) with energies recomputed at the wB97M-V/def2-SVPD level of DFT. This dataset is the primary focus for improved ML model training.
  • OC20: A large dataset for heterogeneous catalysis, used for transfer learning experiments.

Key Insights and Ideas

  • Data Quality is Paramount: The systematic underprediction observed with the original tmQM dataset strongly suggests that the DFT level of theory used was insufficient for certain transition metal complexes, particularly iron-group metals. This highlights that even large datasets can have subtle issues that hinder ML model performance.
  • ML as a Data Diagnostic Tool: GNNs, when unable to learn a consistent pattern across a subset of data, can serve as a powerful diagnostic tool to identify potential inconsistencies or inaccuracies in the underlying quantum mechanical calculations.
  • Improved DFT Yields Better ML: Recomputing energies with a higher-level DFT functional (wB97M-V) and a better basis set (def2-SVPD) in tmQM_wB97MV effectively resolved the underprediction issue and led to significantly lower errors across all tested GNN models.
  • Transfer Learning Boosts Performance: Fine-tuning GNNs pre-trained on the heterogeneous catalysis dataset OC20 provided substantial improvements in MAE and EwT metrics, demonstrating the transferability of learned representations between different catalytic domains.
  • Neutral Structures Offer a Baseline: Training on neutral structures alone generally yielded better performance metrics (lower MAE, higher EwT) compared to using the entire dataset including charged species. However, models trained on charged species showed continued improvement with more data, indicating the importance of capturing charge variations for comprehensive models.

Key Results

  • Reduced Errors: Models trained on tmQM_wB97MV showed significantly lower Mean Absolute Errors (MAE) and higher Energy within Threshold (EwT) compared to those trained on the original tmQM dataset. For example, GemNet-T achieved an MAE of 2.08 meV/atom and EwT of 56.63% on tmQM_wB97MV (Figure 2d), a substantial improvement over its performance on tmQM (Figure 5d).
  • Elimination of Underprediction: Parity plots for models trained on tmQM_wB97MV no longer show the distinct regions of underprediction observed with tmQM (compare Figure 2 with Figure 4 and 5).
  • Model Ranking: GemNet-T consistently outperformed PaiNN, SpinConv, and SchNet across various training scenarios on the tmQM_wB97MV dataset.
  • Transfer Learning Gains: Fine-tuning a GemNet-T model pre-trained on OC20 resulted in a drastic reduction in MAE (e.g., from 7.54 meV/atom to 4.85 meV/atom on the full tmQM_wB97MV dataset) and an increase in EwT (from 11.65% to 17.05%).

Limitations and Future Work

  • Incomplete Chemical Space: The tmQM_wB97MV dataset, while improved, is still insufficient for comprehensive screening of transition metal catalysts. It omits open-shell complexes and reactive intermediates, which are crucial for understanding many catalytic pathways.
  • Charge and Spin State Handling: The current GNN architectures do not explicitly incorporate charge or spin state information, which is a significant limitation for predicting homogeneous catalytic reactions where redox processes are common.
  • Future Directions:
    • Developing models capable of differentiating and learning energies for structures with varying charges and spin states.
  • Expanding the dataset to include a wider variety of charge and spin states, as well as open-shell complexes and reactive intermediates.
  • Further exploring transfer learning approaches, potentially with more sophisticated methods, to leverage knowledge from diverse datasets.
  • The authors suggest that combining a larger, more diverse dataset with advanced models could enable accurate ML prediction of complex reaction pathways in homogeneous catalysis.

Summary updated

BibTeX
@article{garrison2023applying,
  author = {Aaron G. Garrison and Javier Heras-Domingo and John R. Kitchin and Gabriel dos Passos Gomes and Zachary W. Ulissi and Samuel M. Blau},
  title = {{Applying Large Graph Neural Networks to Predict Transition Metal Complex Energies Using the tmQM\_wB97MV Data Set}},
  journal = {Journal of Chemical Information and Modeling},
  year = {2023},
  doi = {10.1021/acs.jcim.3c01226},
}
Figure