TL;DR
This paper introduces the tmQM_wB97MV dataset, a recomputed and filtered version of the tmQM dataset, to address systematic underprediction issues observed when training graph neural networks (GNNs) on homogeneous transition metal complexes. By using a higher-level DFT functional and addressing structural inconsistencies, they demonstrate significantly improved GNN performance and highlight the importance of data quality for generalizable ML models in catalysis.
What Problem Does This Paper Address?
Discovering new catalysts is crucial for advancing technologies like green hydrogen production. While machine learning (ML) offers a powerful way to accelerate this discovery process by acting as surrogates for expensive quantum mechanical calculations (like DFT), current ML models often struggle with generalizability. Specifically, models trained on existing datasets for homogeneous catalysis, like tmQM, exhibit systematic underprediction for certain subsets of transition metal complexes. This indicates a potential issue with the underlying DFT data quality or its suitability for ML training, hindering the development of reliable ML models for high-throughput screening in this domain. The paper aims to resolve this by creating a more robust dataset and evaluating GNN performance on it.
Methods
The core of the methodology involves:
- Dataset Curation: The original
tmQMdataset was filtered to remove structures with implicit hydrogens, which can lead to anomalous geometries. - High-Level DFT Recomputation: Energies for the remaining structures were recomputed using a more accurate DFT functional (
wB97M-V) and a better basis set (def2-SVPD) compared to the originaltmQMdataset's TPSSh/def2-SVP. This aims to improve the accuracy and consistency of the target energies. - Graph Neural Network (GNN) Training: Several state-of-the-art GNN architectures (SchNet, SpinConv, PaiNN, GemNet-T) were trained on both the original
tmQMand the newtmQM_wB97MVdatasets. - Reference Correction: A linear regression strategy was employed to subtract element-specific average energies, yielding a surrogate for formation energy and helping to normalize the target energy distributions.
- Transfer Learning (Fine-tuning): Pre-trained models from the heterogeneous catalysis dataset OC20 were used to initialize weights for models trained on
tmQM_wB97MV, exploring the benefits of transfer learning.
Datasets
- tmQM: The original dataset of 86,665 transition metal complexes, calculated at the TPSSh/def2-SVP level of DFT.
- tmQM_wB97MV: The newly introduced dataset, containing 86,507 structures from
tmQM(after filtering) with energies recomputed at the wB97M-V/def2-SVPD level of DFT. This dataset is the primary focus for improved ML model training. - OC20: A large dataset for heterogeneous catalysis, used for transfer learning experiments.
Key Insights and Ideas
- Data Quality is Paramount: The systematic underprediction observed with the original
tmQMdataset strongly suggests that the DFT level of theory used was insufficient for certain transition metal complexes, particularly iron-group metals. This highlights that even large datasets can have subtle issues that hinder ML model performance. - ML as a Data Diagnostic Tool: GNNs, when unable to learn a consistent pattern across a subset of data, can serve as a powerful diagnostic tool to identify potential inconsistencies or inaccuracies in the underlying quantum mechanical calculations.
- Improved DFT Yields Better ML: Recomputing energies with a higher-level DFT functional (
wB97M-V) and a better basis set (def2-SVPD) intmQM_wB97MVeffectively resolved the underprediction issue and led to significantly lower errors across all tested GNN models. - Transfer Learning Boosts Performance: Fine-tuning GNNs pre-trained on the heterogeneous catalysis dataset OC20 provided substantial improvements in MAE and EwT metrics, demonstrating the transferability of learned representations between different catalytic domains.
- Neutral Structures Offer a Baseline: Training on neutral structures alone generally yielded better performance metrics (lower MAE, higher EwT) compared to using the entire dataset including charged species. However, models trained on charged species showed continued improvement with more data, indicating the importance of capturing charge variations for comprehensive models.
Key Results
- Reduced Errors: Models trained on
tmQM_wB97MVshowed significantly lower Mean Absolute Errors (MAE) and higher Energy within Threshold (EwT) compared to those trained on the originaltmQMdataset. For example, GemNet-T achieved an MAE of 2.08 meV/atom and EwT of 56.63% ontmQM_wB97MV(Figure 2d), a substantial improvement over its performance ontmQM(Figure 5d). - Elimination of Underprediction: Parity plots for models trained on
tmQM_wB97MVno longer show the distinct regions of underprediction observed withtmQM(compare Figure 2 with Figure 4 and 5). - Model Ranking: GemNet-T consistently outperformed PaiNN, SpinConv, and SchNet across various training scenarios on the
tmQM_wB97MVdataset. - Transfer Learning Gains: Fine-tuning a GemNet-T model pre-trained on OC20 resulted in a drastic reduction in MAE (e.g., from 7.54 meV/atom to 4.85 meV/atom on the full
tmQM_wB97MVdataset) and an increase in EwT (from 11.65% to 17.05%).
Limitations and Future Work
- Incomplete Chemical Space: The
tmQM_wB97MVdataset, while improved, is still insufficient for comprehensive screening of transition metal catalysts. It omits open-shell complexes and reactive intermediates, which are crucial for understanding many catalytic pathways. - Charge and Spin State Handling: The current GNN architectures do not explicitly incorporate charge or spin state information, which is a significant limitation for predicting homogeneous catalytic reactions where redox processes are common.
- Future Directions:
- Developing models capable of differentiating and learning energies for structures with varying charges and spin states.
- Expanding the dataset to include a wider variety of charge and spin states, as well as open-shell complexes and reactive intermediates.
- Further exploring transfer learning approaches, potentially with more sophisticated methods, to leverage knowledge from diverse datasets.
- The authors suggest that combining a larger, more diverse dataset with advanced models could enable accurate ML prediction of complex reaction pathways in homogeneous catalysis.
Summary updated