Multitask methods for predicting molecular properties from heterogeneous data

Katharine Fisher, Michael F. Herbst, Youssef MarzoukThe Journal of Chemical Physics 2024 · DOI

TL;DR

This paper introduces a "multitask" Gaussian process regression framework that significantly reduces the cost of generating high-accuracy molecular property predictions. By cleverly combining data from expensive, high-fidelity quantum chemistry methods (like CCSD(T)) with cheaper, lower-fidelity methods (like DFT), their approach achieves near-CCSD(T) accuracy with an order of magnitude less data generation cost.

What Problem Does This Paper Address?

Accurately predicting molecular properties from first-principles quantum mechanics is computationally expensive, often requiring methods like Coupled Cluster (CCSD(T)) that scale poorly with system size. Density Functional Theory (DFT) offers a cheaper alternative but with lower accuracy. This creates a trade-off: use expensive methods for accuracy but incur high computational costs, or use cheaper methods and sacrifice accuracy. While data-driven surrogate models can help by learning from these first-principles calculations, they typically require homogeneous training data (all from the same method). This paper addresses the challenge of leveraging heterogeneous data sources – data generated with different quantum chemistry methods of varying accuracy and cost – to build accurate predictive models more efficiently.

Methods

The core of this work is a multitask Gaussian Process (GP) regression framework. Unlike traditional GP regression that models a single prediction task, multitask GP models multiple related tasks simultaneously. The key idea is to model a "primary task" (e.g., predicting a property at CCSD(T) accuracy) and several "secondary tasks" (e.g., predicting the same property using different DFT functionals). These tasks are assumed to be related, and the framework learns this relationship.

Specifically, the authors propose an asymmetric multitask model. Here, a primary regression function captures the high-fidelity behavior, and secondary regression functions are modeled as a scaled version of the primary function plus a task-specific "disparity" term. This allows the model to learn from both expensive and cheap data sources without requiring them to be perfectly aligned (i.e., the same molecular configurations don't need to be computed at all levels of theory). This contrasts with older methods like "Δ-learning" which explicitly model the difference between two methods and often require paired data.

Datasets

The authors use two main case studies:

  • Water Trimer Three-Body Interaction Energy: They utilize existing CCSD(T) data and supplement it with new DFT calculations (PBE and SCAN functionals) for a portion of the configurations.
  • Ionization Potential of Small Organic Molecules: They leverage existing datasets and perform new CCSD(T) calculations along with DFT calculations using four different functionals (PBE, PBE0, PBE0_DH, and BLYP).

The paper also discusses the use of Smooth Overlap of Atomic Positions (SOAP) features to represent molecular configurations as input to the GP models.

Key Insights and Ideas

  • Leveraging Heterogeneity is Key: The most significant insight is that combining data from different levels of theory, even with varying coverage of molecular configurations, is highly beneficial. The multitask framework can effectively "learn" from this disparity.
  • Order Doesn't Matter (Much): Unlike some previous methods that required an explicit hierarchy of accuracy, the multitask approach can handle heterogeneous data without imposing an arbitrary order on the secondary tasks. This is crucial when dealing with multiple DFT functionals whose relative accuracy can vary.
  • "Data of Opportunity" is Valuable: The framework is designed to readily incorporate existing, potentially imperfect, datasets, reducing the need for expensive, targeted data generation.
  • Flexibility in Training Set Design: The paper explores various ways to construct training sets (e.g., including "core," "additional," and "target" molecules in secondary tasks), demonstrating the framework's adaptability.
  • Cost Reduction is Substantial: The primary motivation and a key finding is the dramatic reduction in data generation cost required to achieve a target accuracy.

Key Results

  • Order-of-Magnitude Cost Reduction: Multitask surrogates achieved CCSD(T)-level accuracy with over an order of magnitude reduction in data generation cost compared to training solely on CCSD(T) data.
  • Improved Accuracy with More Data: Adding more DFT data to the training set consistently improved the accuracy of the multitask models.
  • Comparable to or Better than Δ-learning: While similar in concept, the multitask approach demonstrated comparable or superior accuracy to Δ-learning, especially when dealing with more than two data sources or when data is not perfectly paired.
  • Water Trimer: Multitask models with SCAN as a secondary task achieved accuracy comparable to a GP model that was an order of magnitude more expensive to train.
  • Organic Molecules: Multitask models showed significant benefits, especially when including target molecule data in the secondary training sets.

Limitations and Future Work

  • Pearson Correlation as a Proxy: The paper uses Pearson's correlation coefficient to estimate the relationship between tasks. While this is computationally efficient and works well in practice, a more detailed optimization of these correlation parameters is suggested for future work.
  • SOAP Feature Optimization: While reasonable SOAP parameters were chosen, further exploration of their impact on performance is mentioned.
  • Generalization to Other Models: The framework is demonstrated with Gaussian Processes, but the authors suggest future work could explore using neural networks or other regression tools for the submodels.
  • Full Optimization of Multitask A: The "multitask A" option, which models differences, could be further optimized by exploring correlation parameters beyond simple estimates.
  • Individual Molecule Prediction: While overall MAE is reported, a deeper analysis of prediction quality for individual molecules is deferred to future work.

Summary updated

Abstract

Data generation remains a bottleneck in training surrogate models to predict molecular properties. We demonstrate that multitask Gaussian process regression overcomes this limitation by leveraging both expensive and cheap data sources. In particular, we consider training sets constructed from coupled-cluster (CC) and density functional theory (DFT) data. We report that multitask surrogates can predict at CC-level accuracy with a reduction in data generation cost by over an order of magnitude. Of note, our approach allows the training set to include DFT data generated by a heterogeneous mix of exchange-correlation functionals without imposing any artificial hierarchy on functional accuracy. More generally, the multitask framework can accommodate a wider range of training set structures-including the full disparity between the different levels of fidelity-than existing kernel approaches based on Δ-learning although we show that the accuracy of the two approaches can be similar. Consequently, multitask regression can be a tool for reducing data generation costs even further by opportunistically exploiting existing data sources.
BibTeX
@article{fisher2024multitask,
  author = {Katharine Fisher and Michael F. Herbst and Youssef Marzouk},
  title = {{Multitask methods for predicting molecular properties from heterogeneous data}},
  journal = {The Journal of Chemical Physics},
  year = {2024},
  doi = {10.1063/5.0201681},
}
Figure