Expanding Training Data for Structure-Based Receptor–Ligand Binding Affinity Regression through Imputation of Missing Labels

Paul Francoeur, David Ryan KoesACS 2023 · DOI

TL;DR

This paper tackles the challenge of limited training data for predicting how strongly molecules bind to their protein targets. The authors successfully demonstrate that "imputing" (estimating) missing binding affinity labels using a deep learning model can expand the training dataset and lead to small but statistically significant improvements in prediction accuracy. This offers a promising, computationally inexpensive way to boost the performance of structure-based drug design models.

What Problem Does This Paper Address?

Machine learning models thrive on large datasets. However, in the field of structure-based receptor-ligand binding affinity prediction, the available experimental data is scarce. While previous efforts like the CrossDocked2020 dataset expanded the number of binding poses (how a ligand fits into a receptor), they didn't significantly increase the number of binding affinities (how strongly the ligand binds). This lack of affinity data limits the complexity of models that can be trained and, consequently, their predictive power. The authors aim to address this by exploring methods to generate synthetic binding affinity labels for complexes that currently lack experimental measurements, thereby expanding the training data.

Methods

The core of this work involves using a deep learning model, specifically a 3D convolutional neural network (CNN) architecture (Def2018), to predict binding affinity. The novel aspect is the use of this trained model to impute missing binding affinity labels. Here's a breakdown of their approach:

  • Imputation Strategy: They train a CNN on existing labeled data. Then, for complexes with missing affinity labels, they use this trained model to predict an affinity value. This predicted value is then used as a "label" to augment the training set.
  • Iterative Imputation: They found that repeating the process of training a model, using it to impute labels, and then retraining with the new labels (imputation iterations) can further improve performance.
  • Ensemble Imputation: To reduce noise and improve robustness, they explored using an ensemble of predictions from multiple models (trained with different random seeds) to generate the imputed labels.
  • "Good Poses Only" Imputation: Recognizing that some predicted poses might be of lower quality, they investigated using only imputed labels from poses with low root-mean-square deviation (RMSD) to the crystal structure, aiming to provide more reliable synthetic data.
  • Model Architecture: They used a 3D CNN that takes a grid representation of the receptor-ligand complex as input and outputs both binding affinity and pose scores.

Datasets

  • CrossDocked2020v1.3: This is the primary dataset used for training and imputation. It's an expansion of previous datasets, containing millions of receptor-ligand poses generated through cross-docking. A significant portion (around 59%) of these poses lack experimentally determined binding affinity labels. The dataset is structured with clustered cross-validation splits based on pocket similarity.
  • Roche Data Set: This dataset, containing 1162 PDE10A inhibitors with experimentally determined binding affinities, was used as an independent test set to evaluate the generalizability of their imputation strategy. Crucially, none of the exact structures from this dataset were present in CrossDocked2020v1.3, though the target protein (PDE10A) was.

Key Insights and Ideas

  • Imputation is Viable: The most significant insight is that imputing missing binding affinity labels using a trained deep learning model is a practical and effective strategy for expanding training data.
  • Ensembles are Better: Using an ensemble of predictions from multiple models for imputation leads to more robust and improved results compared to using a single model's predictions.
  • Focusing on "Good" Poses Matters: Imputing labels derived only from high-quality predicted poses (low RMSD to crystal structure) yields better results than using all poses, suggesting that noise from low-quality predictions can be detrimental.
  • Balance is Key: The data suggests that an optimal balance between experimentally known labels and imputed labels is crucial for maximizing performance. Simply flooding the dataset with imputed labels might not be the most effective approach.
  • Orthogonal Improvement: Imputing affinity labels, even though it's an orthogonal technique to model architecture, can lead to improvements in both binding affinity prediction and, to a lesser extent, binding pose classification, likely due to shared model weights.

Key Results

  • Small but Significant Improvements: The imputation strategies consistently led to small but statistically significant improvements in binding affinity prediction metrics like RMSE and Pearson's R on the CrossDocked2020 dataset. For example, the "median good only ensemble" imputation type showed the best performance.
  • Pose Classification Boost: Including imputed binding affinity labels also resulted in a small beneficial effect on the binding pose classification task.
  • Generalizability Tested: On the independent Roche test set, the imputation approach showed a trend of improvement in RMSE, but the statistical significance was not as consistent across all folds, suggesting that performance gains might not always transfer perfectly to entirely new datasets.
  • Iterative Imputation Benefits: Two rounds of imputation generation were found to achieve near-maximal performance gains, with diminishing returns beyond that.

Limitations and Future Work

  • Potential for Reinforcing Biases: The authors acknowledge that using only the CrossDocked2020 dataset for imputation might reinforce existing biases within that dataset. Future work could involve using models trained on larger, more diverse datasets (like ChEMBL) for imputation, though this presents challenges in data selection and representation.
  • Sophisticated Imputation: The current imputation methods are relatively simple. More advanced techniques, potentially involving models that output confidence scores, could lead to even better imputed labels.
  • Generalizability Nuances: While imputation improved performance on the CrossDocked2020 dataset, the gains on the independent Roche dataset were less consistently statistically significant. This highlights the need for further research into imputation strategies that ensure robust generalization.
  • Dynamic Imputation: The authors suggest that if imputed labels could be generated dynamically during training (rather than through an iterative process), the overhead would be minimal, making it an even more attractive option.

Summary updated

Abstract

The success of machine learning is, in part, due to a large volume of data available to train models. However, the amount of training data for structure-based molecular property prediction remains limited. The previously described CrossDocked2020 data set expanded the available training data for binding pose classification in a molecular docking setting but did not address expanding the amount of receptor-ligand binding affinity data. We present experiments demonstrating that imputing binding affinity labels for complexes without experimentally determined binding affinities is a viable approach to expanding training data for structure-based models of receptor-ligand binding affinity. In particular, we demonstrate that utilizing imputed labels from a convolutional neural network trained only on the affinity data present in CrossDocked2020 results in a small improvement in the binding affinity regression performance, despite the additional sources of noise that such imputed labels add to the training data. The code, data splits, and imputation labels utilized in this paper are freely available at https://github.com/francoep/ImputationPaper.
BibTeX
@inproceedings{francoeur2023expanding,
  author = {Paul Francoeur and David Ryan Koes},
  title = {{Expanding Training Data for Structure-Based Receptor–Ligand Binding Affinity Regression through Imputation of Missing Labels}},
  booktitle = {ACS},
  year = {2023},
  doi = {10.1021/acsomega.3c05931},
}
Figure