PharmRL: Pharmacophore elucidation with Deep Geometric Reinforcement Learning

Rishal Aggarwal, David Ryan KoesResearch Square 2024 · DOI

TL;DR

This paper introduces PharmRL, a novel deep learning framework that automatically identifies pharmacophores (key interaction features between a protein and a drug molecule) directly from protein structures, bypassing the need for experimentally determined ligand-bound structures. By combining a CNN for feature prediction with a deep reinforcement learning approach for feature selection, PharmRL demonstrates improved virtual screening performance and offers a more accessible route to drug discovery.

What Problem Does This Paper Address?

Identifying pharmacophores is a crucial step in drug discovery, enabling the virtual screening of large compound libraries to find potential drug candidates. Traditionally, this process relies heavily on having co-crystal structures of a protein bound to a ligand. These structures provide "ground truth" information about favorable interactions. However, obtaining such structures is often difficult, time-consuming, and expensive, especially for novel drug targets. Existing methods to generate pharmacophores without a ligand often involve complex simulations or manual expert intervention, which can be biased and labor-intensive. This paper tackles the challenge of developing an automated, data-driven method to identify pharmacophores solely from the protein's binding site structure, making pharmacophore modeling more accessible and efficient.

Methods

PharmRL employs a two-stage deep learning approach:

  1. Convolutional Neural Network (CNN) for Feature Prediction: A CNN is trained to predict potential "pharmacophore features" (e.g., hydrogen bond donors/acceptors, hydrophobic regions, aromatic interactions) at various points within a protein's binding site. This CNN is trained on features extracted from existing co-crystal structures and then fine-tuned using adversarial examples to ensure its predictions are physically plausible and relevant to the protein's functional groups.
  2. Deep Geometric Reinforcement Learning (RL) for Pharmacophore Assembly: The predicted features from the CNN are then fed into a deep geometric Q-learning algorithm. This RL agent iteratively builds a pharmacophore by selecting an optimal subset of these predicted features. It models this process as a Markov Decision Process (MDP), where the "state" is a heterogeneous graph representing the protein and candidate pharmacophore features, and the "action" is selecting the next best pharmacophore. A SE(3)-equivariant neural network is used to parameterize the Q-function, allowing it to handle the 3D geometric nature of the problem. This RL approach allows the model to consider the overall performance of the assembled pharmacophore, rather than just the immediate value of adding individual features.

Datasets

  • PDBBind V. 2019 dataset: Used for initial training of the CNN model to identify pharmacophore features from known protein-ligand interactions.
  • Dataset of Useful Decoys - Enhanced (DUD-E): Used for training and evaluating the RL algorithm. This dataset is known for its diverse set of protein-ligand systems and provides both active and decoy molecules for benchmarking.
  • LIT-PCBA dataset: Used to evaluate the performance of PharmRL in identifying active molecules.
  • COVID Moonshot dataset: Used to test the method's ability to identify potential lead molecules for SARS-CoV-2, even in the absence of fragment screening data.
  • SARS-CoV-2 Mpro protein-fragment structures: A dataset of 23 protein-fragment complexes used to generate pharmacophore features for the COVID Moonshot screening.

Key Insights and Ideas

  • Ligand-free pharmacophore generation is feasible: The core idea that a CNN can accurately predict potential interaction points from protein structure alone, and an RL agent can assemble these into effective pharmacophores, is a significant advancement.
  • Reinforcement learning excels at feature selection: The paper highlights that a supervised approach focusing on individual features is insufficient. RL's ability to consider the collective performance of a pharmacophore is key to its success.
  • CNN features can sometimes outperform ligand-derived features: In some cases, the pharmacophores generated using CNN-predicted features achieved better screening performance than those derived from ligand-based features, suggesting the CNN can capture subtle binding site characteristics.
  • Open-source and human-in-the-loop capabilities: The framework is designed to be open-source and allows for user intervention in selecting features or controlling pharmacophore size, making it a practical tool.

Key Results

  • PharmRL demonstrated improved prospective virtual screening performance (measured by F1 scores) on the DUD-E dataset compared to random selection of ligand-identified features.
  • On the LIT-PCBA dataset, PharmRL showed comparative performance to a baseline method (Apo2ph4), often yielding better F1 scores and enrichment factors.
  • For the COVID Moonshot dataset, PharmRL successfully identified effective pharmacophores for screening SARS-CoV-2 Mpro inhibitors, even without fragment screening data.
  • The CNN model achieved high precision (ROC-AUC > 0.95) in classifying pharmacophore feature points.

Limitations and Future Work

  • Dataset size limitations: The DUD-E dataset, while diverse, contains only 102 systems, which may limit the generalization of the RL models.
  • Variability in optimal policy: The inherent complexity of protein-ligand binding means there isn't a single "correct" pharmacophore for a given site, leading to variability in the learned RL policies.
  • Potential for bias: While the method aims to be data-driven, biases in the training data (e.g., over-representation of certain chemical series) could still influence results.
  • Future work: The authors suggest further exploration of the RL framework, potentially with larger datasets and more diverse protein targets. The open-source nature and human-in-the-loop design also pave the way for broader adoption and integration into existing drug discovery pipelines.

Summary updated

Abstract

Molecular interactions between proteins and their ligands are important for drug design. A pharmacophore consists of favorable molecular interactions in a protein binding site and can be utilized for virtual screening. Pharmacophores are easiest to identify from co-crystal structures of a bound protein-ligand complex. In this work, however, we develop a deep learning method that can identify pharmacophores in the absence of a ligand. Specifically, we train a CNN model to identify potential favorable interactions in the the binding site, and develop a deep geometric Q-learning algorithm that attempts to select an optimal subset of these interaction points to form a pharmacophore. With this algorithm, we show better prospective virtual screening performance, in terms of F1 scores, on the DUD-E dataset than random selection of ligand identified features from co-crystal structures. We also conduct experiments on the LIT-PCBA dataset and show that it provides efficient solutions for identifying active molecules. Finally, we test our method by screening the COVID moonshot dataset and show that it would be effective in identifying prospective lead molecules even in the absence of fragment screening experiments. Alongside, we provide a Google Colab notebook for ease of use of the developed method.
BibTeX
@inproceedings{aggarwal2024pharmrl,
  author = {Rishal Aggarwal and David Ryan Koes},
  title = {{PharmRL: Pharmacophore elucidation with Deep Geometric Reinforcement Learning}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-5033986/v1},
}
Figure