TL;DR
This paper introduces RuSH, a novel generative reinforcement learning framework for "scaffold hopping" – designing new molecular structures with similar biological activity but different core scaffolds. RuSH achieves this by enabling "unconstrained" full-molecule generation, allowing for greater exploration of chemical space and more diverse scaffold designs compared to existing methods.
What Problem Does This Paper Address?
Designing novel molecular scaffolds that maintain or improve biological activity while differing from existing structures (scaffold hopping) is a critical but challenging task in drug discovery. Current computational approaches often confine scaffold hopping to predefined substructures or linkers, limiting the exploration of chemical space and potentially leading to suboptimal designs. This limitation can hinder the discovery of truly novel and patentable drug candidates. The authors aim to overcome this by developing a method that can generate entire molecules, rather than just connecting predefined parts, thereby offering a more flexible and comprehensive approach to scaffold hopping.
Methods
The core of the proposed method, RuSH (Reinforcement Learning for Unconstrained Scaffold Hopping), is built upon the REINVENT framework. Key components include:
- Generative Model (LSTM Prior): A long short-term memory (LSTM) network is trained on a large dataset of drug-like molecules (ChEMBL) to learn general molecular generation patterns. This "Prior" model can then be "fine-tuned" by transfer learning on a specific reference molecule to better adapt to its structural characteristics.
- Reinforcement Learning Agent: The generative model acts as the agent, iteratively generating molecules (SMILES strings).
- Novel Scoring Function: This is a crucial innovation. It jointly rewards designs for:
- 3D Similarity: High similarity in shape and pharmacophore to a reference molecule (using ROCS).
- Scaffold Dissimilarity: Low similarity in terms of substructures to the reference molecule's scaffold. This is achieved through a new algorithm called ScaffoldFinder, which performs "fuzzy matching" to identify decorations and cleave the scaffold, allowing for some variation in decorations.
- ScaffoldFinder Algorithm: This algorithm identifies reference decorations within a generated molecule, allowing for a degree of "fuzziness" (controlled by parameter 'a') in the matching process. This flexibility helps in exploring variations of decorations and improving the learning process.
- Partial Reward: To address sparse reward issues in reinforcement learning, a partial reward is given for designs that contain some, but not all, decorations.
- Augmented Negative Log-Likelihood (NLLAug): This combines the scoring function's reward with the chemical plausibility of the generated molecule (measured by the NLL of the Prior model) to guide the agent's learning.
- Inception Feature: This feature periodically exposes the agent to previously high-scoring designs to accelerate learning.
Datasets
- ChEMBL: Used to train the initial "Prior" LSTM model, containing approximately 1.5 million drug-like molecules.
- Protein Data Bank (PDB): Used to obtain crystal structures of protein targets and their bound ligands for evaluating 3D similarity.
- Four Protein Targets: The method was evaluated on four specific protein targets: PIM1, HIV1, JNK3, and ADCY10, for which known scaffold-hopping examples and corresponding crystal structures were available.
Key Insights and Ideas
- Unconstrained Generation is Key: The ability to generate full molecules, rather than just linkers or scaffolds, significantly expands the chemical space that can be explored for scaffold hopping. This is a major departure from many existing methods.
- ScaffoldFinder's Fuzzy Matching: The "fuzzy" matching in ScaffoldFinder is critical. It allows for variations in decorations, which is essential for discovering novel structures and improving the robustness of the learning process.
- Transfer Learning Accelerates Learning: Fine-tuning the generative model on a reference molecule significantly speeds up the inclusion of input decorations and the overall learning process, especially for complex targets.
- Joint Optimization is Effective: The scoring function's ability to balance 3D similarity with scaffold dissimilarity is crucial for guiding the agent towards successful scaffold-hopping candidates.
Key Results
- Successful Scaffold Hopping: RuSH successfully generated novel molecular scaffolds for three out of four tested protein targets, producing designs that were structurally similar to known scaffold hops.
- Improved Learning Speed: Transfer learning led to faster inclusion of decorations and quicker convergence to higher scores compared to using a pre-trained model alone.
- High-Quality Designs: RuSH consistently produced a higher percentage of "high quality" designs (passing filtering criteria) for HIV1, ADCY10, and JNK3 compared to benchmark methods.
- Scaffold Diversity: The method demonstrated the ability to generate diverse scaffolds, with RuSH producing a significant proportion of scaffolds without cyclic substructures, which was not observed in linker-based methods.
- Competitive Performance: When compared to established methods like DeLinker and Link-INVENT, RuSH showed competitive or superior performance across various metrics, particularly in generating high-quality and diverse designs. Notably, RuSH outperformed baselines on the challenging HIV1 case.
Limitations and Future Work
- Decoration Inclusion Difficulty: For some targets, learning to include all decorations can be challenging, though transfer learning helps mitigate this.
- Similarity Metrics: While ECFPs and Tanimoto distance were used, other similarity metrics might be more effective for scaffold hopping.
- Postprocessing Importance: The quality of generated designs is still dependent on postprocessing steps.
- Parameter Space Exploration: The optimal weights for the scoring function (w2D, w3D, ws, wc) and the fuzziness parameter 'a' in ScaffoldFinder could be further explored through large-scale experiments.
- Multiobjective Optimization: Future work could explore combining RuSH with secondary objectives for multiobjective optimization, such as solubility or drug-likeness, and using multiple reference molecules simultaneously.
Summary updated