DrugOrchestra: Jointly predicting drug response, targets, and side effects via deep multi-task learning

Yuepeng Jiang, Stefano Rensi, Sheng Wang, Russ B. AltmanbioRxiv 2020 · DOI

TL;DR

This paper introduces DrugOrchestra, a novel deep multi-task learning framework that jointly predicts drug response, targets, and side effects. By leveraging shared molecular representations and learning across these related tasks, DrugOrchestra significantly improves prediction accuracy and, crucially, enables predictions for entirely new drugs based solely on their molecular structure.

What Problem Does This Paper Address?

The discovery and development of new drugs is a complex and costly process. While vast amounts of data exist on drug responses, their targets, and potential side effects, current computational approaches typically tackle these problems in isolation. This siloed approach misses out on the inherent connections between these three critical aspects of drug behavior. For instance, understanding a drug's target can shed light on its efficacy and potential adverse reactions. By failing to model these relationships, existing methods may overlook valuable insights and limit their predictive power, especially when dealing with novel compounds for which limited experimental data is available.

Methods

DrugOrchestra employs a deep multi-task learning (MTL) architecture. The core idea is to train a single model to perform multiple related tasks simultaneously, allowing it to learn a more robust and generalized representation.

Here's a breakdown of their approach:

  • Pre-trained Drug Representations: They start with pre-trained molecular structure-based drug representations (derived from SMILES strings) that capture general chemical knowledge. This serves as a shared foundation for all tasks.
  • Hard Parameter Sharing: The model uses a "hard parameter sharing" strategy, where the initial layers (shared layers) are common across all three tasks (drug response, target prediction, and side effect prediction). This encourages the model to learn common patterns and features relevant to all tasks.
  • Task-Specific Layers: Following the shared layers, each task has its own set of task-specific layers that process unique input features (e.g., gene expression for drug response, disease features for side effects) and learn task-specific representations.
  • Joint Optimization: The model is trained end-to-end by jointly optimizing the loss functions for all three tasks. This allows the model to leverage information from one task to improve performance on others.
  • Dynamic Weight Adjustment: To balance the importance of different tasks during training, they use a dynamic weight adjustment strategy, which automatically tunes the contribution of each task's loss to the overall objective.
  • Prediction for Unseen Drugs: A key design choice is to ensure the model can predict for drugs not seen during training, a critical capability for drug discovery.

Datasets

The authors constructed a comprehensive, heterogeneous drug discovery dataset by integrating data from eight publicly available sources across the three tasks:

  • Drug Target Prediction: Repurposing Hub, Drugbank, STITCH.
  • Drug Response Prediction: Patient-Derived Xenograft (PDX) dataset, Genomics of Drug Sensitivity in Cancer (GDSC), Cancer Therapeutics Response Portal (CCLE).
  • Drug Side Effect Prediction: SIDER, OFFSIDES.

This combined dataset includes over 21,000 drugs, 17,500 genes, 2,000 cell lines, and 1,000 side effects. They meticulously filtered and processed these datasets, ensuring consistent drug representations (SMILES) and handling data quality issues.

Key Insights and Ideas

  • Interconnectedness of Drug Properties: The most significant insight is that drug response, targets, and side effects are not independent. Modeling them jointly leads to better predictions than treating them in isolation.
  • Knowledge Transfer is Key: The pre-trained molecular representations and the multi-task learning framework enable effective knowledge transfer. Information learned about drug targets can improve predictions of drug response, and vice-versa.
  • Robustness to Novel Drugs: DrugOrchestra's ability to predict for unseen drugs is a major breakthrough. This is achieved by learning a more generalized drug representation that is not overly dependent on specific training examples.
  • Task Similarity Matters: Transferability is generally higher between datasets of the same task (e.g., drug response to drug response) than between different tasks, suggesting that while tasks are related, their specific data distributions still hold unique information.
  • Drug Response is a Central Phenotype: The paper highlights that drug response prediction sees the most significant improvement from multi-task learning, suggesting it's a central phenotype that benefits most from understanding targets and side effects.

Key Results

  • Significant Performance Gains: DrugOrchestra achieved significant performance improvements on 7 out of 8 datasets compared to single-task learning (STL-NN), SVM, and Random Forest models.
  • Improved Drug Response Prediction: The method showed particularly strong improvements in drug response prediction, with an example showing a 0.375 SCC compared to 0.315 SCC for STL-NN on the GDSC dataset.
  • Effective Transferability: The study demonstrated positive transferability across datasets and tasks, with some tasks (like drug targets) providing substantial benefits to others (like drug response).
  • Prediction for Unseen Drugs: Even when excluding drugs with high structural similarity to training data, DrugOrchestra consistently outperformed STL-NN, underscoring its ability to generalize to novel compounds. For instance, at a Tanimoto similarity cutoff of 0.6, DrugOrchestra achieved 0.785 AUROC in STITCH, compared to 0.745 for STL-NN.

Limitations and Future Work

  • Negative Transferability: The authors observed instances of "negative transferability," particularly between drug side effect and drug target datasets, suggesting that not all task combinations are equally beneficial and that careful consideration of task relationships is important.
  • Interpretability: While the model shows strong performance, understanding why it makes certain predictions remains a challenge. The authors express interest in developing interpretable methods to shed light on the multi-task learning process.
  • Expanding Tasks: Future work could involve integrating additional tasks into the multi-task learning framework, such as drug perturbation prediction, solubility prediction, and synergistic effect prediction, to further enhance the drug representation.
  • Experimental Validation: The authors suggest experimentally validating the model's prioritized predictions for drug repurposing and development.

Summary updated

Abstract

Abstract Massively accumulated pharmacogenomics, chemogenomics, and side effect datasets offer an unprecedented opportunity for drug response prediction, drug target identification and drug side effect prediction. Existing computational approaches limit their scope to only one of these three tasks, inevitably overlooking the rich connection among them. Here, we propose DrugOrchestra, a deep multi-task learning framework that jointly predicts drug response, targets and side effects. DrugOrchestra leverages pre-trained molecular structure-based drug representation to bridge these three tasks. Instead of directly fine-tuning on an individual task, DrugOrchestra uses deep multi-task learning to obtain a phenotype-based drug representation by simultaneously fine-tuning on drug response, target and side effect prediction. By coupling these three tasks together, DrugOrchestra is able to make predictions for unseen drugs by only knowing their molecular structures. We constructed a heterogeneous drug discovery dataset of over 21k drugs by integrating 8 datasets across three tasks. Our method obtained significant improvement in comparison to methods that were trained on a single task or a single dataset. We further revealed the transferability across 8 datasets and 3 tasks, providing novel insights for understanding drug mechanisms. Availability https://github.com/jiangdada1221/DrugOrchestra
BibTeX
@misc{jiang2020drugorchestra,
  author = {Yuepeng Jiang and Stefano Rensi and Sheng Wang and Russ B. Altman},
  title = {{DrugOrchestra: Jointly predicting drug response, targets, and side effects via deep multi-task learning}},
  year = {2020},
  doi = {10.1101/2020.11.17.385757},
  howpublished = {bioRxiv},
}
Figure