Accurate Clinical Toxicity Prediction using Multi-task Deep Neural Nets and Contrastive Molecular Explanations

Bhanushee Sharma, Vijil Chenthamarakshan, Amit Dhurandhar, Shiranee Pereira, James Hendler, Jonathan S. Dordick, Payel Das

Abstract

Explainable ML for molecular toxicity prediction is a promising approach for efficient drug development and chemical safety. A predictive ML model of toxicity can reduce experimental cost and time while mitigating ethical concerns by significantly reducing animal and clinical testing. Herein, we use a deep learning framework for simultaneously modeling in vitro, in vivo, and clinical toxicity data. Two different molecular input representations are used: Morgan fingerprints and pre-training SMILES embeddings. A multi-task deep learning model accurately predicts toxicity for all endpoints, including clinical, as indicated by AUROC and balanced accuracy. In particular, SMILES embeddings as input to the multi-task model improved clinical toxicity predictions compared to existing models in MoleculeNet benchmark. Additionally, our multi-task approach is comprehensive in the sense that it is comparable to state-of-the-art approaches for specific endpoints in in vitro, in vivo and clinical platforms. Through both the multi-task model and transfer learning, we were able to indicate the minimal need of in vivo data for clinical toxicity predictions. To provide confidence and explain the model's predictions, we adapt a post-hoc contrastive explanation method that returns pertinent positive and pertinent negative features, which correspond well to known mutagenic and reactive toxicophores, such as unsubstituted bonded heteroatoms, aromatic amines, and Michael receptors. Furthermore, toxicophore recovery by pertinent feature analysis captures more of the in vitro (53%) and in vivo (56%), rather than of the clinical (8%), endpoints, and indeed uncovers a preference in known toxicophore data towards in vitro and in vivo experimental data. To our knowledge, this is the first contrastive explanation, using both present and absent substructures, for predictions of clinical and in vivo molecular toxicity.

Introduction

Toxicity is a leading cause of drug candidate failure, contributing significantly to the high costs and long timelines of pharmaceutical development. Traditionally, drug safety is assessed through in vitro (cell-based) and in vivo (animal) testing. However, these results often fail to translate to humans due to biological differences. While machine learning (ML) has been employed to predict toxicity, existing models often suffer from siloed learning—focusing on single platforms—and the "black box" problem, where predictions lack actionable chemical explanations.

Methods

The researchers developed a multi-task deep learning framework designed to bridge the gap between different toxicity testing platforms and provide interpretable results.

Molecular Representations and Architecture

The study compared two primary input types:

  • Morgan Fingerprints (FP): Traditional representations that record the presence of specific atom groups.
  • SMILES Embeddings (SE): Latent representations generated via a translation model trained on millions of chemicals, capturing complex structural relationships.

The core architecture is a Multi-task Deep Neural Network (MTDNN). Unlike single-task models, the MTDNN utilizes a shared representation layer to learn simultaneously from three distinct toxicity benchmarks: ClinTox (human clinical trials), Tox21 (in vitro assays), and RTECS (in vivo acute oral toxicity).

Overview of the multi-task deep learning framework for toxicity prediction and contrastive explanation.

Contrastive Explanations

To address interpretability, the framework incorporates a Contrastive Explanations Method (CEM). This post-hoc analysis identifies Pertinent Positives (PP)—minimal substructures necessary for a "toxic" prediction—and Pertinent Negatives (PN)—features whose absence is critical for a "non-toxic" label.

Results

The multi-task approach demonstrated significant performance gains, particularly for clinical toxicity prediction where data is typically scarce.

Performance Metrics

On the ClinTox dataset, the MTDNN model using SMILES embeddings achieved an exceptional AUROC of 0.99, a substantial improvement over the MoleculeNet baseline of approximately 0.83. The model also maintained high balanced accuracy (~0.96), indicating robustness against class imbalance.

Test AUROC performance across ClinTox, Tox21, and RTECS datasets for various model configurations.

Key Insights

  • Cross-Platform Synergy: Training on in vitro and in vivo data alongside clinical data allows the model to learn general toxicity rules, boosting performance on the smaller clinical dataset.
  • Embedding Superiority: SMILES embeddings consistently outperformed fingerprints for clinical tasks, suggesting that latent chemical features are more relevant to complex human systemic toxicity than simple structural counts.
  • Reduced Animal Reliance: Analysis suggested that in vitro data contributed more significantly to clinical prediction accuracy than in vivo data, hinting at a potential path toward reducing animal testing through advanced ML.
  • Scientific Validation: The CEM method successfully recovered known toxicophores, such as aromatic amines and Michael receptors, capturing over 50% of known in vitro and in vivo toxicophores. The lower recovery rate for clinical toxicophores (8%) underscores the heightened complexity of human systemic toxicity.

Discussion and Future Work

This framework represents a significant step toward explainable and accurate toxicity prediction. By integrating data across biological scales and providing contrastive structural explanations, it offers a tool that is both predictive and informative for medicinal chemists. Future work will focus on expanding clinical datasets, incorporating 3D spatial context into molecular explanations, and accounting for specific biological pathways involved in human systemic toxicity.

Summary updated

Abstract

Explainable ML for molecular toxicity prediction is a promising approach for efficient drug development and chemical safety. A predictive ML model of toxicity can reduce experimental cost and time while mitigating ethical concerns by significantly reducing animal and clinical testing. Herein, we use a deep learning framework for simultaneously modeling in vitro, in vivo, and clinical toxicity data. Two different molecular input representations are used: Morgan fingerprints and pre-training SMILES embeddings. A multi-task deep learning model accurately predicts toxicity for all endpoints, including clinical, as indicated by AUROC and balanced accuracy. In particular, SMILES embeddings as input to the multi-task model improved clinical toxicity predictions compared to existing models in MoleculeNet benchmark. Additionally, our multi-task approach is comprehensive in the sense that it is comparable to state-of-the-art approaches for specific endpoints in in vitro, in vivo and clinical platforms. Through both the multi-task model and transfer learning, we were able to indicate the minimal need of in vivo data for clinical toxicity predictions. To provide confidence and explain the model's predictions, we adapt a post-hoc contrastive explanation method that returns pertinent positive and pertinent negative features, which correspond well to known mutagenic and reactive toxicophores, such as unsubstituted bonded heteroatoms, aromatic amines, and Michael receptors. Furthermore, toxicophore recovery by pertinent feature analysis captures more of the in vitro (53%) and in vivo (56%), rather than of the clinical (8%), endpoints, and indeed uncovers a preference in known toxicophore data towards in vitro and in vivo experimental data. To our knowledge, this is the first contrastive explanation, using both present and absent substructures, for predictions of clinical and in vivo molecular toxicity.
BibTeX
@inproceedings{sharma2023accurate,
  author = {Bhanushee Sharma and Vijil Chenthamarakshan and Amit Dhurandhar and Shiranee Pereira and James Hendler and Jonathan S. Dordick and Payel Das},
  title = {{Accurate Clinical Toxicity Prediction using Multi-task Deep Neural Nets and Contrastive Molecular Explanations}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-31169-8},
}
Figure