Accurate prediction of molecular targets using a self-supervised image representation learning framework

Xiangxiang Zeng, Hongxin Xiang, Linhui Yu, Jianmin Wang, Kenli Li, Ruth Nussinov, Feixiong ChengResearch Square 2022 · DOI

TL;DR

This paper introduces ImageMol, a novel self-supervised deep learning framework that learns molecular representations from images of molecules. ImageMol demonstrates high accuracy in predicting molecular properties and targets, including identifying potential COVID-19 treatments, and outperforms existing state-of-the-art methods.

What Problem Does This Paper Address?

Discovering new drugs is a complex and expensive process, largely due to the challenge of accurately predicting a drug's efficacy and safety, which is determined by its molecular targets. Traditional experimental methods for evaluating these targets across a vast number of compounds are impractical. While computational approaches, particularly those leveraging Artificial Intelligence (AI), offer a promising solution to reduce costs and time, a key challenge lies in effectively learning meaningful representations from chemical structures. Existing methods often rely on hand-crafted features or require extensive domain knowledge, limiting their ability to capture the full spectrum of molecular information. This paper aims to address this by developing a novel framework that learns rich molecular representations from molecular images using self-supervised learning.

Methods

The core of this work is the ImageMol framework, which utilizes a self-supervised, image-based approach for learning molecular representations. Key components include:

  • Molecular Encoder: A deep learning model (specifically ResNet18) is employed to extract latent features from molecular images.
  • Pre-training Strategies: Five pretext tasks are designed to optimize the molecular encoder's latent representations by considering chemical knowledge and structural information from the images. These tasks are based on three principles: consistency, relevance, and rationality.
  • Multi-Granularity Chemical Clusters Classification (MG3C): Predicts chemical structural information by clustering molecules based on their fingerprints.
  • Molecular Image Reconstruction (MIR): Reconstructs molecular images from latent features using a Generative Adversarial Network (GAN) framework.
  • MASK-based Contrastive Learning (MCL): Learns representations by maximizing similarity between masked and unmasked molecular images.
  • Molecular Rationality Discrimination (MRD): Distinguishes between "rational" (chemically sensible) and "irrational" molecular image arrangements.
  • Jigsaw Puzzle Prediction (JPP): Predicts the correct permutation of shuffled image patches.
  • Fine-tuning: The pre-trained molecular encoder is then fine-tuned on specific downstream tasks (e.g., molecular property prediction, drug metabolism prediction, anti-viral activity prediction) by adding a fully connected layer.

What's novel here is the use of molecular images as the primary input for learning representations, combined with a suite of self-supervised pretext tasks that incorporate chemical intuition and structural understanding, moving beyond traditional graph or sequence-based methods.

Datasets

The paper utilizes several large-scale datasets for pre-training and evaluation:

  • Pre-training:
    • ZINC: Approximately 8 million unlabeled drug-like molecules.
  • ChEMBL: Approximately 0.43 million unlabeled molecules.
  • Downstream Tasks (Benchmark Datasets):
    • Molecular Property Prediction: Five binary classification datasets from MoleculeNet: Tox21, ClinTox, BBBP, HIV, and BACE.
  • Drug Metabolism Prediction: PubChem Data Set I (Training) and PubChem Data Set II (Validation) for human cytochrome P450 (CYP) inhibition.
  • Anti-SARS-CoV-2 Activities Prediction: 13 high-throughput experimental datasets from the National Center for Advancing Translational Sciences (NCATS) COVID-19 portal.

Key Insights and Ideas

  • Molecular Images as Rich Representations: The study demonstrates that molecular images, when processed with a deep learning framework, can capture highly informative structural and chemical features for downstream tasks, often outperforming traditional methods.
  • Self-Supervision for Chemical Intuition: The designed pretext tasks (rationality, consistency, relevance) effectively imbue the model with a form of "chemical common sense," allowing it to learn more robust and interpretable molecular representations without explicit labels.
  • Bridging the Gap in Drug Discovery: ImageMol offers a powerful, generalizable tool that can accelerate drug discovery by accurately predicting molecular properties and identifying potential drug candidates, particularly for emerging diseases like COVID-19.
  • Interpretability through Visualization: Techniques like Grad-CAM and t-SNE are used to visualize what ImageMol "learns," showing that it focuses on biologically relevant structural features rather than spurious patterns.

Key Results

  • High Accuracy Across Tasks: ImageMol achieved high Area Under the Curve (AUC) values across various benchmark datasets for molecular property prediction (e.g., HIV AUC=0.821, BACE AUC=0.902, BBBP AUC=0.931, Tox21 AUC=0.809).
  • Superior Performance: ImageMol consistently outperformed state-of-the-art methods, including sequence-based (e.g., SMILES Transformer), graph-based (e.g., Jure's GNN), and other image-based methods (e.g., Chemception, ADMET-CNN, QSAR-CNN) on multiple tasks. For instance, it showed an average performance advantage of 8.5% across 13 SARS-CoV-2 targets compared to Jure's GNN.
  • Effective SARS-CoV-2 Inhibitor Identification: ImageMol successfully identified potential anti-SARS-CoV-2 molecules, with 65% of the top 20 predicted drugs validated by experimental and clinical evidence. It also identified 6 out of 10 known 3CL protease inhibitors with 60% success rate in external validation.
  • Robustness to Imbalanced Data: ImageMol demonstrated better capability on datasets with extreme imbalances of positive and negative samples compared to traditional deep learning and machine learning approaches.
  • Pre-training Impact: Pre-training with larger datasets (up to 8.5 million molecules) significantly improved performance, with average ROC-AUC increasing from 75.7% (no pre-training) to 85.9% (with all 3 pretext tasks).

Limitations and Future Work

  • Sparsity of Molecular Images: The paper acknowledges that molecular images are often sparse (over 90% blank space), which can affect latent feature extraction. While data augmentation was used, optimizing for sparsity was not a primary focus.
  • Explicit Chemical Properties: Explicitly defining chemical properties of atoms and bonds remains challenging compared to graph-based methods, potentially leading to insufficient chemical information.
  • Future Directions:
    • Larger-Scale Data and Models: Integrating larger biomedical datasets and employing larger capacity models (e.g., Vision Transformers - ViT) for molecular images.
  • Multi-View Learning: Combining molecular images with other representations like SMILES strings and graphs.
  • Enhanced Chemical Knowledge Integration: Introducing more explicit chemical knowledge (atomic properties, 3D information) into the image or pixel level.

Summary updated

Abstract

The clinical efficacy and safety of a drug is determined by its molecular targets in the human proteome. However, proteome-wide evaluation of all compounds in human, or even animal models, is challenging. In this study, we present an unsupervised pre-training deep learning framework, termed ImageMol, from 8.5 million unlabeled drug-like molecules to predict molecular targets of candidate compounds. The ImageMol framework is designed to pretrain chemical representations from unlabeled molecular images based on local- and global-structural characteristics of molecules from pixels. We demonstrate high performance of ImageMol in evaluation of molecular properties (i.e., drug's metabolism, brain penetration and toxicity) and molecular target profiles (i.e., human immunodeficiency virus) across 10 benchmark datasets. ImageMol shows high accuracy in identifying anti-SARS-CoV-2 molecules across 13 high-throughput experimental datasets from the National Center for Advancing Translational Sciences (NCATS) and we re-prioritized candidate clinical 3CL inhibitors for potential treatment of COVID-19. In summary, ImageMol is an active self-supervised image processing-based strategy that offers a powerful toolbox for computational drug discovery in a variety of human diseases, including COVID-19.
BibTeX
@inproceedings{zeng2022accurate,
  author = {Xiangxiang Zeng and Hongxin Xiang and Linhui Yu and Jianmin Wang and Kenli Li and Ruth Nussinov and Feixiong Cheng},
  title = {{Accurate prediction of molecular targets using a self-supervised image representation learning framework}},
  booktitle = {Research Square},
  year = {2022},
  doi = {10.21203/rs.3.rs-1477870/v1},
}
Figure