Abstract
Extracting rich molecular representation is a crucial prerequisite for accurate drug discovery. Recent molecular representation learning methods achieve impressive progress, but the paradigm of learning from a single modality gradually encounters the bottleneck of limited representation capabilities. In this work, we fully consider the rich visual information contained in 3D conformation molecular images (i.e., texture, shadow, color and planar spatial information) and distill graph-based models for more discriminative drug discovery. Specifically, we propose an image-enhanced molecular graph representation learning framework (IEM) that leverages multi-view molecular images rendered from 3D conformations to boost molecular graph representations. To extract useful auxiliary knowledge from multi-view images, we design a teacher pre-trained on 2 million molecules. We transfer knowledge via an efficient cross-modal knowledge distillation strategy. IEM significantly improves the capabilities of existing graph-based models like GIN, EdgePred, GraphMVP, and MoleBERT. Notably, GraphMVP and MoleBERT equipped with IEM achieve new state-of-the-art performance on the MoleculeNet benchmark, achieving average 73.89% and 73.81% ROC-AUC, respectively. Code is available at https://github.com/HongxinXiang/IEM.
TL;DR
The Image-enhanced Molecular Graph Representation (IEM) framework boosts the performance of standard graph neural networks (GNNs) by "teaching" them to see molecules as 3D images. By distilling visual knowledge from multi-view renderings into graph-based models, IEM achieves new state-of-the-art results on the MoleculeNet benchmark without requiring any extra computational overhead during inference.
Introduction
In the field of drug discovery, the accuracy of molecular representation is paramount. Traditionally, researchers have relied on graph-based models where atoms and bonds are represented as nodes and edges. While effective, these models often reach a "representation bottleneck." The authors observe that 2D and 3D graph representations are highly correlated, leading to diminishing returns when combined.
In contrast, 3D molecular images—renderings that include texture, shadow, and color—offer a fundamentally different perspective. These images capture spatial arrangements and chirality in ways that discrete graphs may struggle to encode. The core challenge addressed by this research is how to integrate this "visual intuition" into graph-based models without increasing computational overhead during inference.
The IEM Framework
The proposed Image-enhanced Molecular Graph Representation (IEM) framework utilizes a "Teacher-Student" architecture to bridge the gap between visual and graph modalities.
1. The Visual Teacher
The "Teacher" is a 3D image encoder (ResNet-18) pre-trained on the PCQM4Mv2 dataset, which contains 2 million molecules with 3D conformations. The teacher is trained using five specialized tasks, including contrastive learning (aligning 2D and 3D views) and distribution prediction of molecular properties.
2. Cross-Modal Knowledge Distillation (CMKD)
To transfer knowledge to the "Student" (a standard GNN), the authors employ a CMKD strategy featuring two enhancers:
- Knowledge Enhancer: Aligns the student's internal representations of molecular attributes with the teacher's.
- Task Enhancer: Aligns the student’s final predictions for specific tasks with the teacher’s predictions.
Importantly, the teacher and image-processing components are only used during training. During inference, the student operates as a standard, high-performance GNN.
Key Insights and Results
A critical finding of this study is the "modality gap." As shown in Figure 6(a), while 2D and 3D graphs are closely related (high Pearson correlation), 3D images show a low correlation with 2D graphs. This suggests that images provide complementary information. Furthermore, 3D images alone demonstrate superior prediction ability (lower RMSE) for certain chemical properties compared to graph-only models.
Performance on MoleculeNet
The IEM framework was evaluated on the MoleculeNet benchmark across 8 classification and 4 regression tasks.
- State-of-the-Art Results: Models like GraphMVP and MoleBERT, when enhanced with IEM, achieved average ROC-AUC scores of 73.89% and 73.81%, respectively.
- Consistent Gains: Across 8 classification tasks, IEM provided an absolute improvement of 1.3% to 3.5% over baseline models (Figure 6c).
- Regression Accuracy: On regression tasks, IEM achieved up to an 8.56% relative improvement in RMSE on the ESOL dataset.
Conclusion
The IEM framework demonstrates that visual information from 3D conformations can significantly enhance molecular graph representations through knowledge distillation. By treating the modality gap as a source of complementary information rather than a hurdle, IEM sets new benchmarks for drug discovery tasks. Future work may extend this approach to more complex biological structures like proteins and RNA, where visual data is even more abundant.
Summary updated