An Image-enhanced Molecular Graph Representation Learning Framework

Hongxin Xiang, Shuting Jin, Jun Xia, Man Zhou, Jianmin Wang

TL;DR

The Image-enhanced Molecular Graph Representation Learning Framework (IEM) breaks the performance bottleneck of traditional graph-based models by using 3D molecular images as "teachers." By distilling visual knowledge—like texture and spatial arrangements—into standard Graph Neural Networks (GNNs), IEM achieves new state-of-the-art results on drug discovery benchmarks without requiring any image data during final inference.

What Problem Does This Paper Address?

In drug discovery, representing molecules accurately is essential. Most current methods use molecular graphs, where atoms are nodes and bonds are edges. While effective, graph-based learning is hitting a plateau. Even when researchers add 3D geometric information (3D graphs), the performance gains are often marginal.

The authors identify two main culprits:

  1. Modality Redundancy: 2D and 3D graphs are mathematically very similar. They use the same encoding logic, meaning they don't offer truly "complementary" information.
  2. The Missing Visual Perspective: Graphs are discrete and symbolic. They miss out on the rich, continuous visual information—texture, shadows, and intuitive spatial arrangements—that a chemist might see when looking at a 3D rendering.

The challenge is: how do we capture this visual "intuition" to improve our models without making the final software too slow or complex to use?

Methods: Image-Guided Distillation

The authors propose a cross-modal knowledge distillation strategy called IEM. The process happens in three main stages:

  1. Pre-training a Knowledgeable Teacher: The researchers rendered 3D molecular images from four different viewpoints for 2 million molecules. They trained a "Teacher" model (using ResNet-18) on tasks including contrastive learning and distribution predictions for atoms, bonds, and geometry.
  2. Cross-Modal Distillation: This is the core innovation. Instead of just forcing a GNN (the "Student") to mimic the teacher's raw features, they use two specialized "enhancers": a Knowledge Enhancer (KE) to align pre-training task understanding and a Task Enhancer (TE) to align final property predictions.
  3. Inference: Once trained, the teacher and the images are discarded. The "Student" GNN now has the visual knowledge "baked in," allowing it to make superior predictions using only simple 2D graph data.
The IEM framework addresses modality redundancy (a) by distilling knowledge from 3D images into molecular graphs (b).

Key Insights and Results

A critical finding is that images provide a fresh perspective that graphs lack. While 2D and 3D graphs have a high Pearson correlation (~0.8-1.0), 3D images have a significantly lower correlation with graphs (~0.2). This low correlation is exactly why images are useful—they offer non-redundant information.

Performance Gains

IEM acts as a universal "plug-in," consistently improving performance across various GNN architectures (GCN, GIN, GAT, and GraphSAGE).

  • New State-of-the-Art: Models like GraphMVP and MoleBERT, when equipped with IEM, achieved new SOTA performance on MoleculeNet, with average ROC-AUC scores of 73.89% and 73.81%, respectively.
  • Consistent Gains: IEM provided an absolute improvement of 1.3% to 3.5% in ROC-AUC across various classification baselines.
  • Regression Success: In property prediction tasks, the framework achieved up to an 8.56% relative improvement in RMSE.
A comparison of performance gains across different baseline models when equipped with IEM.

Limitations and Future Work

While images aren't needed for inference, rendering millions of 3D images for pre-training is computationally expensive. Additionally, bridging the gap between discrete graphs and continuous images remains a complex theoretical challenge. Future directions include applying this framework to even more complex biological entities, such as proteins and RNA, where visual morphology plays an even larger role in function.

Summary updated

BibTeX
@inproceedings{xiang2024image,
  author = {Hongxin Xiang and Shuting Jin and Jun Xia and Man Zhou and Jianmin Wang},
  title = {{An Image-enhanced Molecular Graph Representation Learning Framework}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=X3eln5C2of},
}
Figure