Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction

Christopher Fifty, Joseph M. Paggi, Ehsan Amid, Jure Leskovec, Ron O. Dror

Introduction

Predicting the biological activity of new molecules is a cornerstone of drug discovery, yet it remains one of the most challenging tasks in computational chemistry. The primary bottleneck is the scarcity of high-quality experimental data. Measuring properties like toxicity or binding affinity often requires expensive and time-consuming laboratory work, resulting in "few-shot" datasets where only a handful of labeled examples are available.

While deep learning has advanced molecular property prediction, most existing models represent molecules as 2D graphs. However, biological function is fundamentally a 3D process. Molecules are dynamic entities that adopt various conformations to interact with protein targets. Standard 2D representations struggle to capture these complex 3D geometric and chemical interactions, making it difficult for few-shot learning algorithms to generalize from limited data.

Methodology: The IGNITE Framework

To address this, we introduce IGNITE (Implicit Geometry aNd InTeraction Embedding). The core idea is to leverage physics-based simulations to pre-train molecular representations that "implicitly" understand 3D interactions before they are applied to real-world tasks.

The IGNITE workflow: (I) Pre-training on synthetic docking data, (II) Extracting 3D-aware embeddings, and (III) Applying these embeddings to few-shot learning tasks.

The IGNITE pipeline consists of four main stages:

  1. Synthetic Data Generation: The researchers used the physics-based docking program Glide to simulate how approximately 32,000 molecules bind to over 2,000 diverse protein targets, creating a massive synthetic dataset of binding energies.
  2. Multi-Task Pre-training: A 10-layer Message Passing Neural Network (MPNN) was trained to predict these synthetic binding energies. This task forces the model to learn internal representations that capture the relationship between a molecule's 2D graph and its 3D binding potential.
  3. Embedding Extraction: After pre-training, the final prediction layer is removed. The resulting vector—the IGNITE embedding—serves as a rich, 3D-aware feature set.
  4. Few-Shot Learning: These embeddings are used as a "plug-and-play" replacement for standard features in established few-shot algorithms, such as Model-Agnostic Meta-Learning (MAML), Prototypical Networks (PROTO), and Multi-Task (MT) learning.

Key Insights

  • Implicit 3D Knowledge: IGNITE does not require explicit 3D coordinates at test time. Instead, the embedding space is structured such that molecules with similar 3D binding profiles are clustered together, even if their 2D structures appear different.
  • Bridging the 2D-3D Gap: Traditional chemical fingerprints often fail to recognize similarity between molecules that are structurally diverse in 2D but functionally similar in 3D. IGNITE successfully bridges this gap by learning from the "context" of thousands of potential protein interactions.

Experimental Results

The framework was evaluated on several benchmarks, including the comprehensive FS-Mol dataset (157 test tasks) and classic MoleculeNet datasets (BACE, Tox21, and HIV).

Performance comparison on the FS-Mol benchmark. IGNITE consistently improves performance across different few-shot algorithms and support sizes.

The results demonstrate that IGNITE provides a robust performance boost across various algorithms and data regimes. For instance, on the FS-Mol benchmark with a support size of 16, IGNITE improved the Multi-Task baseline by 40.2%. Similar gains were observed for MAML and Prototypical Networks, confirming that IGNITE is a versatile feature upgrade. On the BACE dataset, IGNITE-MAML achieved performance gains exceeding 20% compared to standard MAML.

Conclusion and Future Work

IGNITE demonstrates that pre-training on large-scale synthetic 3D data can significantly enhance the performance of few-shot molecular property prediction. By embedding implicit geometric and interaction knowledge into the feature space, the model can make more accurate predictions with far less experimental data.

While the current approach relies on physics-based docking, which is computationally intensive, future work could explore using generative diffusion models to create even more accurate or efficient synthetic training data. Scaling this approach to larger molecular libraries remains a promising direction for accelerating drug discovery.

Paper Citation: Christopher Fifty, Joseph M. Paggi, Ehsan Amid, Jure Leskovec, Ron Dror. "Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction." arXiv:2302.02055v2, 2023.

Summary updated

Abstract

Few-shot learning is a promising approach to molecular property prediction as supervised data is often very limited. However, many important molecular properties depend on complex molecular characteristics -- such as the various 3D geometries a molecule may adopt or the types of chemical interactions it can form -- that are not explicitly encoded in the feature space and must be approximated from low amounts of data. Learning these characteristics can be difficult, especially for few-shot learning algorithms that are designed for fast adaptation to new tasks. In this work, we develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations, and a multi-task learning paradigm to structure the embedding space. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance. Our code is available at https://github.com/cfifty/IGNITE.
BibTeX
@misc{fifty2023implicit,
  author = {Christopher Fifty and Joseph M. Paggi and Ehsan Amid and Jure Leskovec and Ron O. Dror},
  title = {{Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction}},
  year = {2023},
  eprint = {2302.02055},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.02055},
}
Figure