Harnessing Simulation for Molecular Embeddings

Christopher Fifty, Joseph M. Paggi, Ehsan Amid, Jure Leskovec, Ron O. DrorarXiv 2023 · OpenReview

Abstract

Predicting molecular properties is a cornerstone of drug discovery, yet experimental data is often prohibitively scarce. While few-shot learning offers a potential solution, standard molecular representations—typically 2D graphs—fail to capture the critical 3D geometric interactions that dictate biological activity. We introduce IGNITE (Implicit Geometry aNd InTeraction Embedding), a method that leverages physics-based simulations to pre-train geometry-aware embeddings. By training on massive synthetic docking datasets, IGNITE enables models to infer complex 3D physical rules from 2D structures, significantly outperforming traditional methods in low-data regimes.

Introduction: The Challenge of Molecular Data Scarcity

A primary hurdle in drug discovery is the extreme scarcity of experimental data. Determining a molecule's toxicity or its binding affinity to a target protein often requires weeks of expensive laboratory work, resulting in datasets with only dozens or hundreds of samples.

Few-shot learning (FSL) is designed for such scenarios, but its application to chemistry is limited by how molecules are represented. Most AI models treat molecules as 2D graphs of atoms and bonds. However, biological activity is a 3D "lock and key" mechanism. Properties depend on 3D conformations and specific chemical interactions, such as hydrogen bonds. Standard 2D features do not explicitly encode this information, forcing FSL models to attempt to "guess" complex physics from a handful of examples—a task that is often insurmountable.

The IGNITE Framework

To address this, we developed IGNITE. The core philosophy is to use synthetic data to perform the heavy lifting of learning molecular physics before the model ever sees a real-world experimental task.

The IGNITE workflow: pre-training on synthetic docking data to extract geometry-aware embeddings for few-shot learning.

The IGNITE pipeline consists of four main stages:

  1. Synthetic Data Generation: We utilized Glide, a physics-based docking program, to simulate the binding of approximately 32,000 molecules to 1,600 distinct protein structures, generating a massive dataset of binding energies.
  2. Multi-Task Pre-training: A 10-layer Message Passing Neural Network (MPNN) was trained to predict these simulated energies. To succeed, the MPNN must learn an internal representation that captures the underlying 3D geometry and interactions used by the docking software.
  3. Embedding Extraction: After training, the final prediction layer is removed. The resulting internal vector—the graph representation—serves as the IGNITE embedding.
  4. Few-Shot Adaptation: These embeddings are then used as input features for standard FSL algorithms like MAML, Prototypical Networks, and Multi-Task learning. This "plug-and-play" approach requires minimal modification to existing architectures.

Evaluation and Key Results

We evaluated IGNITE on the comprehensive FS-Mol benchmark and several MoleculeNet datasets (BACE, Tox21, HIV) adapted for few-shot settings.

IGNITE demonstrated near-universal performance gains, particularly in the most data-constrained scenarios. On the FS-Mol benchmark, using IGNITE embeddings with a Multi-Task (MT) baseline improved the $\Delta$AUPRC by 40.2% when only 16 training examples were available.

Performance comparison on the FS-Mol benchmark across different few-shot algorithms and support sizes.

Key findings include:

  • Algorithm Agnosticism: IGNITE consistently boosted performance across different meta-learning algorithms (MAML, Prototypical Networks) by 3-8% on FS-Mol, suggesting that representation quality is a primary driver of success.
  • Significant Gains on Real-World Tasks: On the BACE dataset, IGNITE-MAML showed a 21.6% improvement over standard MAML at the 16-shot level. On the HIV dataset, the improvement reached 32.1%.

Discussion: Simulation as a Teacher

The success of IGNITE highlights that synthetic data does not need to be a perfect replica of reality to be highly effective. While docking is a computational approximation, it encodes the fundamental rules of 3D interactions. Pre-training on these rules provides the model with a "biophysical intuition" that 2D fingerprints lack.

Furthermore, IGNITE learns to infer the importance of geometry from the 2D graph alone, bypassing the need for explicit 3D coordinates which are often unavailable for new molecules. Qualitative analysis confirmed that IGNITE correctly maps molecules with similar 3D binding profiles close together in the embedding space, even when their 2D structures appear unrelated.

Limitations and Future Work

The primary limitation is that embeddings are inherently tied to the quality of the synthetic data; biases in the docking software may be inherited. Additionally, generating the initial synthetic dataset is computationally intensive. Future research may explore using modern generative diffusion models to create even more accurate synthetic training data and scaling to larger, more expressive neural architectures.

Summary updated

BibTeX
@misc{fifty2023harnessing,
  author = {Christopher Fifty and Joseph M. Paggi and Ehsan Amid and Jure Leskovec and Ron O. Dror},
  title = {{Harnessing Simulation for Molecular Embeddings}},
  year = {2023},
  url = {https://openreview.net/forum?id=qbX5VVvjAQ},
}
Figure