Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

Boshra Ariguib, Mathias Niepert, Andrei Manolache

Introduction

In drug discovery and materials science, the ability to predict chemical properties accurately hinges on high-quality molecular representations. While self-supervised learning (SSL) has emerged as a powerful tool to leverage unlabeled data, existing approaches often face significant hurdles. Contrastive learning methods require complex negative sampling strategies to avoid trivial solutions, while generative methods, which reconstruct masked molecular components, are computationally intensive and often neglect critical 3D structural information.

To address these challenges, we introduce C-FREE (Contrast-Free Representation learning on Ego-nets). C-FREE is a self-supervised framework that learns molecular representations by predicting the latent embeddings of local subgraphs from their surrounding neighborhoods. By integrating 2D topology with 3D geometric conformers without the need for negative samples or complex augmentations, C-FREE achieves state-of-the-art performance across various molecular property prediction benchmarks.

Methodology: The C-FREE Framework

C-FREE is inspired by the Joint-Embedding Predictive Architecture (JEPA). It moves away from pixel-level or atom-level reconstruction, focusing instead on predicting high-level representations in a latent space.

$k$-EgoNets as Structural Patches

Rather than masking individual atoms, C-FREE treats molecules as a collection of overlapping subgraphs. The model selects a central node and its $k$-hop neighborhood—termed a $k$-EgoNet—as the "context." The goal is then to predict the latent embedding of the remaining part of the molecule (the "target"). This approach provides a strong inductive bias, capturing local chemical environments effectively.

Multimodal Architecture

The framework employs a hybrid backbone to process both 2D and 3D data. A Graph Isomorphism Network with Edge features (GINE) processes the molecular topology, while a SchNet architecture processes the 3D spatial coordinates. These embeddings are fused by a Transformer to capture global dependencies.

The C-FREE architecture: A multimodal joint-embedding predictive architecture (JEPA) that integrates 2D topology and 3D geometry.

Asymmetric Training and Latent Prediction

To prevent "representation collapse"—where the model learns to output a constant vector—C-FREE utilizes an asymmetric training strategy. It includes a Predictor network and a target encoder updated via Exponential Moving Average (EMA). The model predicts the embedding of the target subgraph rather than its raw structure, making the task more computationally efficient and focused on semantic features.

Experimental Results

C-FREE was pretrained on the GEOM dataset, which contains 0.33 million molecules with diverse 3D conformations. Its performance was then evaluated on the MoleculeNet benchmark, as well as regression tasks like Kraken and Drugs-75K.

State-of-the-Art Performance

C-FREE consistently outperformed existing contrastive (e.g., GraphCL, JOAO) and non-contrastive (e.g., AttrMask, EdgePred) baselines. Notably, it achieved the highest average performance across the MoleculeNet suite, demonstrating the robustness of its learned representations.

Performance comparison of C-FREE against various self-supervised baselines on the MoleculeNet benchmark using a linear probe evaluation.

Key Insights

  • The Power of 3D Pretraining: One of the most significant findings is that incorporating 3D conformers during pretraining boosts performance even when only 2D data is available during downstream fine-tuning.
  • Expressiveness Beyond 1-WL: C-FREE is mathematically more expressive than the standard 1-WL graph isomorphism test, allowing it to distinguish subtle structural differences that other GNNs might miss.
  • Label Efficiency: In scenarios with extremely limited labeled data (e.g., 1% labels), C-FREE provides a substantial performance lift, making it highly valuable for real-world drug discovery where experimental data is scarce.

Conclusion

C-FREE represents a significant step forward in molecular SSL by combining a contrast-free predictive objective with multimodal 2D/3D integration. Its efficiency, coupled with its superior performance in low-data regimes, makes it a promising tool for chemical property prediction. Future work will explore scaling the model to larger datasets and incorporating additional modalities to further enhance its representational power.

Summary updated

BibTeX
@inproceedings{ariguib2026learning,
  author = {Boshra Ariguib and Mathias Niepert and Andrei Manolache},
  title = {{Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=Vbj1jSaqTo},
}
Figure