Self-Supervised Molecular Pretraining

245 concepts162 papers1 proposal

In chemistry and drug discovery, labeled data is notoriously scarce. Determining whether a molecule is toxic, how it binds to a specific protein, or if it exhibits superconductivity requires expensive, time-consuming laboratory experiments. However, while labeled molecules are rare, unlabeled molecular structures are abundant—spanning public databases with tens of millions of compounds.

The core intuition behind self-supervised molecular pretraining is that molecules possess an inherent "chemical grammar." Just as human language is governed by syntax and semantics, chemical structures are bound by the laws of physics, valency, and geometry. By training deep learning models to master this grammar on massive unlabeled datasets, we can extract rich, generalizable representations that dramatically accelerate downstream prediction tasks where data is limited. As reviewed by Afnan Sultan et al. in Transformers for molecular property prediction: Lessons learned from the past five years, this paradigm shifts the burden of learning chemistry from scarce task-specific labels to large-scale self-supervised objectives.

Molecules as Text: Masked Language Modeling

The simplest way to represent a molecule is as a text string, most commonly using the Simplified Molecular Input Line Entry System (SMILES). This format maps a 2D molecular graph into a linear sequence of characters representing atoms, bonds, and rings.

To learn from these strings without labels, researchers adapt the Masked Language Modeling (MLM) objective popularized by BERT in natural language processing. The model is given a SMILES string with certain tokens hidden or "masked" and is tasked with predicting the missing characters based on the surrounding context.

Early pioneering work by Bonggun Shin et al. in Self-Attention Based Molecule Representation for Predicting Drug-Target Interaction demonstrated that pretraining a Transformer on 97 million unlabeled SMILES strings allows it to learn robust molecular embeddings that significantly boost performance on drug-target interaction tasks. Similarly, Zachary R Fox et al. in Transferring a Molecular Foundation Model for Polymer Property Predictions applied MLM to small molecules to transfer foundational chemical knowledge to polymer property prediction.

By forcing the model to fill in the blanks, it implicitly learns valid valency rules (e.g., that carbon forms four bonds) and identifies common functional groups. Muhammad Arslan Masood et al. in Deep Bayesian Experimental Design for Drug Discovery showed that these high-dimensional string embeddings capture structural knowledge far better than traditional, fixed-length chemical fingerprints, particularly in low-data toxicity assays. This textual intuition can even be extended to higher-level chemical substructures. For instance, Naifeng Wen et al. in A fingerprints based molecular property prediction method using the BERT model treated molecules as "sentences" composed of substructure "words," pretraining a model to predict masked functional groups to capture broader contextual relationships.

Molecules as Graphs: Topology and Motif Masking

While text strings are highly effective, they can struggle with long-range dependencies caused by rings and branching structures. An alternative approach represents molecules as graphs, where nodes are atoms and edges are chemical bonds.

Self-supervised learning on graphs often involves masking individual atoms or entire structural motifs (functional groups) and training Graph Neural Networks (GNNs) or Graph Transformers to reconstruct them. Jiannan Yang et al. in Masking in Molecular Graphs Leveraging Reaction Context introduced a motif-masking strategy that leverages chemical reaction contexts, pooling atom embeddings within a masked motif to predict its identity. To push models further, Han Li et al. in A knowledge-guided pre-training framework for improving molecular representation learning demonstrated that using exceptionally high masking rates (such as 50%) combined with global knowledge guidance forces the network to develop highly robust representations capable of reconstructing identity even when half the molecule is missing.

Beyond direct masking, models can learn from graph topology by predicting computationally derived properties. Jongmin Han et al. in Improving chemical reaction yield prediction using pre-trained graph neural networks pretrained GNNs to predict calculated molecular descriptors (like PCA-reduced physical properties) as pseudo-labels, effectively using established domain knowledge to guide the model's structural intuition before exposing it to real experimental data.

Molecules in 3D Space: Geometric Denoising

Chemical properties are ultimately dictated by how a molecule folds and interacts in three-dimensional space. To capture this, recent architectures incorporate 3D spatial coordinates into the pretraining process.

A powerful objective in this domain is coordinate denoising. The model is given a molecular structure where the 3D atomic positions have been artificially perturbed or "noised," and it must predict the original coordinates. Xiaohong Ji et al. in Exploring Molecular Pretraining Model at Scale successfully scaled this approach up to a 1.1 billion parameter model (Uni-Mol2), combining masked token prediction with 3D coordinate denoising to achieve state-of-the-art downstream performance.

Crucially, high-precision experimental 3D structures are rare, but Xiaomin Fang et al. in Geometry-enhanced molecular representation learning for property prediction discovered that pretraining on simulated, coarse 3D coordinates generated by cheap cheminformatics tools provides massive performance gains. This proves that models can learn fundamental geometric principles without needing flawless experimental data.

However, 3D pretraining must be carefully designed: Chang Wu et al. in 3D-GSRD: 3D Molecular Graph Auto-Encoder with Selective Re-mask Decoding highlighted the risk of "structural leakage," where a model cheats during reconstruction by exploiting 2D topology, and introduced selective re-masking to force the encoder to prioritize true 3D spatial semantics. To manage the computational complexity of 3D data, the author Tuan Le in Navigating the Design Space of Equivariant Diffusion-Based Generative Models for De Novo 3D Molecule Generation demonstrated that pretraining on simplified molecular representations—such as omitting explicit hydrogen atoms—can efficiently transfer broad structural priors before fine-tuning on high-resolution targets.

Cross-Modal Bridges and Fingerprint Pretraining

In many real-world applications, the challenge is not just understanding a clean molecular structure, but mapping it to noisy, experimental measurements like mass spectrometry. Paired data between mass spectra and molecular structures is incredibly scarce.

To overcome this, researchers use molecular fingerprints—binary vectors representing the presence or absence of specific chemical substructures—as an intermediate bridge. Yang Han et al. in MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidation utilized fingerprints to abstract away hardware-dependent noise in mass spectrometry, pretraining a model to translate predicted fingerprints into chemical structures. Similarly, Montgomery Bohde et al. in DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra pretrained a generative decoder on millions of computationally derived fingerprint-structure pairs, teaching the model chemical connectivity rules before exposing it to limited experimental spectra.

Other approaches unify these diverse views directly. Bing Qin 0001 et al. in MolFusion: Multimodal Fusion Learning for Molecular Representations via Multi-granularity Views developed a multi-granularity fusion framework that integrates pre-trained text and graph encoders to capture complementary information. In a broader material science context, Seiji Takeda et al. in Foundation Model for Material Science leveraged contrastive learning to align chemical strings, physical properties, and images into a single, united latent space.

Emergent Intuition and Downstream Success

Why does this massive investment in self-supervised pretraining pay off? Because the internal representations learned by these models capture fundamental chemical realities that transcend individual tasks.

Remarkably, Farzaneh Taleb et al. in Can Transformers Smell Like Humans? discovered that foundation models pretrained purely on unlabeled chemical structures naturally develop internal representations that align with human olfactory perception, allowing them to predict how a molecule smells without any explicit sensory training.

When fine-tuned, these models excel across highly diverse domains. They enable rapid discovery of superconductors from unlabeled crystal structures (Ke Liu et al. in S2SNet: A Pretrained Neural Network for Superconductivity Discovery), allow terrestrial chemical knowledge to generalize to sparse astronomical datasets (Zachary T. P. Fried et al. in Implementation of Rare Isotopologues into Machine Learning of the Chemical Inventory of the Solar-Type Protostellar Source IRAS 16293-2422), and accelerate the development of ionizable lipids for mRNA delivery via contrastive graph pretraining (Yue Xu et al. in AGILE platform: a deep learning powered approach to accelerate LNP development for mRNA delivery). By mastering the universal grammar of chemistry first, self-supervised models turn data-scarce biological and physical bottlenecks into highly manageable fine-tuning tasks.

Key Foundational Work

Self-Attention Based Molecule Representation for Predicting Drug-Target Interaction (2019) — contributed 4 concept elements to this direction.
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction (2020) — contributed 3 concept elements to this direction.
Self-Attention Based Molecule Representation for Predicting Drug-Target Interaction (2019) — contributed 2 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 170 papers
  2. Version 1 · 2026-10-04 · 250 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_self_supervised_molecular_pre_training_ab0e76a4,
  title = {self-supervised molecular pre-training},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-molecular-pre-training/rdr_ab0e76a4c0b047db87599fde263c11e9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_self_supervised_molecular_pre_training_ab0e76a4_v2,
  title = {self-supervised molecular pre-training},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-molecular-pre-training/rdr_ab0e76a4c0b047db87599fde263c11e9/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake