Geometric Denoising for Molecular Foundation Models

This direction is split into other directions: coordinate denoising. Its last version stays readable and citable here.

Version 1 of 186 papers103 conceptsfrozen 2026-10-04

In the quest to design new drugs, catalysts, and materials, researchers face a daunting "data bottleneck." While we have access to millions of unlabeled 3D molecular structures, the experimental labels required to train traditional models—such as binding affinity, toxicity, or crystal stability—are expensive and slow to acquire.

To bridge this gap, a powerful new paradigm has emerged: Geometric Denoising. By perturbing the 3D coordinates of atoms and training models to "clean" them, researchers are teaching neural networks the fundamental laws of physics and chemistry without a single human-provided label.

The Intuition: Denoising as Force Approximation

At its heart, geometric denoising is a simple game of "put it back where it belongs." A researcher takes a stable molecular structure and applies stochastic noise—jiggling the atoms slightly out of their equilibrium positions. The model is then tasked with predicting the restorative vector needed to return each atom to its original spot.

Why does this work? In physical chemistry, a molecule at equilibrium sits at a local minimum of a Potential Energy Surface (PES). When you move an atom away from this minimum, the physical forces of the system naturally want to push it back. As Jun Xia and colleagues note in their survey on Chemical Pre-trained Models, this denoising task forces the model to implicitly learn the underlying physics of molecular conformations and the forces governing atomic interactions.

By learning to denoise, the model isn't just memorizing shapes; it is learning to approximate the gradient of the energy landscape. As Rui Feng and Qi Zhu demonstrate with ET-OREO, the resulting denoising vectors serve as a proxy for atomic forces, allowing models to simulate molecular dynamics significantly faster than traditional methods.

Respecting Physical Symmetries

A molecule is the same molecule whether it is rotated 90 degrees or moved three inches to the left. For a denoising model to be effective, it must be equivariant: if the input structure rotates, the predicted restorative forces must rotate accordingly.

Rui Jiao and Wenbing Huang addressed this with 3D-EMGP, a framework that uses equivariant energy-based backbones. By ensuring the learned representations respect these geometric constraints, the model captures the complex energy landscapes of molecules more robustly than 2D graph-based methods.

Beyond Cartesian Coordinates

While jiggling individual atoms is effective, it can sometimes ignore the rigid constraints of chemistry, like bond lengths and angles. Recent work has moved toward more sophisticated noise strategies:

  • Internal Coordinates: Yuyan Ni and Shikun Feng introduced SliDe, which denoises bond lengths and angles rather than raw 3D coordinates. This ensures the model captures fundamental physical constraints like bond stretching and bending.
  • Fractional Denoising: In their work on Fractional Denoising, Shikun Feng suggests that removing only a subset of noise (like coordinate displacements) while ignoring others (like dihedral rotations) helps maintain a valid physical force field.
  • Block-Level Denoising: For large proteins, perturbing every atom is computationally heavy. Rui Jiao and Xiangzhe Kong developed the Equivariant Pretrained Transformer (EPT), which perturbs entire structural blocks as rigid bodies. This forces the network to capture long-range spatial relationships while preserving local chemical constraints.

Scaling to the Biological Universe

The success of geometric denoising has led to the creation of "foundation models" for biology and materials science. Models like Sable by Qiwei Ye and Uni-Mol2 by Xiaohong Ji demonstrate that scaling these denoising objectives to millions of structures leads to emergent understanding.

In the realm of proteins, Wei Wu and Chao Wang show with SEPIT that pre-training an encoder to reconstruct 3D coordinates enhances a model's ability to describe protein functions even when 3D data is unavailable during inference. Similarly, Zixing Song applied these concepts to crystals, using a diffusion-based pre-training framework to learn the lattice constraints that define stable materials.

A Universal Prior for Spatial Data

Perhaps the most surprising aspect of geometric denoising is its versatility. The same intuition—that learning to reconstruct structure teaches a model about the "rules" of a system—is being applied outside of chemistry. Shuang Zeng and the team behind PriorDrive use coordinate denoising on vectorized road data to help autonomous vehicles "see" through occlusions by leveraging learned geometric priors of road layouts.

By treating 3D coordinates not just as data points, but as the result of underlying physical and structural constraints, geometric denoising has provided a way for AI to learn the "common sense" of the physical world.

Papers in this version

Showing 60 of 86 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_geometric_denoising_for_molecular_founda_75ca701f,
  title = {Geometric Denoising for Molecular Foundation Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-denoising-for-molecular-foundation-models/rdr_75ca701f77134074b76c345ea1ac45ce},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_geometric_denoising_for_molecular_founda_75ca701f_v1,
  title = {Geometric Denoising for Molecular Foundation Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-denoising-for-molecular-foundation-models/rdr_75ca701f77134074b76c345ea1ac45ce/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}