In 3D molecular representation learning, denoising has become a dominant pre-training paradigm. The core idea is elegant: take a valid 3D molecular conformation, perturb the atomic coordinates with random noise, and train a neural network to predict and reverse that noise.
Mathematically, this process is deeply tied to learning a molecule's underlying potential energy surface. The score function (the gradient of the log data distribution) estimated by denoising models corresponds directly to the physical forces acting on the atoms.
However, standard diffusion and denoising models suffer from a fundamental physical flaw: they apply uniform, isotropic Gaussian noise across all atomic coordinates. This treats a molecule as a "bag of independent points," ignoring the directional constraints of chemical bonds. In reality, molecular energy landscapes are highly anisotropic—meaning the energy cost of moving an atom depends entirely on the direction of the movement and its local chemical environment.
Isotropic Noise (Standard) Anisotropic Noise (Physics-Informed)
. : . \ | /
: o : : o=====o
' : ' / | \
(Equal perturbation in (Perturbation respects bond
all directions) stiffness and geometry)
By forcing models to denoise unphysical, spherically symmetric perturbations, standard architectures struggle to learn accurate force fields. A new wave of research addresses this by designing or learning structure-aware, anisotropic noise distributions that respect the laws of molecular physics.
The Breakdown of Isotropic Denoising
When we apply uniform Gaussian noise to a molecule, we assume that every atom can fluctuate equally in any direction. But a molecule is held together by stiff covalent bonds, rigid aromatic rings, and flexible rotatable chains.
As Yuyan Ni et al. point out, the theoretical equivalence between denoising and learning a molecule's force field only holds if the noise distribution matches the physical properties of the system. When uniform random displacement is applied without considering classical mechanical principles, the model is forced to learn from configurations that violate basic chemical realities, causing its predicted forces to deviate significantly from quantum chemical ground truths.
This mismatch manifests in several ways:
- Deformation vs. Conformation: Uniform noise cannot distinguish between high-energy atomic vibrations (like stretching a stiff Carbon-Carbon double bond) and low-energy conformational changes (like rotating a dihedral angle). As a result, models waste capacity learning to fix impossible bond stretches rather than mastering global shape variations.
- Structural Bias: Applying the same noise scale to a rigid benzene ring and a floppy alkane chain introduces systematic physical biases. As Zhiran Hou et al. argue, treating all structural components with the same level of perturbation ignores the distinct stability and vibration profiles of different molecular geometries.
- Distributional Drift: Zihan Zhou et al. observed that during standard coordinate-based noising, the distribution of inter-atomic distances shifts from a Gaussian to a Maxwell-Boltzmann distribution. Failing to account for this shifting distribution means the generative process remains physically incomplete.
To build better representations, the denoising process must be constrained by the molecule's local environment and structural topology.
Strategy 1: Hardcoding Physical Degrees of Freedom
The first intuitive solution is to replace Cartesian coordinate jiggling with perturbations that operate directly on a molecule's internal degrees of freedom or functional substructures.
Instead of treating atoms independently, Jiying Zhang et al. propose subgraph-level diffusion, recognizing that atoms are organized into stable, functional units. By respecting these substructures, the model avoids destroying the core chemical identity of the molecule during the noising process.
Taking a classical mechanics perspective, Yuyan Ni et al. introduced SliDe (Sliced Denoising), which replaces generic coordinate noise with explicit perturbations of bond lengths, bond angles, and torsional dihedrals. By operating in the space of internal coordinates, the noise naturally adapts to the varying stiffness of different chemical bonds.
Cartesian Noise Internal Coordinate Noise
(Unconstrained) (Physics-Preserving)
o --> [Noise] o ~ ~ ~ o (Bond Stretch)
/ / \
o o o (Angle Bend)
Similarly, Shikun Feng et al. developed Fractional Denoising (Frad), which applies a combination of rigid-body dihedral angle rotations and local coordinate displacements, but only tasks the model with reconstructing the latter. This clever split allows the model to implicitly approximate an anisotropic force field, successfully separating low-energy conformational changes from high-energy structural deformations.
Strategy 2: Learning Environment-Aware Noise
While hardcoding physical constraints works well for simple molecules, complex materials and macro-molecules possess intricate, non-local environments that are difficult to model with manual rules. A more flexible alternative is to let the neural network learn an atom-specific, directional noise distribution based on the local geometry.
Instead of a scalar variance ($\sigma^2 \mathbf{I}$) that creates a sphere of noise, these methods assign a full covariance matrix ($\mathbf{\Sigma}_i$) to each atom $i$. The eigenvectors of this matrix define an ellipsoid of noise, oriented to allow more variance in flexible directions (like empty space) and less variance in rigid directions (like along a chemical bond).
Standard Noise Variance: Anisotropic Covariance Matrix:
[ σ² 0 0 ] [ Σxx Σxy Σxz ]
[ 0 σ² 0 ] [ Σyx Σyy Σyz ]
[ 0 0 σ²] [ Σzx Σzy Σzz ]
(Spherical Noise) (Ellipsoidal Noise)
Xixian Liu et al. developed AniDS, a framework that generates atom-specific covariance matrices by looking at an atom's neighboring interactions. This allows the noise distribution to dynamically adapt its orientation and magnitude based on the local geometric environment, capturing the directional stiffness of covalent bonds.
In a similar vein, Yurou Liu et al. proposed DenoiseVAE, which embeds a learnable Noise Generator within a Variational Autoencoder framework. The generator takes the equilibrium conformation of a molecule and predicts atom-specific noise distributions. This architecture automatically learns that atoms inside rigid rings should receive tightly constrained perturbations, while atoms in open, flexible chains can tolerate larger displacements.
Finally, Zhiran Hou et al. utilize an "Energy Sentinel" in their framework, MOES-Pred. The sentinel adaptively tailors the noise based on motif-level chemical priors and local stability, ensuring that the model avoids introducing systematic physical biases during pre-training.
Why It Works: Aligning Tasks with Reality
Anisotropic denoising works because it fixes a fundamental data-efficiency problem in self-supervised learning: task alignment.
When a model is trained with isotropic noise, it spends most of its capacity learning to solve unphysical tasks—such as trying to fix a bond that has been stretched to a chemically impossible length. Because the model is overwhelmed by these high-energy, unphysical perturbations, it fails to develop a fine-grained understanding of the subtle, low-energy equilibrium fluctuations that dictate actual molecular properties and drug interactions.
By shifting to anisotropic, environment-aware noise, the pre-training task is forced to mirror the true potential energy surface of the molecule. The model learns to respect the boundaries of chemical structures, resulting in representations that naturally capture the directional, complex physics governing molecular behavior.
Key Foundational Work
Fractional Denoising for 3D Molecular Pre-training (2023) — contributed 4 concept elements to this direction.
Sliced Denoising: A Physics-Informed Molecular Pre-Training Method (2024) — contributed 2 concept elements to this direction.
SubgDiff: A Subgraph Diffusion Model to Improve Molecular Representation Learning (2024) — contributed 1 concept elements to this direction.