Diffusion-based Molecule Generation with Informative Prior Bridges

Lemeng Wu, Chengyue Gong, Xingchao Liu, Mao Ye, Qiang Liu

TL;DR

This paper introduces Informative Prior Bridges, a novel framework that "steers" diffusion models by injecting physical and statistical constraints directly into the generative process. By using Lyapunov functions to construct these bridges, the authors achieve state-of-the-art results in generating stable 3D molecules and more uniformly distributed 3D point clouds, even with significantly fewer sampling steps.

The Challenge of Uninformative Noise

Deep generative models, particularly diffusion models, have revolutionized 3D molecule and point cloud generation. However, standard diffusion models typically start from "uninformative" noise (like a standard Gaussian) and learn to reverse a corruption process.

The challenge is that molecules aren't just random collections of points; they are governed by strict physical laws, including bond lengths, angles, and electrostatic forces. Similarly, in 3D vision, we want point clouds to be distributed uniformly across a surface to make them useful for downstream tasks like mesh generation. Existing methods often try to bake these constraints into the neural network architecture (e.g., making them SE(3) equivariant), but they rarely incorporate this "prior knowledge" into the underlying stochastic process used for training.

Methodology: Informative Prior Bridges

The authors propose a framework based on Diffusion Bridges—stochastic processes guaranteed to reach a specific data point at a fixed terminal time.

  1. Lyapunov-based Construction: They develop a systematic method using Lyapunov functions to design these bridges. This ensures that the "drift" of the diffusion process is mathematically forced toward valid configurations.
  2. Informative Priors: Instead of a simple "noise-to-data" bridge, they inject "forces" derived from energy functions:
  3. For Molecules: They use AMBER-inspired physical energy (accounting for bonds, angles, and van der Waals forces) and Statistical Energy (learned from the mean and variance of bond lengths/angles in the dataset).
  4. For Point Clouds: They use Riesz Energy (repulsive forces to push points apart) and KNN Distance Energy to promote a uniform distribution.
  5. Training & Architecture: They train an E(n) Equivariant Graph Neural Network (EGNN) to imitate these prior-informed bridge processes using a denoised score matching loss.
Diagram illustrating the generation process where noise is processed through a GNN and guided by energy-based drift to form a molecule.

Key Insights and Ideas

  • Steering vs. Constraining: Rather than just forcing the neural network to be "smarter," this method modifies the training signal itself. By showing the model "physically plausible" paths from noise to data during training, the model naturally learns to generate higher-quality structures.
  • The Power of Bridges: The authors show that standard Score-based Modeling (SMLD) is actually just a special case of their bridge framework using an uninformative bridge.
  • Efficiency in Sampling: Because the prior bridges provide a "stronger" guide, the model remains highly effective even when the number of sampling steps is drastically reduced (e.g., from 1000 steps down to 50 or 100).
  • Uniformity Matters: In point cloud generation, simply adding a repulsive force (Riesz energy) during the diffusion process significantly improves the visual and mathematical regularity of the generated shapes, preventing "clumping" of points.

Datasets and Key Results

The researchers evaluated their approach on three major benchmarks: QM9 (small molecules), GEOM-DRUG (larger, drug-like molecules), and ShapeNet (3D point clouds).

Table showing generation stability metrics on QM9 and GEOM-DRUG datasets, highlighting the superior performance of the Bridge + Force method.

As demonstrated in the evaluation, the "Bridge + Force" method yields notable improvements:

  • Molecule Stability: On the QM9 dataset, the method improved molecular stability (the percentage of molecules where all atoms have correct valency) from 82.0% (the previous state-of-the-art) to 84.6%.
  • Macro-molecules: On the challenging GEOM-DRUG dataset, atom stability improved from 81.3% to 82.4%.
  • Low-Step Performance: In 10-step point cloud generation, their method outperformed standard diffusion models by a wide margin in terms of Minimum Matching Distance (MMD) and Coverage (COV).
  • Low Overhead: Adding these informative priors only increased training time by about 8% and inference time by 3%, making it a highly practical upgrade.

Limitations and Future Work

While highly effective, the authors identify a few areas for improvement. First, the current energy functions do not account for torsional (dihedral) angles, as it is difficult to determine which four atoms are bonded together during the intermediate, noisy stages of the stochastic process. Second, like most diffusion-based methods, training still requires significant computational resources (e.g., 10 days on a V100 GPU for GEOM-DRUG).

Future work aims to generate valid proteins where the number of atoms can vary significantly, and to apply the framework to specific engineering tasks like antibody or vaccine design.

Summary updated

Abstract

AI-based molecule generation provides a promising approach to a large area of biomedical sciences and engineering, such as antibody design, hydrolase engineering, or vaccine development. Because the molecules are governed by physical laws, a key challenge is to incorporate prior information into the training procedure to generate high-quality and realistic molecules. We propose a simple and novel approach to steer the training of diffusion-based generative models with physical and statistics prior information. This is achieved by constructing physically informed diffusion bridges, stochastic processes that guarantee to yield a given observation at the fixed terminal time. We develop a Lyapunov function based method to construct and determine bridges, and propose a number of proposals of informative prior bridges for both high-quality molecule generation and uniformity-promoted 3D point cloud generation. With comprehensive experiments, we show that our method provides a powerful approach to the 3D generation task, yielding molecule structures with better quality and stability scores and more uniformly distributed point clouds of high qualities.
BibTeX
@inproceedings{wu2022diffusion,
  author = {Lemeng Wu and Chengyue Gong and Xingchao Liu and Mao Ye and Qiang Liu},
  title = {{Diffusion-based Molecule Generation with Informative Prior Bridges}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=rc360gS4vJ},
}
Figure