MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation

Clément Vignac, Nagham Osman, Laura Toni, Pascal Frossard

TL;DR

MiDi is a novel diffusion model that jointly generates molecular graphs (2D connectivity) and 3D atomic coordinates in a single, end-to-end differentiable process. By simultaneously denoising both the "skeleton" of the molecule and its spatial arrangement, MiDi achieves a 92% stability rate on the complex GEOM-DRUGS dataset—a massive leap over previous state-of-the-art models that struggled to reach even 40%.

What Problem Does This Paper Address?

In drug discovery, a molecule is defined by two things: its molecular graph (which atoms are bonded and how) and its 3D conformation (how those atoms sit in space). Most existing generative models pick a side. They either generate a 2D graph and ignore the 3D shape, or they generate a 3D "cloud" of atoms and rely on hard-coded rules or external software (like OpenBabel) to guess where the bonds should be.

The "guess the bonds" approach has two major flaws:

  1. It’s not differentiable: You can't easily backpropagate through a rule-based bond predictor to improve the 3D generator.
  2. It fails on complex molecules: In large, drug-like molecules, interatomic distances alone aren't enough to determine bond types accurately.

The authors of MiDi argue that 2D and 3D structures are inextricably linked and should be generated together.

Methods

MiDi uses a Mixed Diffusion approach, combining two different types of noise models:

  • Gaussian Diffusion: Used for continuous data (3D coordinates).
  • Discrete Diffusion: Used for categorical data (atom types, formal charges, and bond types).

The "Relaxed" EGNN (rEGNN)

Standard Equivariant Graph Neural Networks (EGNNs) are popular because they respect the symmetries of 3D space (rotation and translation). However, they are strictly translation-invariant. MiDi introduces rEGNN, which exploits the fact that molecules are centered at their center-of-mass (CoM). By using features relative to the CoM (like the distance of an atom from the center), the model gains more descriptive power while remaining equivariant to rotations and translations.

Adaptive Noise Schedule

One of the most clever parts of MiDi is how it handles the "timing" of the noise. The authors found that it’s better to figure out the 3D shape and the bonds before deciding exactly which atom is a Carbon vs. a Nitrogen. They implemented an adaptive noise schedule that adds noise to atom types and charges faster than to coordinates and bonds, forcing the network to prioritize the "big picture" structure during the early stages of denoising.

Datasets

The model was evaluated on two primary benchmarks:

  • QM9: A standard dataset of small molecules (up to 9 heavy atoms).
  • GEOM-DRUGS: A much more challenging dataset featuring drug-sized molecules (averaging 44 atoms, up to 181). This dataset is considered a "stress test" for molecular generators because of its size and chemical complexity.

Key Insights and Ideas

  • Jointness is Key: Generating bonds and coordinates together allows the model to learn the physical constraints of chemistry (e.g., bond angles and lengths) much more effectively than treating them as separate steps.
  • The Hierarchy of Features: Not all molecular features are created equal. The adaptive noise schedule suggests that "structural" features (bonds/positions) should be the foundation upon which "identity" features (atom types) are refined.
  • Differentiability Matters: Because MiDi is end-to-end differentiable, it opens the door for "pocket-conditioned" generation, where a molecule is grown to fit perfectly into a specific protein binding site.

Key Results

The performance gap on the GEOM-DRUGS dataset is the paper's headline result:

  • Stability: MiDi generates 91.6% stable molecules. For comparison, the previous leading 3D model (EDM) only achieved 5.5% stability when using distance-based bond prediction, and 40.3% even when assisted by OpenBabel's optimization algorithms.
  • Validity: MiDi maintained high validity and uniqueness across both datasets, proving it isn't just memorizing the training set but learning the underlying "grammar" of chemistry.
  • 3D Accuracy: MiDi produces significantly more realistic bond angles and lengths compared to models that don't explicitly model bond types during the diffusion process.

Limitations and Future Work

While MiDi is a significant step forward, the authors note a few areas for growth:

  • Conditional Generation: This paper focuses on unconditional generation (creating any valid molecule). The next logical step is conditional generation—designing molecules with specific properties (like solubility or toxicity profiles).
  • Computational Scaling: While MiDi handles GEOM-DRUGS well, the use of Transformers on very large molecular graphs can become computationally expensive.
  • Hydrogen Atoms: Although the model works with explicit hydrogens, the authors acknowledge that in many workflows, hydrogens are added later. Optimizing the model to work with "heavy-atom-only" graphs might further improve efficiency.

Summary updated

Abstract

This work introduces MiDi, a novel diffusion model for jointly generating molecular graphs and their corresponding 3D atom arrangements. Unlike existing methods that rely on predefined rules to determine molecular bonds based on the 3D conformation, MiDi offers an end-to-end differentiable approach that streamlines the molecule generation process. Our experimental results demonstrate the effectiveness of this approach. On the challenging GEOM-DRUGS dataset, MiDi generates 92% of stable molecules, against \(6\%\) for the previous EDM model that uses interatomic distances for bond prediction, and \(40\%\) using EDM followed by an algorithm that directly optimizes bond orders for validity. Our code is available at github.com/cvignac/MiDi.
BibTeX
@inproceedings{vignac2023midi,
  author = {Clément Vignac and Nagham Osman and Laura Toni and Pascal Frossard},
  title = {{MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation}},
  booktitle = {ECML/PKDD},
  year = {2023},
  doi = {10.1007/978-3-031-43415-0\_33},
}

Loading related content...

Figure