MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation

Clément Vignac, Nagham Osman, Laura Toni, Pascal Frossard

Abstract

This work introduces MiDi, a novel diffusion model for jointly generating molecular graphs and their corresponding 3D arrangement of atoms. Unlike existing methods that rely on predefined rules to determine molecular bonds based on the 3D conformation, MiDi offers an end-to-end differentiable approach that streamlines the molecule generation process. Our experimental results demonstrate the effectiveness of this approach. On the challenging GEOM-DRUGS dataset, MiDi generates 92% of stable molecules, against 6% for the previous EDM model that uses interatomic distances for bond prediction, and 40% using EDM followed by an algorithm that directly optimize bond orders for validity.

Introduction

In drug discovery, a molecule is defined by two critical components: its molecular graph (the 2D connectivity of atoms and bonds) and its 3D conformation (the spatial arrangement of those atoms). Traditionally, generative models have treated these as separate problems. Some generate a 2D graph and ignore the 3D shape, while others generate a 3D "cloud" of atoms and rely on non-differentiable, rule-based software to guess bond types post-hoc.

These decoupled approaches struggle with complex, drug-like molecules where interatomic distances alone are insufficient to accurately predict bond orders. MiDi addresses this by treating 2D and 3D structures as inextricably linked, generating both simultaneously in a single, end-to-end differentiable process.

3D and 2D views of molecules generated by MiDi, illustrating the joint modeling of spatial coordinates and chemical bonds.

Methodology

Mixed Diffusion and rEGNN

MiDi utilizes a Mixed Diffusion approach to handle the heterogeneous nature of molecular data. It employs Gaussian Diffusion for continuous 3D coordinates and Discrete Diffusion for categorical data, including atom types, formal charges, and bond types.

To process this data, the authors introduce the Relaxed Equivariant Graph Neural Network (rEGNN). While standard EGNNs are strictly translation-invariant, rEGNN exploits the fact that molecules are centered at their center-of-mass (CoM). By incorporating features relative to the CoM, the model gains descriptive power while maintaining necessary equivariance to rotations and translations.

Adaptive Noise Schedule

A key innovation in MiDi is the adaptive noise schedule. The researchers discovered that the model performs best when it establishes the structural "skeleton" (bonds and coordinates) before finalizing the specific identity of the atoms.

The adaptive cosine noise schedule shows that information regarding atom types is removed faster than information about bonds and coordinates during the diffusion process.

As shown in the noise schedule, information about atom types and charges is added faster than noise for coordinates and bonds. This hierarchy forces the network to prioritize the global structure during the early stages of the denoising process.

Experimental Results

The model was benchmarked against the standard QM9 dataset and the more rigorous GEOM-DRUGS dataset, which contains large molecules averaging 44 atoms. The results on GEOM-DRUGS highlight a significant performance leap:

  • Stability: MiDi achieved a 91.6% stability rate. In contrast, the previous state-of-the-art 3D model (EDM) reached only 5.5% stability using standard distance-based bond prediction, and only 40.3% when assisted by specialized optimization algorithms.
  • Validity and Accuracy: MiDi maintained high chemical validity and produced significantly more realistic bond angles and lengths compared to models that do not explicitly model bond types during diffusion.

Conclusion and Future Work

MiDi demonstrates that joint generation of 2D and 3D molecular structures is not only feasible but superior for generating stable, drug-like molecules. Its end-to-end differentiable nature provides a foundation for future work in "pocket-conditioned" generation, where molecules are designed to fit specific protein binding sites. Future iterations may focus on scaling to even larger graphs and incorporating conditional generation for specific pharmacological properties like solubility or toxicity.

Summary updated

Abstract

This work introduces MiDi, a novel diffusion model for jointly generating molecular graphs and their corresponding 3D arrangement of atoms. Unlike existing methods that rely on predefined rules to determine molecular bonds based on the 3D conformation, MiDi offers an end-to-end differentiable approach that streamlines the molecule generation process. Our experimental results demonstrate the effectiveness of this approach. On the challenging GEOM-DRUGS dataset, MiDi generates 92% of stable molecules, against 6% for the previous EDM model that uses interatomic distances for bond prediction, and 40% using EDM followed by an algorithm that directly optimize bond orders for validity. Our code is available at github.com/cvignac/MiDi.
BibTeX
@inproceedings{vignac2023midi,
  author = {Clément Vignac and Nagham Osman and Laura Toni and Pascal Frossard},
  title = {{MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation}},
  booktitle = {ECML/PKDD},
  year = {2023},
  doi = {10.1007/978-3-031-43415-0\_33},
}
Figure