An Autoregressive Flow Model for 3D Molecular Geometry Generation from Scratch

Youzhi Luo, Shuiwang JiOpenReview 2022 · OpenReview

Abstract

We consider the problem of generating 3D molecular geometries from scratch. While multiple methods have been developed for generating molecular graphs, generating 3D molecular geometries from scratch is largely under-explored. In this work, we propose G-SphereNet, a novel autoregressive flow model for generating 3D molecular geometries. G-SphereNet employs a flexible sequential generation scheme by placing atoms in 3D space step-by-step. Instead of generating 3D coordinates directly, we propose to determine 3D positions of atoms by generating distances, angles and torsion angles, thereby ensuring both invariance and equivariance properties. In addition, we propose to use spherical message passing and attention mechanism for conditional information extraction. Experimental results show that G-SphereNet outperforms previous methods on random molecular geometry generation and targeted molecule discovery tasks.

Introduction

In drug discovery and materials science, a molecule's 3D geometry is as critical as its chemical formula. While many AI models successfully generate 2D molecular graphs, creating 3D structures "from scratch" remains challenging due to the requirement of symmetry. A molecule's representation must be invariant to translation and rotation. Most deep learning models struggle to maintain these properties when predicting raw Cartesian $(x, y, z)$ coordinates. Furthermore, existing 3D generation methods often produce chemically invalid structures or fail to capture complex spatial dependencies.

G-SphereNet addresses these issues by treating molecular generation as a sequential decision process, building a molecule atom-by-atom while inherently respecting physical symmetries through the use of internal coordinates.

Methods

G-SphereNet is an autoregressive flow model that generates molecules sequentially. The process involves several key innovations:

  1. Internal Coordinates: Instead of absolute positions, the model predicts the distance ($d$), angle ($\theta$), and torsion angle ($\phi$) of a new atom relative to a "focal atom" and its neighbors. This ensures the model is naturally invariant to rotation and translation.
  2. Autoregressive Flows: Normalizing flows map simple Gaussian distributions to the complex distributions of atom types and positions, allowing for exact likelihood computation.
  3. Hybrid Architecture: The model uses SphereNet, a graph neural network for spherical message passing, to capture local 3D features, combined with Multi-Head Attention for global context.
  4. Training Strategy: Trajectories are created using Prim’s algorithm, ensuring each new atom is the nearest neighbor to the existing structure, providing a logical building order.
The sequential generation process of G-SphereNet using internal coordinates to place atoms step-by-step.

Evaluation and Results

The model was evaluated on the QM9 dataset, which contains approximately 130,000 small organic molecules. G-SphereNet was compared against baselines like G-SchNet and E-NFs on validity and structural accuracy.

Chemical Validity and Structural Accuracy

G-SphereNet achieved a chemical validity of 88.18%, significantly outperforming G-SchNet (81.49%) and the one-shot E-NFs model (39.77%). It also achieved the lowest Maximum Mean Discrepancy (MMD) for bond lengths, indicating that its generated structures closely match the statistical distributions of real molecules.

Comparison of G-SphereNet with baseline methods on chemical validity and MMD distances for various bond types.

Targeted Discovery

In tasks aimed at generating molecules with specific quantum properties, such as low HOMO-LUMO gaps or high isotropic polarizability, G-SphereNet successfully produced novel molecules that surpassed the properties found in the original dataset and those generated by baseline models.

Discussion and Limitations

While G-SphereNet sets a new standard for 3D molecular generation, it has limitations. The sequential generation process is inherently slower than one-shot methods, particularly as molecular size increases. Additionally, the current focus is on small molecules from the QM9 dataset; scaling to larger structures like proteins remains a future challenge. Finally, because the process is sequential, errors in early steps can propagate through the generation chain.

Despite these challenges, G-SphereNet demonstrates that by designing for symmetry and incorporating both local and global context, we can generate highly accurate and valid 3D molecular geometries from scratch.

Summary updated

Abstract

We consider the problem of generating 3D molecular geometries from scratch. While multiple methods have been developed for generating molecular graphs, generating 3D molecular geometries from scratch is largely under-explored. In this work, we propose G-SphereNet, a novel autoregressive flow model for generating 3D molecular geometries. G-SphereNet employs a flexible sequential generation scheme by placing atoms in 3D space step-by-step. Instead of generating 3D coordinates directly, we propose to determine 3D positions of atoms by generating distances, angles and torsion angles, thereby ensuring both invariance and equivariance properties. In addition, we propose to use spherical message passing and attention mechanism for conditional information extraction. Experimental results show that G-SphereNet outperforms previous methods on random molecular geometry generation and targeted molecule discovery tasks. Our code is publicly available as part of the DIG package (https://github.com/divelab/DIG).
BibTeX
@misc{luo2022autoregressive,
  author = {Youzhi Luo and Shuiwang Ji},
  title = {{An Autoregressive Flow Model for 3D Molecular Geometry Generation from Scratch}},
  year = {2022},
  url = {https://openreview.net/forum?id=c62fVK8zI9w},
  howpublished = {OpenReview},
}
Figure