Equivariant Flow Matching with Hybrid Probability Transport

Yuxuan Song, Jingjing Gong, Minkai Xu, Ziyao Cao, Yanyan Lan, Stefano Ermon, Hao Zhou, Wei-Ying Ma

Abstract

Generating 3D molecular structures requires simultaneous modeling of continuous spatial coordinates and categorical atom types. While diffusion models have shown promise, they often suffer from sampling inefficiency and inconsistent evolution of different features. We present EquiFM (Equivariant Flow Matching), a generative framework that utilizes Equivariant Optimal Transport and a hybrid probability path to streamline molecule generation. EquiFM achieves a 4.75× speedup in sampling while producing more chemically stable structures compared to existing diffusion-based methods.

Introduction

The generation of 3D molecules is a complex "multi-modal" task. A model must determine both the 3D positions of atoms (continuous) and their chemical identities (categorical). Current state-of-the-art approaches primarily rely on Diffusion Models (DMs). However, DMs face two significant hurdles:

  1. Sampling Inefficiency: The curved and often "chaotic" probability paths in DMs necessitate many iterative steps to transform noise into a valid molecule.
  2. Modality Inconsistency: Coordinates and atom types evolve at different rates. In standard DMs, atom types are often fixed before the 3D skeleton is properly formed, leading to "twisted" paths and invalid chemical valency.

EquiFM addresses these issues by shifting from diffusion to a simulation-free Flow Matching (FM) objective, optimized for equivariant 3D data.

Overview of the EquiFM framework showing the noise-to-molecule generation process via hybrid paths.

Methodology

The EquiFM framework introduces several key innovations to stabilize and accelerate 3D molecular generation:

Equivariant Flow Matching and Optimal Transport

To ensure the generative process respects physical symmetries, EquiFM is designed to be invariant to rotations and translations (SE(3) invariance). A core component is Equivariant Optimal Transport (EOT), which identifies the "straightest" possible path from a noise distribution to a target molecule. By solving for optimal rotation and permutation alignments between noise and data, EOT minimizes unnecessary movement, leading to a more stable vector field.

Hybrid Probability Transport

Recognizing that different features require different mathematical treatments, EquiFM employs a hybrid approach:

  • Coordinates (x): Follow the straight-line EOT path.
  • Atom Types (h): Follow a Variance Preserving (VP) path.

To prevent modality inconsistency, the authors use an information-theoretic alignment. By synchronizing the "mutual information" across these paths, the model ensures that atom identities and positions emerge at a chemically consistent rate, respecting valency rules during the generation process.

Experimental Evaluation

EquiFM was evaluated on standard 3D molecular benchmarks, including QM9 (small molecules) and GEOM-DRUG (larger, drug-like molecules).

Performance and Stability

As shown in the table below, EquiFM demonstrates superior performance in generating stable molecular structures. On the QM9 dataset, it achieved a Molecule Stability of 88.3%, significantly outperforming the Equivariant Diffusion Model (EDM) baseline of 82.0%.

Performance metrics for EquiFM compared to various baselines on the QM9 and DRUG datasets.

Efficiency Gains

Beyond stability, EquiFM offers a massive efficiency advantage. It is 4.75× faster on average than diffusion baselines, reaching peak performance in approximately 210 function evaluations, compared to the 1,000 typically required by DMs.

Discussion and Conclusion

The success of EquiFM underscores the importance of "straight paths" in generative modeling. By forcing the model to learn a linear vector field via EOT, training becomes more stable and inference significantly faster. Furthermore, aligning the timing of feature emergence acts as a crucial inductive bias for chemical validity.

While EquiFM shows a spike in function evaluations near the end of sampling due to the "unsmoothness" of categorical atom types, it represents a significant step forward. Future work will explore extending this framework to larger biological systems, such as protein pocket-based drug design and antibody generation.

Summary updated

BibTeX
@inproceedings{song2023equivariant,
  author = {Yuxuan Song and Jingjing Gong and Minkai Xu and Ziyao Cao and Yanyan Lan and Stefano Ermon and Hao Zhou and Wei-Ying Ma},
  title = {{Equivariant Flow Matching with Hybrid Probability Transport}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=hHUZ5V9XFu},
}
Figure