An End-to-End Framework for Molecular Conformation Generation via Bilevel Programming

Minkai Xu, Wujie Wang, Shitong Luo, Chence Shi, Yoshua Bengio, Rafael Gómez-Bombarelli, Jian Tang

Abstract

Predicting molecular conformations (or 3D structures) from molecular graphs is a fundamental problem in many applications. Most existing approaches are usually divided into two steps by first predicting the distances between atoms and then generating a 3D structure through optimizing a distance geometry problem. However, the distances predicted with such two-stage approaches may not be able to consistently preserve the geometry of local atomic neighborhoods, making the generated structures unsatisfying. In this paper, we propose an end-to-end solution for molecular conformation prediction called ConfVAE based on the conditional variational autoencoder framework. Specifically, the molecular graph is first encoded in a latent space, and then the 3D structures are generated by solving a principled bilevel optimization program. Extensive experiments on several benchmark data sets prove the effectiveness of our proposed approach over existing state-of-the-art approaches.

Introduction

In computational chemistry and drug discovery, a molecule's 3D shape—its conformation—is as critical as its chemical formula. This structure determines how a molecule interacts with biological targets. While traditional physics-based simulations are accurate, they are computationally prohibitive for large-scale screening.

Recent deep learning efforts have attempted to accelerate this process using a two-stage approach: first predicting inter-atomic distances and then applying distance geometry algorithms to find 3D coordinates. However, this separation often leads to geometric inconsistencies. Because the distance predictor is not trained to understand the constraints of 3D Euclidean space, it may produce mathematically impossible distance sets, resulting in unrealistic molecular structures. This paper introduces ConfVAE, an end-to-end framework that bridges this gap using bilevel programming.

The ConfVAE Framework

ConfVAE is built upon the Conditional Variational Autoencoder (CVAE) architecture, designed to learn the conditional distribution of 3D conformations given a 2D molecular graph. The core innovation lies in its treatment of conformation generation as a bilevel optimization problem.

The overall architecture of ConfVAE, illustrating the flow from molecular graph to latent space and the inner-loop geometry solver.

Bilevel Optimization

The framework consists of two nested loops:

  1. The Inner Loop (Geometry Solver): Given predicted distances, the model performs a differentiable gradient descent to find the optimal 3D coordinates ($R$). By making this step differentiable, the geometry solver becomes an integral part of the neural network's forward pass.
  2. The Outer Loop (The Learner): The model evaluates the generated 3D structure against ground truth data using Root-Mean-Square Deviation (RMSD). The resulting error is backpropagated through the entire optimization process, allowing the distance predictor to learn the implicit geometric constraints of 3D space.

To manage the computational complexity of this "optimization-inside-optimization," the authors utilize Continuous Normalizing Flows (CNF) in the decoder. This allows for complex distance distribution mapping with constant memory costs. Message Passing Neural Networks (MPNNs) are employed to encode the initial molecular graph, capturing the local and global chemical environment.

Experimental Results

The researchers evaluated ConfVAE against state-of-the-art baselines (GraphDG, CGCF) on three major benchmarks: GEOM-QM9, GEOM-Drugs, and ISO17.

Quantitative Performance

ConfVAE demonstrated significant improvements in both diversity and accuracy. On the challenging GEOM-Drugs dataset, which features complex organic compounds, ConfVAE achieved a Coverage (COV) score of 52.59% and a Matching (MAT) score of 1.2330 Å, outperforming previous deep learning models.

Quantitative results on GEOM-QM9 and GEOM-Drugs datasets, comparing ConfVAE against various baselines and classical toolkits.

Comparison with Classical Toolkits

A major milestone of this work is its performance relative to RDKit, the industry-standard classical toolkit. When ConfVAE's outputs are refined with a standard rule-based force field (FF), it outperforms RDKit on 7 out of 8 primary evaluation metrics. This indicates that the end-to-end learned model can capture chemical nuances that were previously only accessible through expert-crafted rules.

Conclusion

ConfVAE represents a significant shift in molecular modeling by moving from disjointed two-stage pipelines to a unified, end-to-end differentiable framework. By incorporating bilevel programming, the model ensures that predicted distances are physically and geometrically grounded. Future work aims to scale this framework to larger biological entities like proteins, where the principles of end-to-end geometry optimization could further accelerate structural biology research.

Summary updated

Abstract

Predicting molecular conformations (or 3D structures) from molecular graphs is a fundamental problem in many applications. Most existing approaches are usually divided into two steps by first predicting the distances between atoms and then generating a 3D structure through optimizing a distance geometry problem. However, the distances predicted with such two-stage approaches may not be able to consistently preserve the geometry of local atomic neighborhoods, making the generated structures unsatisfying. In this paper, we propose an end-to-end solution for molecular conformation prediction called ConfVAE based on the conditional variational autoencoder framework. Specifically, the molecular graph is first encoded in a latent space, and then the 3D structures are generated by solving a principled bilevel optimization program. Extensive experiments on several benchmark data sets prove the effectiveness of our proposed approach over existing state-of-the-art approaches. Code is available at \url{https://github.com/MinkaiXu/ConfVAE-ICML21}.
BibTeX
@inproceedings{xu2021end,
  author = {Minkai Xu and Wujie Wang and Shitong Luo and Chence Shi and Yoshua Bengio and Rafael Gómez-Bombarelli and Jian Tang},
  title = {{An End-to-End Framework for Molecular Conformation Generation via Bilevel Programming}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=5bNqkRQ-Iow},
}
Figure