Swallowing the Bitter Pill: Simplified Scalable Conformer Generation

Yuyang Wang, Ahmed A. Elhag, Navdeep Jaitly, Joshua M. Susskind, Miguel Ángel Bautista

Abstract

We present a novel way to predict molecular conformers through a simple formulation that sidesteps many of the heuristics of prior works and achieves state of the art results by using the advantages of scale. By training a diffusion generative model directly on 3D atomic positions without making assumptions about the explicit structure of molecules (e.g. modeling torsional angles) we are able to radically simplify structure learning, and make it trivial to scale up the model sizes. This model, called Molecular Conformer Fields (MCF), works by parameterizing conformer structures as functions that map elements from a molecular graph directly to their 3D location in space. This formulation allows us to boil down the essence of structure prediction to learning a distribution over functions. Experimental results show that scaling up the model capacity leads to large gains in generalization performance without enforcing inductive biases like rotational equivariance. MCF represents an advance in extending diffusion models to handle complex scientific problems in a conceptually simple, scalable and effective manner.

Introduction

Predicting the 3D "conformers" (shapes) of a molecule from its 2D graph is a cornerstone of drug discovery. A molecule's shape determines how it interacts with proteins, its energy state, and its chemical properties. However, the search space for these shapes is massive, growing exponentially as molecules get larger and more flexible.

Historically, researchers have tackled this by "shoe-horning" domain knowledge into models—for example, by forcing the model to only predict rotatable bond angles (torsional angles) or by building complex "equivariant" neural networks that are mathematically guaranteed to handle rotations and translations. While effective, these hand-crafted constraints are difficult to implement, hard to scale, and can fail on complex or unusual molecular structures. In this work, we present Molecular Conformer Fields (MCF), which leverages the "Bitter Lesson": that general-purpose methods scaled with computation eventually outperform domain-specific ones.

Methods: Molecular Conformer Fields

MCF treats a molecule's 3D structure as a "field"—a function that maps nodes in a molecular graph directly to their 3D coordinates. This formulation allows the model to learn a distribution over functions rather than just fixed vectors.

Conceptual illustration of the field mapping graph coordinates to 3D space.

Diffusion Probabilistic Fields (DPF)

Instead of diffusing pixels or simple vectors, MCF applies the diffusion process to functions. It starts with random 3D noise assigned to atoms and iteratively "denoises" them into a valid molecular shape. By training directly on 3D atomic positions without explicit structural assumptions, we radically simplify the learning process.

Architecture and Spectral Encodings

Instead of using specialized graph neural networks, MCF utilizes PerceiverIO, a scalable Transformer variant. It treats the molecule as a set of points, allowing the model to learn complex global interactions without being restricted by local bond-by-bond rules. To provide the Transformer with a sense of the molecule's topology, we use Spectral Positional Encodings—the eigenvectors of the graph Laplacian. This provides a unique coordinate system for the graph itself.

Overview of the MCF architecture showing the transition from graph features to 3D coordinates via attention mechanisms.

Unlike "Torsional Diffusion," which only moves rotatable bonds, MCF moves every atom in 3D space simultaneously. It doesn't explicitly enforce rotational equivariance; instead, it learns these symmetries directly from the data through augmentation.

Experimental Results

The model was evaluated on standard industry benchmarks: GEOM-QM9 (~130k small molecules), GEOM-DRUGS (~304k larger molecules), and GEOM-XL (a challenging set of 102 molecules with >100 atoms).

MCF established a new state-of-the-art (SOTA) across the board:

  • GEOM-QM9: MCF achieved a 95.0% Coverage (Recall), outperforming the previous SOTA, Torsional Diffusion (92.8%).
  • GEOM-DRUGS: The gap widened on larger molecules. MCF-L achieved 84.7% Coverage, a significant jump over Torsional Diffusion’s 72.7%.
  • Generalization (GEOM-XL): On massive molecules, MCF-L achieved an Average Minimum RMSD (AMR) of 1.64Å, significantly better than Torsional Diffusion (1.93Å).

Importantly, MCF is more efficient in "wall-clock" inference time than Torsional Diffusion because it avoids expensive equivariant operations, despite having a larger parameter count.

Discussion: The Bitter Lesson in Chemistry

The success of MCF reinforces Rich Sutton’s "Bitter Lesson": leveraging computation and scale is more effective than building in human knowledge. A large, general Transformer outperformed models with years of hand-coded chemical "inductive biases."

We found that equivariance is negotiable; a sufficiently large model can learn to be invariant to rotations simply through data augmentation. Furthermore, scaling works: testing three sizes (Small, Base, Large) showed that in every metric, the "Large" model (242M parameters) significantly outperformed the smaller ones. Even without explicit instructions on bond lengths or angles, the model learned to generate chemically plausible structures, including correct chiral centers, purely by observing 3D data.

While training large Transformers requires significant GPU resources, the resulting model is conceptually simple, scalable, and highly effective, paving the way for applications in molecular docking and de novo drug design.

Summary updated

Abstract

We present a novel way to predict molecular conformers through a simple formulation that sidesteps many of the heuristics of prior works and achieves state of the art results by using the advantages of scale. By training a diffusion generative model directly on 3D atomic positions without making assumptions about the explicit structure of molecules (e.g. modeling torsional angles) we are able to radically simplify structure learning, and make it trivial to scale up the model sizes. This model, called Molecular Conformer Fields (MCF), works by parameterizing conformer structures as functions that map elements from a molecular graph directly to their 3D location in space. This formulation allows us to boil down the essence of structure prediction to learning a distribution over functions. Experimental results show that scaling up the model capacity leads to large gains in generalization performance without enforcing inductive biases like rotational equivariance. MCF represents an advance in extending diffusion models to handle complex scientific problems in a conceptually simple, scalable and effective manner.
BibTeX
@inproceedings{wang2024swallowing,
  author = {Yuyang Wang and Ahmed A. Elhag and Navdeep Jaitly and Joshua M. Susskind and Miguel Ángel Bautista},
  title = {{Swallowing the Bitter Pill: Simplified Scalable Conformer Generation}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=I44Em5D5xy},
}
Figure