Swallowing the Bitter Pill: Simplified Scalable Conformer Generation

Yuyang Wang, Ahmed A. Elhag, Navdeep Jaitly, Joshua M. Susskind, Miguel Angel Bautista

TL;DR

Molecular Conformer Fields (MCF) radically simplifies 3D molecule prediction by treating conformers as continuous functions (fields) rather than complex geometric puzzles. By ditching hand-crafted chemical heuristics in favor of a scalable, domain-agnostic diffusion model, MCF achieves state-of-the-art results, proving that massive scale can often outperform specialized inductive biases.

What Problem Does This Paper Address?

Predicting the 3D structure—or conformer—of a molecule from its 2D graph is a cornerstone of drug discovery. A molecule's shape determines how it interacts with proteins, its energy state, and its overall chemical behavior.

However, the "conformational space" is vast and grows exponentially with molecule size. Traditional methods like Molecular Dynamics are too slow, while rule-based generators often fail on complex, novel molecules. Recent machine learning approaches have tried to bridge this gap by "shoe-horning" chemical knowledge—like rotatable bond angles or rotational equivariance—directly into model architectures. While clever, these domain-specific constraints often make models harder to scale and prone to errors from the very heuristics they rely on.

Methods

The authors introduce Molecular Conformer Fields (MCF), which reframes conformer generation as learning a distribution over functions.

  1. Conformer Fields: Instead of predicting a list of coordinates, MCF treats a conformer as a function $f$ that maps a point on a molecular graph to a 3D location.
  2. Graph Positional Encoding: To represent the graph structure without complex Graph Neural Networks (GNNs), they use the leading eigenvectors of the graph Laplacian. This provides a unique, intrinsic coordinate system for each atom based purely on the graph's topology.
  3. Diffusion on Functions: MCF uses a diffusion process. It starts with random noise in 3D space and iteratively "denoises" it into a valid molecular structure. Crucially, it diffuses the function itself.
  4. PerceiverIO Architecture: The "brain" of the model is a PerceiverIO, a scalable Transformer variant. It uses cross-attention to map a "context set" of atoms to a latent space, which is then used to predict the positions of a "query set" of atoms.
Figure 1. Overview of proposed MCF. The structure of molecular graph is encoded through eigenvectors of Laplacian eigen-decomposition φ(V) and atomic features A. MCF directly operates on atom coordinates in 3D space and trains the diffusion model to denoise the function in 3D coordinates. The score network is developed with attention-based PerceiverIO architecture. Context pairs Ct attend to a latent array of learnable parameters via cross attention and the latent array goes through several self attention blocks. Finally, the query pairs Qt cross-attend to the latent array to produce the final noise prediction ˆϵq in 3D space.

Datasets

The model was evaluated on three primary benchmarks:

  • GEOM-QM9: ~130,000 small molecules (up to 29 atoms).
  • GEOM-DRUGS: ~304,000 larger, pharmaceutically relevant molecules (average 44 atoms).
  • GEOM-XL: A challenging "out-of-distribution" set containing 102 very large molecules with more than 100 atoms.

Key Insights and Ideas

  • The "Bitter Lesson" Applied: The title refers to Rich Sutton’s famous essay, "The Bitter Lesson," which argues that general methods that leverage computation and scale eventually win out over those based on human-designed heuristics. MCF proves this for chemistry: a large, general-purpose Transformer outperforms models with hand-crafted chemical constraints.
  • Trading Bias for Scale: The authors found that explicitly enforcing rotational equivariance (making sure the model treats a rotated molecule the same way) isn't strictly necessary. If the model is large enough and the data is diverse, it learns these symmetries naturally.
  • Continuous Representation: Because MCF defines a field, it can technically predict the 3D position of any point on the graph, even points along a bond where no atom exists. This suggests the model captures a deep understanding of the underlying spatial distribution.

Key Results

MCF sets a new high bar for conformer generation across all tested datasets:

  • Superior Accuracy: On the GEOM-DRUGS dataset, even the smallest MCF model (13M parameters) outperformed the previous state-of-the-art, Torsional Diffusion, by roughly 15% in precision and recall.
  • Scalability: Performance improved consistently as the model grew. The "Large" version (242M parameters) showed a 20% improvement over the small version, highlighting the benefits of scaling.
  • Robustness to Size: MCF excelled on the GEOM-XL dataset. While some baseline methods failed to even generate structures for these massive molecules, MCF generalized effectively to these unseen, large-scale structures.
  • Inference Efficiency: Using an efficient sampler (DDIM), MCF can generate high-quality conformers in just 5 steps, matching the quality that specialized models require 50 steps to achieve.

Limitations and Future Work

Despite its success, MCF has a few hurdles:

  • Computational Cost: Training large Transformers is expensive and requires significant GPU resources (up to 16 Nvidia A100s for the largest model).
  • Data Hunger: As a domain-agnostic model, it likely requires more data to learn chemical rules from scratch compared to models that have those rules "baked in."
  • Unconditional Generation: Currently, the model generates conformers in a vacuum. Future work aims to extend this to conditional inference, such as predicting how a drug molecule folds when it is inside a specific protein pocket (molecular docking).

Summary updated

Abstract

We present a novel way to predict molecular conformers through a simple formulation that sidesteps many of the heuristics of prior works and achieves state of the art results by using the advantages of scale. By training a diffusion generative model directly on 3D atomic positions without making assumptions about the explicit structure of molecules (e.g. modeling torsional angles) we are able to radically simplify structure learning, and make it trivial to scale up the model sizes. This model, called Molecular Conformer Fields (MCF), works by parameterizing conformer structures as functions that map elements from a molecular graph directly to their 3D location in space. This formulation allows us to boil down the essence of structure prediction to learning a distribution over functions. Experimental results show that scaling up the model capacity leads to large gains in generalization performance without enforcing inductive biases like rotational equivariance. MCF represents an advance in extending diffusion models to handle complex scientific problems in a conceptually simple, scalable and effective manner.
BibTeX
@inproceedings{wang2024swallowing,
  author = {Yuyang Wang and Ahmed A. Elhag and Navdeep Jaitly and Joshua M. Susskind and Miguel Angel Bautista},
  title = {{Swallowing the Bitter Pill: Simplified Scalable Conformer Generation}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=I44Em5D5xy},
}
Figure