Cormorant: Covariant Molecular Neural Networks

Brandon M. Anderson, Truong-Son Hy, Risi Kondor

TL;DR: Cormorant is a novel neural network architecture designed specifically for molecular systems by "baking in" the laws of rotational and translational symmetry. By using spherical tensors and Clebsch-Gordan decompositions as its core mathematical building blocks, it achieves state-of-the-art performance in predicting molecular potential energy surfaces and chemical properties.

The Challenge: Physics-Aware Machine Learning

In computational chemistry, researchers face a brutal trade-off. Quantum Mechanics (like the Schrödinger equation) offers perfect accuracy but is computationally prohibitive for anything beyond a few dozen atoms. Conversely, Classical Force Fields are lightning-fast but often too simplistic to capture complex chemical behaviors.

Machine Learning (ML) aims to bridge this gap by learning from quantum data to create fast, accurate surrogate models. However, standard neural networks lack an inherent understanding of physics. If a molecule is rotated in space, a standard network might process it as a completely different object. To be truly effective and data-efficient, a model must be covariant: its internal representations should rotate exactly as the molecule does, ensuring the final prediction (like total energy) remains invariant (unchanged) regardless of the molecule's orientation.

The Cormorant Architecture

To address this, the authors introduce Cormorant, a rotationally covariant network built on three foundational pillars:

  1. Spherical Tensors as Activations: Instead of simple vectors of numbers, every "neuron" in Cormorant outputs a spherical tensor. These mathematical objects have specific, predictable transformation rules under rotation (linked to Wigner D-matrices).
  2. Clebsch-Gordan (CG) Non-linearities: Standard networks use functions like ReLU or Sigmoid. Cormorant replaces these with the Clebsch-Gordan decomposition—a method of combining two tensors to create a new one. This allows the network to operate entirely in "Fourier space," mimicking how physical multipole expansions (like dipole or quadrupole interactions) naturally behave.
  3. Geometric Correspondence: Each neuron explicitly corresponds to a specific atom or a subset of atoms. This structural choice makes the model's internal logic significantly more interpretable compared to traditional "black-box" graph neural networks.

The network consists of an input featurization layer, several "covariant" layers that pass messages between atoms using CG products, and a final "invariant" layer that collapses the complex tensor representations into a single scalar value, such as energy.

Key Results and Benchmarks

The model was evaluated on two gold-standard benchmarks in the field: QM9 (GDB-9) and MD-17. By hard-coding physical symmetries into its math rather than relying on massive data augmentation, Cormorant proves to be highly data-efficient and physically grounded.

QM9: Ground-State Properties

The QM9 dataset contains approximately 134,000 small organic molecules. The task is to predict 12 different ground-state properties, including electronic, energetic, and thermodynamic metrics.

QM9 Benchmark Results

Cormorant achieved leading or highly competitive results on 6 out of the 12 targets (such as polarizability and heat capacity). While it did not sweep every single metric, its strong performance across the board demonstrates the robustness of the covariant approach.

MD-17: Potential Energy Surfaces

The MD-17 dataset contains the molecular dynamics trajectories of eight small organic molecules. The goal here is to learn the Potential Energy Surface (PES)—the "map" of forces dictating how atoms move during a simulation.

MD-17 Benchmark Results

In this domain, Cormorant showed absolute dominance. It significantly outperformed all competing algorithms, including DeepMD, SchNet, and GDML. For several molecules, Cormorant reduced the Mean Absolute Error (MAE) by a substantial margin, proving the power of its physics-motivated architecture.

Limitations and Future Work

While groundbreaking, the Cormorant architecture comes with trade-offs:

  • Computational Complexity: Using Clebsch-Gordan products is mathematically elegant but computationally expensive, making these networks more difficult and slower to train than standard architectures.
  • Scaling: The paper demonstrates success on small molecules (up to 17 atoms in MD-17). Scaling this specific tensor-based approach to massive proteins or bulk materials remains a significant challenge.
  • Integration: The authors suggest that the next major step is directly integrating Cormorant-learned potentials into active Molecular Dynamics (MD) simulation frameworks to evaluate their stability and performance in "live" environments.

Summary updated

Abstract

We propose Cormorant, a rotationally covariant neural network architecture for learning the behavior and properties of complex many-body physical systems. We apply these networks to molecular systems with two goals: learning atomic potential energy surfaces for use in Molecular Dynamics simulations, and learning ground state properties of molecules calculated by Density Functional Theory. Some of the key features of our network are that (a) each neuron explicitly corresponds to a subset of atoms; (b) the activation of each neuron is covariant to rotations, ensuring that overall the network is fully rotationally invariant. Furthermore, the non-linearity in our network is based upon tensor products and the Clebsch-Gordan decomposition, allowing the network to operate entirely in Fourier space. Cormorant significantly outperforms competing algorithms in learning molecular Potential Energy Surfaces from conformational geometries in the MD-17 dataset, and is competitive with other methods at learning geometric, energetic, electronic, and thermodynamic properties of molecules on the GDB-9 dataset.
BibTeX
@inproceedings{anderson2019cormorant,
  author = {Brandon M. Anderson and Truong-Son Hy and Risi Kondor},
  title = {{Cormorant: Covariant Molecular Neural Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=JPdsW0So4o},
}
Figure