Abstract
We propose Cormorant, a rotationally covariant neural network architecture for learning the behavior and properties of complex many-body physical systems. We apply these networks to molecular systems with two goals: learning atomic potential energy surfaces for use in Molecular Dynamics simulations, and learning ground state properties of molecules calculated by Density Functional Theory. Some of the key features of our network are that (a) each neuron explicitly corresponds to a subset of atoms; (b) the activation of each neuron is covariant to rotations, ensuring that overall the network is fully rotationally invariant. Furthermore, the non-linearity in our network is based upon tensor products and the Clebsch-Gordan decomposition, allowing the network to operate entirely in Fourier space. Cormorant significantly outperforms competing algorithms in learning molecular Potential Energy Surfaces from conformational geometries in the MD-17 dataset, and is competitive with other methods at learning geometric, energetic, electronic, and thermodynamic properties of molecules on the GDB-9 dataset.
Introduction
In computational chemistry, researchers often face a trade-off between the high accuracy of Quantum Mechanics (QM) and the computational efficiency of Classical Force Fields. While Machine Learning (ML) models aim to bridge this gap, standard neural networks often struggle because they do not inherently respect the 3D symmetries of physical space. Many existing models rely on "invariant" features, such as inter-atomic distances, which can discard critical directional information like the orientation of dipoles. Cormorant is designed to respect the fundamental symmetries of physics—rotation and translation—by ensuring its internal representations are rotationally covariant.
Methods
The core innovation of Cormorant lies in its use of spherical tensors and representation theory to maintain physical covariance throughout the network.
- Covariance vs. Invariance: While an invariant model's output remains unchanged by input rotation, a covariant model's internal states rotate in a mathematically predictable way. This preserves the geometric "texture" of the molecule across layers.
- Clebsch-Gordan (CG) Non-linearity: Instead of standard activations like ReLU, Cormorant uses the Clebsch-Gordan transform—a tool from quantum mechanics used to combine angular momenta. This allows the network to perform non-linear interactions entirely in Fourier space using spherical harmonics.
- Atom-Centric Architecture: Each neuron in the network explicitly corresponds to a specific atom or pair of atoms. This structure mimics the multipole expansion used in classical electrostatics, making the model's "reasoning" more physically interpretable.
Results
The authors evaluated Cormorant on two primary benchmarks: MD-17 (for potential energy surfaces) and QM9 (for ground-state properties).
MD-17: Potential Energy Surfaces
Cormorant demonstrated state-of-the-art performance on the MD-17 dataset, which consists of eight small organic molecules. It significantly outperformed competing algorithms, including DeepMD, SchNet, and sGDML, in predicting conformational energies across the entire dataset.
QM9: Molecular Properties
On the QM9 dataset of ~134,000 molecules, Cormorant achieved leading or highly competitive results on 6 out of 12 regression targets. It was particularly effective at predicting electronic properties such as the dipole moment ($\mu$) and the gap between molecular orbitals ($\Delta\epsilon$).
Discussion and Future Work
Cormorant's physics-informed architecture allows for high data efficiency, as the laws of physics are effectively "baked in." However, the model faces challenges regarding training stability due to the quadratic nature of the Clebsch-Gordan product, which can lead to exploding gradients. Future research will focus on optimizing these high-order polynomial networks, scaling the architecture to larger systems, and integrating Cormorant directly into real-time molecular dynamics simulations.
Summary updated