Anisotropic Message Passing: Graph Neural Networks with Directional and Long-Range Interactions

Moritz Thürlemann, Sereina RinikerICLR 2023

Standard Message Passing Neural Networks (MPNNs) have revolutionized molecular modeling, but they suffer from two fundamental blind spots: isotropy and the "long-range" bottleneck. Most Graph Neural Networks (GNNs) treat interactions as isotropic, primarily relying on the distance between atoms. While some modern models incorporate directional information like bond angles, they often struggle to capture complex directional forces—such as quadrupolar interactions or torsion—without resorting to computationally expensive mathematical frameworks. Furthermore, message passing is inherently local. Capturing long-range forces like electrostatics and polarization, which are crucial in real-world chemistry, requires many layers, often leading to "over-smoothing" of information.

To address these limitations, researchers have introduced Anisotropic Message Passing (AMP). AMP brings physics-based directionality to GNNs by introducing a novel "anisotropic state" based on Cartesian multipoles. This approach explicitly models directional and long-range interactions efficiently, enabling the seamless integration of quantum molecular systems with large-scale classical environments (QM/MM).

The Anisotropic State and Local Frames

The core innovation of AMP is the Anisotropic State ($M_i$). Instead of representing an atom solely with a hidden scalar vector, the authors augment the state with Cartesian multipoles of order $k$:

  • $k=0$: Monopole (charge)
  • $k=1$: Dipole (vector)
  • $k=2$: Quadrupole (tensor)
Local frames and angular dependence

These multipoles are expressed as linear combinations of local basis functions, ensuring the model is equivariant—if the molecule rotates, its internal representation rotates accordingly. The model calculates anisotropic interaction features by decomposing the interaction between two sites into radial components and angular coefficients, mimicking classical multipole expansion physics.

For systems where a small quantum (QM) region is surrounded by a massive classical (MM) environment (e.g., a protein in water), AMP introduces a Direct Polarization (AMP-D) term. This allows the environment to polarize the QM atoms directly without requiring expensive message-passing steps for thousands of solvent molecules.

Capturing Complex Torsion and Field Effects

A critical finding of this work is the necessity of higher-order multipoles ($k \ge 2$). Models limited to vectorial information ($k=1$, like PaiNN) fail to describe certain torsional interactions. By moving to quadrupoles ($k=2$), AMP can resolve these complex directional dependencies.

Torsional angles and local coordinate frames

Furthermore, by using multipoles, the model can treat the environment as a field. This hybrid particle-field treatment is a massive win for efficiency. It accounts for the influence of thousands of solvent molecules on a target molecule without the computational burden of passing messages between all of them. Unlike models that use mathematically heavy Spherical Harmonics, Cartesian multipoles provide a "sweet spot" of fast computation and chemical accuracy.

Performance and Scaling

The authors evaluated AMP across several scales, demonstrating significant improvements over standard models:

  • Directional Accuracy: In water dimer tests, AMP(2) correctly modeled the potential energy surface during rotation, whereas $k=1$ models and bond-angle-based models (like DimeNet) showed significant errors or total invariance.
  • Extensivity and Scaling: When tested on water clusters, AMP showed superior transferability. A model trained only on clusters of 10 molecules accurately predicted energies for clusters of 20, outperforming the popular PaiNN model.
  • QM/MM Efficiency: On complex systems like Retinoic Acid solvated in water, AMP(3)-DP achieved high accuracy. Remarkably, calculating the energy and gradients for a system with ~1,500 atoms took only 10 ms on a single GPU.
  • QM9 Benchmark: AMP proved competitive with top-tier models like Allegro and PaxNet on the standard QM9 dataset. While not the absolute leader in every category, its inference speed and handling of long-range interactions make it highly practical for large-scale molecular dynamics.

Limitations and Future Work

While AMP's complexity is lower than tensor-product models, adding multipoles still increases the constant computational cost per message compared to simple GNNs (AMP(3) is roughly 2-3x slower than a basic GNN). Additionally, while the paper notes that Fast Multipole Method (FMM) integration is a natural benefit of the multipole formalism, a full $O(N)$ implementation for massive systems remains a future goal. Finally, although the Mean Absolute Error (MAE) is low, further testing is required to ensure these potentials remain stable for long-term (nanosecond-scale) molecular dynamics simulations.

Summary updated

BibTeX
@inproceedings{thurlemann2023anisotropic,
  author = {Moritz Thürlemann and Sereina Riniker},
  title = {{Anisotropic Message Passing: Graph Neural Networks with Directional and Long-Range Interactions}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
}
Figure