Relaxed Equivariant Graph Neural Networks

Elyssa F. Hofgard, Rui Wang, Robin Walters, Tess E. Smidt

Abstract

3D Euclidean symmetry equivariant neural networks have demonstrated notable success in modeling complex physical systems. We introduce a framework for relaxed $E(3)$ graph equivariant neural networks that can learn and represent symmetry breaking within continuous groups. Building on the existing e3nn framework, we propose the use of relaxed weights to allow for controlled symmetry breaking. We show empirically that these relaxed weights learn the correct amount of symmetry breaking.

Introduction

Equivariant Neural Networks (ENNs) have become a cornerstone for modeling physical systems, such as molecules and crystals, because they inherently respect fundamental symmetries like rotation, translation, and inversion. However, strict equivariance is often an idealization. In real-world scenarios, symmetries are frequently broken by external forces (e.g., an electric field), phase transitions (e.g., liquid to crystal), or inherent biological asymmetries.

Standard ENNs are mathematically constrained to produce symmetric outputs from symmetric inputs, making them incapable of representing these "broken" physical states. This paper introduces Relaxed Equivariant Graph Neural Networks, a framework designed to bridge the gap between the efficiency of equivariant models and the flexibility required to model approximate symmetries.

Methodology: Relaxing the Constraints

The proposed framework extends the e3nn library, which utilizes irreducible representations (irreps) of the $O(3)$ group. The core innovation is the introduction of relaxed weights ($\tilde{\theta}$).

Unlike standard scalar weights, relaxed weights are defined as a direct sum of irreps. These weights interact with the spherical harmonic projections of graph edges via a tensor product. To maintain physical interpretability and prevent the model from collapsing into a standard unconstrained network, the authors employ:

  1. Initialization: Weights are initialized such that the model begins as perfectly equivariant (only scalar terms are non-zero).
  2. Regularization: $L_2$ regularization is applied to the non-scalar components of the weights, encouraging the model to find the minimal amount of symmetry breaking necessary to fit the data.
Visualizing symmetry breaking: The model generates spherical signals that correspond to the geometric symmetries of the output shapes (Cube, Prism, and Random).

This approach allows the learned weights to be projected back into 3D space as "spherical signals," providing a visual and mathematical representation of how symmetry is broken within the system.

Experimental Results

The framework was evaluated on synthetic benchmarks designed to test its ability to detect and represent hidden symmetry breaking.

3D Shape Deformations

The model was tasked with mapping a high-symmetry cube ($O_h$ group) to lower-symmetry shapes, such as a rectangular prism ($D_{4h}$) and a random asymmetric shape. The relaxed weights successfully "discovered" the target symmetries, identifying which axes remained symmetric and which became unique.

Charged Particle Trajectories

In a more complex physical simulation, the model predicted the force on a particle moving through hidden electromagnetic fields. While a standard equivariant model was unable to represent the directional bias of the fields, the relaxed model accurately recovered the vector and pseudovector components corresponding to the electric and magnetic fields, respectively.

Training loss comparison: The standard equivariant model (Equiv) fails to converge on asymmetric tasks, while the relaxed model successfully learns the underlying symmetry-breaking patterns.

As shown in the loss curves, the standard equivariant model remains "stuck" at a high Mean Squared Error (MSE) of 0.07, whereas the relaxed model achieves a near-perfect MSE of $10^{-6}$.

Conclusion

Relaxed Equivariant Graph Neural Networks provide a powerful middle ground for physical modeling. By treating weights as "order parameters" that can evolve during training, the model can learn to represent broken symmetries while remaining grounded in the mathematical language of group theory. Future work will focus on applying this framework to complex experimental datasets in materials science and fluid dynamics.

Summary updated

Abstract

3D Euclidean symmetry equivariant neural networks have demonstrated notable success in modeling complex physical systems. We introduce a framework for relaxed $E(3)$ graph equivariant neural networks that can learn and represent symmetry breaking within continuous groups. Building on the existing e3nn framework, we propose the use of relaxed weights to allow for controlled symmetry breaking. We show empirically that these relaxed weights learn the correct amount of symmetry breaking.
BibTeX
@inproceedings{hofgard2024relaxed,
  author = {Elyssa F. Hofgard and Rui Wang and Robin Walters and Tess E. Smidt},
  title = {{Relaxed Equivariant Graph Neural Networks}},
  booktitle = {GRaM @ ICML},
  year = {2024},
  url = {https://openreview.net/forum?id=eVB1fn37Ay},
}
Figure