When modeling physical systems—like the folding of a protein or the collision of particles in a fluid—the laws of physics do not change just because you rotate the coordinate system. If a molecule rotates, its energy remains the same (invariance), and the forces acting on its atoms rotate with it (equivariance).
Traditional Graph Neural Networks (GNNs) often struggle with this because they treat 3D coordinates as simple flat vectors. To solve this, researchers developed Equivariant Message Passing, a way to ensure that the network's internal representations respect the symmetries of 3D space, specifically the Euclidean group $E(3)$. However, this mathematical rigor often comes with a "complexity tax" that makes models slow and difficult to scale. Recent research focuses on breaking this bottleneck by moving from heavy group-theoretic machinery toward efficient geometric representations.
The Complexity Tax of Spherical Harmonics
The first generation of high-performance equivariant models, such as NequIP by Simon Batzner and others, demonstrated that using geometric tensors instead of just scalars leads to incredible data efficiency. These models typically rely on Spherical Harmonics to represent directional information and Clebsch-Gordan tensor products to combine them.
As Johannes Brandstetter and Max Welling show in their work on SEGNNs, these operations allow for "steerable" features that can represent complex physical quantities like velocity or torque. However, the Clebsch-Gordan decomposition is computationally expensive. It involves high-dimensional "mixing" of features that does not map naturally to the highly optimized matrix multiplication kernels used in modern GPUs. As Erpai Luo notes in the context of SPHNet, the complexity of these operations can scale with the sixth power of the orbital order, creating a massive barrier for large-scale simulations.
Swapping Spherical Harmonics for Cartesian Tensors
To avoid the overhead of spherical harmonics, a new wave of models like TensorNet by Guillem Simeon and Gianni De Fabritiis proposes a return to Cartesian Tensors.
Instead of decomposing signals into abstract spherical basis functions, these models represent geometric features as simple $3 \times 3$ matrices (rank-2 tensors). The intuition is that most physical interactions can be captured through basic linear algebra—dot products, outer products, and matrix multiplications. By staying in Cartesian space, TensorNet achieves state-of-the-art accuracy on molecular benchmarks while being significantly faster and more parameter-efficient than its spherical counterparts.
Similarly, Tuan Le and Frank Noé introduced EQGAT, which uses a per-channel attention mechanism directly on Cartesian coordinates. This bypasses the need for complex group-theoretic transformations entirely, allowing the model to scale to large biological structures like proteins that contain thousands of atoms.
Equivariance via Local Reference Frames
Another powerful strategy for efficiency is to shift the burden of equivariance from the layers to the data representation. If you define a local coordinate system (a reference frame) for every node in a graph, you can describe all neighboring interactions relative to that frame.
Peter Lippmann and colleagues demonstrate in Beyond Canonicalization that by using "tensorial messages"—which transform features between these local frames—standard, non-equivariant architectures like PointNet++ can achieve exact equivariance. This approach effectively "unwraps" the 3D symmetry, allowing the network to use standard MLPs while maintaining geometric rigor.
Mohammad Mohaiminul Islam takes this further with Platonic Transformers. By projecting geometric data onto the faces of Platonic solids (like the icosahedron), the model gains a discrete geometric inductive bias. This allows the use of highly optimized vanilla Transformer blocks without the "complexity tax" of continuous rotation groups.
Optimizing the Tensor Product
For tasks where full group-theoretic expressivity is still required—such as predicting the quantum Hamiltonian of a molecule—researchers are finding ways to prune the math.
Haiyang Yu and Xiaoning Qian developed QHNet, which identifies that many paths in the standard tensor product are redundant. By reinventing the symmetry-handling mechanism, they reduced expensive tensor operations by 92%, resulting in a 6x speedup. SPHNet builds on this by introducing adaptive sparsity, selectively pruning unimportant atom-pair interactions and tensor combinations during training.
As YuQing Xie and Tess Smidt explore in their analysis of expressivity and runtime tradeoffs, the field is moving toward a nuanced understanding of where complexity is necessary. The goal is no longer just to achieve equivariance, but to do so using the most efficient geometric primitives available—whether that means sparse tensor products, Cartesian matrices, or local reference frames.