Equivariant neural networks are networks built so that rotating a molecule or point cloud rotates their internal features in a known way. They represent each atom with a stack of channels sorted by angular order: plain numbers ($l=0$), 3D vectors ($l=1$), and higher-order tensors ($l\ge 2$). They are used for atomic forces, molecular Hamiltonians and protein pockets, and in practice they are almost always shallow. Graph over-smoothing is the tendency of deep message-passing networks to make every node's features look the same as layers are stacked. It is well studied for ordinary scalar features, but nobody in the literature we found has asked how it acts on the vector and tensor channels of an equivariant network.
This proposal makes a specific claim and tests it. Each layer of an equivariant network mixes a neighbor's features with the spherical harmonics of the bond direction. Linearized, that layer is diffusion on a cellular sheaf: a graph where every edge carries its own linear map between node feature spaces, and here those maps are fixed by the 3D geometry. The spectrum of this geometry-induced sheaf Laplacian should predict, channel by channel, which parts of the representation collapse with depth and how fast. It also suggests a cheap, exactly equivariant correction that removes only the collapsing component.
Core Claim
Consider one message path in a tensor-product network. It sends input order $l_{in}$ to output order $l_{out}$ through a spherical harmonic of order $l_f$. Its edge operator is
$$F_{ij}\,h_j = w(r_{ij})\; C_{l_{in} l_f}^{\,l_{out}}\big(h_j \otimes Y_{l_f}(\hat r_{ij})\big),$$
where $h_j$ is node $j$'s feature of order $l_{in}$, $r_{ij}$ and $\hat r_{ij}$ are the bond length and direction, $w$ is the learned radial weight, $Y_{l_f}$ is the spherical harmonic, and $C$ is the Clebsch–Gordan coupling. Stacking these maps gives a block "geometric sheaf Laplacian" $L_G = D - A$ with $A_{ij}=F_{ij}$.
The claim has two parts. Paths with $l_f=0$ act like the identity on each edge, so they behave exactly like ordinary graph diffusion. Their kernel is the constant signals, and they should collapse at the rate set by the graph's spectral gap. Paths with $l_f\ge 1$ carry direction-dependent maps. Around rings and crystal cycles these maps do not compose back to the identity, so the kernel shrinks and collapse slows or stops.
If this holds, three things follow. Over-smoothing in equivariant networks is uneven across channels and can be predicted from geometry. Depth limits in such networks come mostly from the $l=0$-routed paths. A fix should target the near-kernel of $L_G$ and leave the rest alone.
This framing borrows from three lines of work. Sheaf neural networks show that learned edge maps can prevent collapse on abstract graphs: Joint Diffusion Processes as an Inductive Bias in Sheaf Neural Networks diffuses features and edge maps jointly, and Cooperative Sheaf Neural Networks uses directed source and target maps. Their "rotation invariance", however, refers to rotations of internal feature space, not to physical 3D rotations. Random Walks, Conductance, and Resistance for the Connection Graph Laplacian develops the theory of Laplacians whose edges carry group elements, but never implements them as network layers. We differ from all three in the source of the edge maps: they come from the bond geometry already inside standard equivariant layers, and we learn nothing new on the edges.
Research Program
Theory. We will write out $L_G$ for each path family in a NequIP/MACE-style layer and characterize its near-kernel on generic geometries. We expect constants for $l_f=0$, and geometric "parallel sections" for $l_f\ge1$ that exist only when holonomy is trivial, as on trees or chains. For a linearized residual layer $h \leftarrow h - \eta L_G h$, the non-kernel energy shrinks by roughly $(1-\eta\lambda_{\min}^{+})^2$ per layer, where $\lambda_{\min}^{+}$ is the smallest nonzero eigenvalue of $L_G$. This is the standard contraction argument for a symmetric diffusion step, and it gives a per-path decay prediction we can test.
We will also treat the asymmetric case. $F_{ji}$ differs from $F_{ij}^\top$ by a parity factor $(-1)^{l_f}$, so the operator need not be symmetric, and the analysis there uses singular values rather than eigenvalues.
Measurement. We need a collapse metric. Are we measuring oversmoothing in graph neural networks correctly? shows that effective rank (the entropy of the normalized singular values of the node-feature matrix) tracks accuracy with average Pearson correlation 0.91. Dirichlet energy, by contrast, stays flat while rank collapses. One point deserves stating plainly. Effective rank of the flattened per-order feature matrix is already rotation-invariant, because a global rotation multiplies that matrix on the right by an orthogonal block. So we do not claim a new metric. We simply measure effective rank separately for each order $l$ and for each path family.
We will train depth sweeps of 2 to 32 layers for tensor-product networks and EGNN-style networks. For each depth we record per-$l$ effective rank alongside the eigenvalues of $L_G$ computed on the test geometries.
Intervention. At each layer we estimate the bottom-$k$ eigenspace of $L_G$ for each path using a few sparse power or Lanczos iterations, at a cost of $O(kE)$ per layer for $E$ edges. We project features off that eigenspace with a learned per-order gate and rescale by the norm in each order. Because $L_G$ is built equivariantly, its eigenspaces rotate with the input, and the projection is exactly equivariant.
For $l_f=0$ paths this reduces to per-order mean-centering, essentially an equivariant PairNorm. For $l_f\ge1$ paths it removes geometric parallel sections that PairNorm cannot see. The gate matters because Smoothness Errors in Dynamics Models and How to Avoid Them proves that refusing all smoothing is also wrong: strictly unitary convolutions cannot represent physical dissipation. The gate lets the model keep smoothing when the target process smooths.
Baselines
The naive reference is the same equivariant backbone at its usual shallow depth with no correction, together with its deep counterpart. Comparing the two tells us whether depth hurts at all. Every method will share the same data splits, the same parameter budget at each depth, and the same hyperparameter search budget, which we fix at an equal number of trials per method on the validation split.
The first strong competitor is equivariant normalization with residual connections, specifically per-order norm rescaling of the kind standard in tensor-product libraries. It is the cheapest thing practitioners already use against deep degradation. We beat it if our method gives lower error at the best depth and a measurably higher per-$l$ effective rank at matched depth.
The second strong competitor on molecular dynamics is GGND from Geometric Graph Neural Diffusion for Stable Molecular Dynamics Simulations. It replaces cutoff-based message passing with continuous equivariant diffusion over the fully connected graph. On 3BPA at 1200 K it reports simulation stability of 11.209 ps for VisNet+GGND against 0.004 ps for VisNet. At 600 K it reports a force MAE of 0.193 eV/Å for SEGNO+GGND against 0.893 eV/Å for SEGNO. Beating it means matching or exceeding 11.209 ps stability at lower cost than GGND's $O(N^2)$ edges, or adding our correction on top of GGND and improving it. We will also measure GGND's internal rank decay over integration time, which its authors do not report.
On Hamiltonians, the strongest baseline is QHNet from QH9, which reports an MAE of 76.31×10⁻⁶ Hartree on QH9-stable-id. The same paper reports that its error roughly doubles on the molecule-wise out-of-distribution split. On pockets, the strongest baseline is VN-EGNN, which reports a COACH420 DCC success rate of 0.677. For 3BPA we found no reported result on Lacuna for MACE or NequIP.
The key ablation is the constant-kernel projection alone, which is per-order equivariant PairNorm. If it matches the full geometric-kernel projection, then the sheaf view adds nothing as a method and the contribution reduces to the diagnosis.
Why not just…?
Why not just stay shallow, since equivariant force fields already work at two to six layers? This is the most serious threat. If accuracy saturates early regardless of collapse, deeper networks are pointless on local tasks such as 3BPA forces. We therefore separate local tasks from receptive-field-limited ones. These are molecule-wise generalization on QH9 and pocket detection on proteins, where VN-EGNN's own motivation is that signal gets diluted across large protein graphs. The proposal succeeds only if depth helps on the second group once collapse is corrected. If it does not, we will report the per-channel diagnosis as the main result.
Why not just use per-order PairNorm or layer normalization? Our theory predicts that these fix exactly the $l_f=0$ collapse mode and miss the geometric mode. The constant-kernel ablation tests this directly. If the per-$l\ge1$ effective rank still decays under PairNorm but not under our projection, the geometric component matters. If both behave the same, the reviewer is right.
Why not just connect everything, with virtual nodes or full attention? Virtual nodes shrink the graph's diameter and accelerate over-smoothing unless regularized, as shown in Understanding Virtual Nodes: Oversmoothing, Oversquashing, and Node Heterogeneity. Attention layers also homogenize tokens with depth, per Setting the Record Straight on Transformer Oversmoothing. Fully connected operators also scale quadratically with the number of atoms. We include VN-EGNN and GGND as direct comparisons and add our projection to both, which tests whether it is complementary to global connectivity rather than a substitute for it.
Proposed Experiments
The first decisive check is whether the theory predicts what trained networks do. On QH9 geometries and 3BPA conformers we compute the spectrum of $L_G$ for each path and correlate its smallest nonzero eigenvalue with the measured per-layer effective-rank decay of that path in trained deep networks. A null result here, meaning low correlation or no difference between $l_f=0$ and $l_f\ge1$ paths, would falsify the core claim.
The second check is whether the fix extends useful depth. We sweep depth on QH9-dynamic-mol (Hamiltonian block MAE), 3BPA at 600 K and 1200 K (force MAE and stability time in picoseconds), and COACH420 and HOLO4K (DCC and DCA success rates). Each sweep compares the shallow and deep backbones, the deep backbone with equivariant normalization, the constant-kernel ablation, our full projection, and GGND or VN-EGNN where applicable.
The third check guards against breaking physics. On the MeshPDE heat task, where the ground truth genuinely smooths, our learned gate should reach a Rayleigh error competitive with the 9.1 that Smoothness Errors in Dynamics Models and How to Avoid Them reports for R-UNIMESH, and it must not undersmooth. Finally, we will report the wall-clock overhead of the Lanczos step against the backbone's cost, so the gains are judged at matched compute.
References
- Joint Diffusion Processes as an Inductive Bias in Sheaf Neural Networks
- Cooperative Sheaf Neural Networks
- Random Walks, Conductance, and Resistance for the Connection Graph Laplacian
- Are we measuring oversmoothing in graph neural networks correctly?
- PairNorm: Tackling Oversmoothing in GNNs
- Smoothness Errors in Dynamics Models and How to Avoid Them
- Geometric Graph Neural Diffusion for Stable Molecular Dynamics Simulations
- QH9: A Quantum Hamiltonian Prediction Benchmark for QM9 Molecules
- VN-EGNN: Equivariant Graph Neural Networks with Virtual Nodes Enhance Protein Binding Site Identification
- Understanding Virtual Nodes: Oversmoothing, Oversquashing, and Node Heterogeneity
- Setting the Record Straight on Transformer Oversmoothing