Neural networks for 3D data, such as molecules, point clouds and triangle meshes, usually avoid raw coordinates. Instead they feed each layer quantities that stay the same when the object is rotated: distances between neighbouring points, angles between surface normals, dihedral angles on mesh edges. Graph networks have a separate, well-known failure called over-smoothing. As layers are stacked, repeated neighbour averaging makes all node representations look alike. Nearly everything known about over-smoothing comes from citation and web graphs, where nodes have no geometry. This note argues that in geometric networks the failure takes a different form, and that the standard tools miss it. Each layer re-reads the rotation-invariant geometry, so node features do not collapse to a single vector. They collapse to something subtler: a function of each node's immediate local shape. Depth stops adding information, yet the representations still look diverse.
The core claim can be put in two sentences. In a deep invariant or equivariant graph network, once the propagated features homogenize, the freshly injected local geometry dominates every message, so layer-$L$ features become predictable from one-hop shape alone ("reversion to local shape"). Rank-based over-smoothing diagnostics cannot see this, and we propose both a diagnostic that can and a message design that keeps propagated differences alive. The study also tracks a second, geometry-specific collapse route: in networks that update coordinates, the learned positions can contract, which makes the re-read distances themselves uninformative.
What prior work establishes and what it leaves open
The geometric side supplies the architectures we study. E(n) Equivariant Graph Neural Networks computes every message from invariant node features and the squared distance $\|x_i-x_j\|^2$, and it also moves the coordinates along relative vectors. Its authors note that large graphs "might require many layers", but they evaluate only shallow models. MeshCNN convolves over mesh edges using five invariant features and avoids deep flat diffusion by collapsing edges as it goes. Rotation-Invariant Transformer for Point Cloud Matching replaces positional encodings with point-pair features, which combine distances and normal angles. It also documents "symmetry ambiguity": identical patches on opposite sides of a symmetric object receive identical invariant features. Triangle-Net uses triangle invariants and adds shortcut connections for depth without ever measuring collapse. These papers motivate the proposal, because they show deep invariant message passing is wanted and unexamined. Symmetry ambiguity also predicts that reversion to local shape will be lossy, since local shape alone cannot separate symmetric parts.
The over-smoothing side supplies the measurement lens and the candidate remedies. PairNorm shows that averaging contracts the total pairwise distance between features and rescales the features to hold that spread fixed. Complete the Missing Half and Geometric Scattering Attention Networks add high-pass channels, built from differences between a node and its neighbours, alongside the usual averaging. Are we measuring oversmoothing in graph neural networks correctly? shows that Dirichlet energy misses collapse in trained networks, while effective rank tracks accuracy with average Pearson correlation 0.91. The same paper concedes that high rank is necessary but not sufficient, since noise can produce it.
That caveat is the point of contact. We predict that geometric networks are exactly the case where rank stays high while information is lost, because the rank is supplied by injected local geometry rather than by propagated context. Rethinking Graph Neural Networks From A Geometric Perspective Of Node Features treats over-smoothing as the shrinking of the simplex formed by class centroids. We use it as a second lens: symmetric 3D parts should yield nearly coincident centroids before any message passing happens.
All of the over-smoothing work evaluates only on non-geometric graphs, and none of the geometric work measures collapse. The closest obvious ideas are "run rank metrics on deep EGNNs" and "port the filterbank high-pass channel to invariant features". We include both as components and baselines. The new contribution is the specific mechanism, the diagnostic that separates it from ordinary collapse, and the test of whether existing remedies address it.
Core Claim and Diagnostic
Let $h_i^L$ be node $i$'s features after $L$ layers. Let $g_i^{(k)}$ be a fixed rotation-invariant summary of its $k$-hop neighbourhood, for example a histogram of pairwise distances and point-pair features within $k$ hops. We define the local-shape explainability
$$\rho_k(L) = R^2\big(h^L \mid g^{(k)}\big),$$
meaning the fraction of variance in $h_i^L$ explained by a small probe network trained to predict it from $g_i^{(k)}$ on held-out nodes. A network that uses its depth well should show $\rho_1(L)$ falling as $L$ grows, while $\rho_k(L)$ for $k$ near $L$ stays high. Reversion to local shape is the signature where $\rho_1(L)$ rises back toward 1 as depth increases, and effective rank stays well above 1.
For coordinate-updating models, we also track the radius of gyration of the learned coordinates relative to the input coordinates, and the effective rank of the learned distance matrix. Together these separate geometric contraction from feature homogenization.
Research Program
The first stage is measurement. We train invariant networks, which read distances from fixed input geometry, and EGNN-style coordinate-updating networks at depths 4, 8, 16, 32 and 64. At every layer we record numerical rank, effective rank, Dirichlet energy, $\rho_1$ through $\rho_4$, and coordinate spread. The prediction is concrete. Accuracy degrades with depth while effective rank stays high, and $\rho_1$ rises over the same layers. In coordinate-updating models a second degradation route appears in which learned coordinates contract.
The second stage is a targeted remedy we call anchored difference messaging. Messages take the form $m_{ij}=\phi_m(h_i,\ \hat\delta_{ij},\ g_{ij})$, where $g_{ij}$ is the invariant edge geometry computed from input coordinates and $\hat\delta_{ij}$ is the feature difference $h_j-h_i$. That difference is rescaled per layer so that its average magnitude over edges is held constant. This is PairNorm applied to local differences rather than to absolute features, and it keeps the high-pass signal from shrinking until the geometry term dominates. Reading $g_{ij}$ from input coordinates, while still allowing equivariant coordinate updates as an output stream, closes the geometric-contraction route. Both pieces cost almost nothing and preserve rotation invariance, because the differences are taken between invariant scalars.
The third stage is the symmetry analysis. On objects with known symmetric part pairs, we measure the distance between centroid features of mirror-symmetric part classes as a function of depth. The question is whether deep propagation recovers the global context needed to separate them, or whether reversion to local shape locks them together.
Baselines
The naive reference points come in two kinds. One is a shallow network with 4 to 7 layers, the depth used by EGNN, because the main question is whether depth helps at all. The other is a coordinate-based network trained with random rotation augmentation. CRIN and 3D-Rotation-Equivariant Quaternion Neural Networks argue that augmentation leaves models fragile on unseen poses, and we want that cost measured under our protocol rather than assumed.
The deep competitors share one backbone and one tuning budget. That budget is an equal number of hyperparameter trials per method, an equal number of parameters within 5%, and an equal number of training epochs. The competitors are a deep EGNN, the same network with residual connections, the same with PairNorm on the invariant features, and the same with a filterbank-style high-pass channel following Complete the Missing Half. The last is the strongest transferable over-smoothing remedy. We treat PairNorm plus residuals as the real threat, because it is the simplest thing a practitioner would try.
The strongest existing task-specific method on meshes is MeshCNN, which reports 92.30% accuracy on human body segmentation and 99.63% on COSEG chairs. It avoids deep diffusion through pooling rather than by fixing it. On rotated sparse ModelNet40, Triangle-Net reports over 70% accuracy with 16 points under arbitrary rotation. For deep (16+ layer) invariant geometric networks on any of these benchmarks, no reported result was found on Lacuna, so we will establish those numbers ourselves.
Two ablations isolate the key ingredients. One removes the difference rescaling but keeps the difference input. The other reads geometry from updated rather than input coordinates.
Beating the baselines has a precise meaning here. The method must improve accuracy at depth 32 or more over the best of the residual, PairNorm and high-pass variants. It must also reach or exceed the shallow model's best accuracy on the long-range tasks, and its $\rho_1$ must fall with depth where the baselines' rises.
Why not just…?
Why not just keep the network shallow and enlarge the neighbourhood radius, or connect all nodes? For small molecules this probably suffices, which is why QM9 serves as a negative control rather than a target. Fully connected geometric graphs scale quadratically, and EGNN's authors already flag that cost on large structures. Our long-range synthetic task and full-resolution mesh segmentation include a matched-compute arm: a shallow network with a radius chosen so that its cost equals that of the deep model. If this arm wins, the honest conclusion is that depth is the wrong tool on these graphs, and we will report that.
Why not just add residual connections and PairNorm? This is the most serious alternative. PairNorm itself reports that holding the global spread fixed prevents decay with depth but often fails to raise peak accuracy. Our mechanism also predicts a specific failure for it: PairNorm keeps global spread constant, and injected local geometry can supply that spread on its own. If $\rho_1$ stays low and accuracy holds under residuals plus PairNorm, the remedy half of this proposal is unnecessary. The work would then become a measurement paper, which the experiments are designed to accept.
Why not just use raw coordinates with rotation augmentation and avoid invariant features entirely? Coordinates give every node a distinct position, which might resist collapse. But CRIN and 3D-Rotation-Equivariant Quaternion Neural Networks document accuracy drops on unseen poses for such models. Triangle-Net reports a 58% drop under heavy jitter for coordinate models versus 28.5% for its invariant model. We include this arm at matched depth to test whether invariance trades robustness against depth.
Why not just pool, as MeshCNN does? Pooling sidesteps over-smoothing on classification but discards resolution needed for dense prediction and registration. We compare against MeshCNN at its reported numbers and against a pooling-free variant.
Proposed Experiments and Decisive Checks
The first decisive check is a synthetic long-range task. On random point clouds connected by k-nearest-neighbour graphs, each node's label depends on an invariant property, such as a local triangle's angle class, located exactly $k$ hops away, with $k$ between 4 and 32. On this task, rank metrics, $\rho$ curves and accuracy should separate cleanly.
The real-data evaluation uses five benchmarks. COSEG and human body segmentation run without edge-collapse pooling. ModelNet40 and ScanObjectNN run under arbitrary rotation, with dense k-nearest-neighbour graphs. N-body trajectory prediction tests coordinate contraction. QM9 serves as the control where we expect no depth benefit.
The claim is supported if four things hold together. Effective rank stays high while accuracy degrades with depth in the baselines. $\rho_1(L)$ rises over the same layers. Anchored difference messaging reverses both. Symmetric-part centroids separate with depth only under our method.
The claim is refuted in either of two cases. Degradation could coincide with ordinary rank collapse, in which case the existing diagnostic of Zhang et al. already suffices. Or residuals plus PairNorm could match the method on the long-range task.
References
- E(n) Equivariant Graph Neural Networks
- MeshCNN: A Network with an Edge
- Rotation-Invariant Transformer for Point Cloud Matching
- Triangle-Net: Towards Robustness in Point Cloud Learning
- PairNorm: Tackling Oversmoothing in GNNs
- Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks
- Geometric Scattering Attention Networks
- Are we measuring oversmoothing in graph neural networks correctly?
- Rethinking Graph Neural Networks From A Geometric Perspective Of Node Features
- CRIN: Rotation-Invariant Point Cloud Analysis and Rotation Estimation via Centrifugal Reference Frame
- 3D-Rotation-Equivariant Quaternion Neural Networks