Graph neural networks classify a node by repeatedly averaging its features with those of its neighbors. Stack many such layers and every node's representation drifts toward the same vector, a failure called over-smoothing. A large literature treats this collapse as one disease and proposes cures: normalization, new activation functions, sheaf-valued diffusion, multi-track routing. We argue that two different things are destroyed when a deep graph network degrades, and current measurements cannot tell them apart. The first is the node's own feature signal, which a plain multilayer perceptron (MLP) that ignores the graph would have kept. The second is genuine structural information that only the edges carry. The first loss is a symptom of feature-structure entanglement: message passing fuses a node's attributes and its neighborhood into one vector, so averaging over a misleading neighborhood also erases the attributes. The second loss is the over-smoothing story proper.
The proposal is to measure the two losses separately. We do this by giving every deep model a fixed, structure-free path from the raw features to the output, then reading off how much accuracy the graph path adds at each depth. Our hypothesis is that on most node-classification benchmarks, especially the cleaned heterophilous ones where neighbors tend to have different labels, nearly all of the familiar accuracy-versus-depth decay is lost features. Real structural over-smoothing should appear only where the edges carry information the features lack. If the hypothesis is right, much depth-oriented architecture work is fixing the wrong problem on the wrong datasets. If it is wrong, we will have the first clean evidence that collapse costs structural information even after attributes are protected.
Core Claim
Define the structural gain at depth L as the accuracy of a model with an attribute path plus an L-layer graph path, minus the accuracy of the same attribute path alone (a tuned MLP). This node-level analogue of the structural gap of Rethinking the Effectiveness of Graph Classification Datasets in Benchmarks for Assessing GNNs makes depth degradation decomposable.
The total drop of an ordinary deep network from its best shallow accuracy splits into two parts. The attribute loss is the part recovered by adding the structure-free path. The structural loss is the decline of structural gain with L. Our claim is that attribute loss dominates whenever the shallow structural gain is small. Structural loss should be substantial only in regimes where structure is necessary, such as missing features or long-range dependencies. And only the structural loss should follow the per-node degree-and-homophily failure cases described by Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks.
The prior work this builds on falls into three roles. Motivating evidence comes from three papers. PairNorm: Tackling Oversmoothing in GNNs flattens depth-decay curves, but it does not raise peak accuracy on feature-rich graphs. A critical look at the evaluation of GNNs under heterophily: are we really making progress? finds that many specialized heterophily models fail to beat a graph-agnostic ResNet. Re-evaluating the Advancements of Heterophilic Graph Learning labels some datasets "malignant" heterophily, meaning a tuned MLP wins. All three hint that depth harms attributes more than it harms structural reasoning, but none of them measures this.
Constraining evidence comes from two papers. Oversmoothing, "Oversquashing'', Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learning shows that Dirichlet energy and the Rayleigh quotient give contradictory collapse diagnoses on Cora. Backward Oversmoothing: why is it hard to train deep Graph Neural Networks? attributes part of deep failure to optimization rather than representation. Consequently we do not rely on energy metrics, and we separate trained-end-to-end accuracy from frozen-representation probes.
The closest architectural work is DGNN. DGNN keeps an attribute branch that never passes through smoothing, but it does so with quadratic-cost auxiliary graphs. It never studies depth, and it was evaluated on duplicate-leaked Chameleon and Squirrel. We use a linear-cost version of the same separation as a measuring instrument, not as a new state-of-the-art model.
Research Program
The instrument is an attribute-anchored network. The output for node $v$ is a learned gate mixing an MLP prediction from the raw features $x_v$ with a prediction from an L-layer graph network. The graph network can be GCN, GAT, a sheaf model, or anything else. The gate is per node, computed from $x_v$ and the current graph embedding. It costs time linear in the number of edges, with no kNN graph and no class-indexed tracks.
Three readouts accompany it. The first is end-to-end accuracy of the anchored and unanchored versions at depths 2, 4, 8, 16, 32, and 64. The second is linear probes on frozen layer-L embeddings of the unanchored network. One probe predicts the tuned MLP's logits, which measures how much attribute information survives. The other predicts the labels after regressing out those logits, which measures how much structural information survives. The probes decouple representation loss from trainability. The third readout is per-node attribution of errors to the three cases of Two Sides of the Same Coin: low homophily; high homophily with low relative degree; and high homophily with high relative degree. Node homophily is computed from ground-truth labels on evaluation nodes only, for analysis. The model never sees it.
A controlled synthetic sweep then makes causation testable. We start from real graphs, specifically the clean roman-empire and amazon-ratings graphs. We replace features with label-correlated vectors of tunable signal-to-noise ratio, and we rewire edges at tunable homophily while preserving each node's degree. This gives a grid over feature informativeness and edge informativeness, with degree held fixed so that the degree-driven drift mechanism can be probed separately by a second sweep that perturbs only relative degree. The prediction is that attribute loss scales with feature signal-to-noise and is largely indifferent to homophily, whereas structural loss scales with homophily and relative-degree disparity. Gap 3 in the literature asks whether deep degradation is degree-driven drift or feature dilution. This design answers it with an ablation instead of an argument.
Baselines
The naive reference point is a tuned MLP on raw features, using the same hyperparameter budget as every graph model. It defines zero structural gain. Any claim that depth "helps" must clear it.
The strongest existing competitor on the cleaned heterophily suite is the family of standard GAT and graph-transformer models with ego-neighbor separation from A critical look at the evaluation of GNNs under heterophily. These models give each node a weight matrix separate from its aggregated neighbors, and they took 13 of the top 15 spots there. It is the right competitor because ego separation is itself a partial attribute path. If it already preserves attributes at depth, our anchor adds nothing. No reported per-dataset number was found on Lacuna for these models; the file states only the 13-of-15 result. We will rerun them ourselves on the published splits.
For depth robustness we include three competitors. PairNorm reports 74.5% accuracy on Cora with 100% missing features on unlabeled nodes, versus 42.3% for vanilla SGC. Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors uses non-monotone activations that stay stable to 64 layers on Cora; no single accuracy figure appears in the file. Two Sides of the Same Coin reports GGCN, also stable to 64 layers. As a heterophily-adaptive reference we include MixMP from Heterophily-informed Message Passing, which reports 74.5% on minesweeper against 52.3% for base GCN. The file states this as accuracy, and we will report the benchmark's own metric alongside it.
All methods get the same splits, the same per-depth tuning budget in trials, the same early-stopping rule, and the same hardware. Results are reported over ten seeds.
The key ablation isolates the anchor. We compare four arms: no anchor; ego separation at every layer; initial-feature reinjection at every layer, as in the GCNII-style deep models that Complete the Missing Half tested against; and a single late-fused raw-feature anchor. "Beating" a baseline here means one of two things. Either the anchored version of a backbone has a significantly flatter accuracy-versus-depth curve than its unanchored version on datasets with small structural gain, or it shows that the baseline's flatness already comes from implicit attribute preservation. Either outcome counts, because the proposal is a measurement.
Why not just…?
Why not just use a tuned MLP and stop worrying about depth? On feature-dominated datasets this may be the right answer, and our structural-gain curve will say so explicitly rather than leave it implied. But MLPs cannot function in the missing-feature regime, where PairNorm's 74.5% versus 42.3% shows that propagation is essential. They also fall short on minesweeper, where MixMP's 74.5% versus GCN's 52.3% indicates a large structural signal. These are exactly the regimes where we expect real structural loss at depth, and an MLP cannot reveal it.
Why not just use ego-neighbor separation, which already keeps the node's own vector? This is a real threat. At layer L, the "ego" term is the node's layer-(L−1) representation, which has already been mixed. Separation therefore protects attributes at depth only if information survives the chain of layers. Nobody has tested this beyond shallow models, which is Gap 4. Our ablation compares per-layer ego separation and per-layer initial reinjection against the single raw anchor, and the frozen-embedding attribute probe measures retention directly. If ego separation matches the anchor at 64 layers, we report that it already solves attribute loss. Our decomposition then reduces to a measurement protocol for the remaining structural loss, which is still the open quantity.
Why not just add PairNorm? PairNorm keeps embeddings spread apart, but spread is not information: features can stay far apart while no longer encoding what the raw attributes said. PairNorm also leaves the direction of heterophilous messages unchanged. The attribute probe on PairNorm models tests this directly. If PairNorm embeddings retain MLP-level attribute information at depth, the alternative wins on attribute loss, and we will say so.
Proposed Experiments
The homophilous datasets are Cora, CiteSeer, and PubMed. PubMed is notable because Measuring and Improving the Use of Graph Information in Graph Neural Networks reports low feature smoothness there, 0.91, explaining weak gains over MLPs. The cleaned heterophilous suite is roman-empire, amazon-ratings, minesweeper, tolokers, and questions, plus deduplicated Chameleon and Squirrel. Original Chameleon and Squirrel are run only to show how leakage distorts depth curves. Regimes where structure is required are SSNC-MV on Cora and CiteSeer at increasing missing-feature rates, and Tree-NeighborsMatch, where Multi-Track Message Passing reports that standard GCN and GAT fail beyond depth 5. Finally, the synthetic signal-to-noise by homophily grid.
Four outcomes are decisive. First, on datasets whose shallow structural gain is near zero, anchoring should remove most of the depth decay of GCN and GAT, with attribute loss accounting for most of the drop. If anchored deep models still decay, the decay is not attribute loss. We then check whether frozen probes are healthy while trained accuracy is not, which would mean optimization failure in the sense of backward over-smoothing. Second, on SSNC-MV, minesweeper, and Tree-NeighborsMatch, anchoring should not rescue deep decay, and the residual structural loss should concentrate in low-homophily and low-relative-degree nodes. If it does not, the case taxonomy of Two Sides of the Same Coin does not explain real-data collapse. Third, in the synthetic grid, attribute loss should rise with feature signal-to-noise at fixed homophily, and structural loss should rise as homophily falls at fixed signal-to-noise. Failure of this separation would falsify the two-mechanism picture. Fourth, Dirichlet energy and Rayleigh quotient curves will be logged beside both losses, to test whether either correlates with structural loss once attribute loss is factored out. That would resolve, at least empirically, the metric contradiction noted in Demystifying Common Beliefs.
References
- Rethinking the Effectiveness of Graph Classification Datasets in Benchmarks for Assessing GNNs
- Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
- PairNorm: Tackling Oversmoothing in GNNs
- A critical look at the evaluation of GNNs under heterophily: are we really making progress?
- Re-evaluating the Advancements of Heterophilic Graph Learning
- Oversmoothing, "Oversquashing'', Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learning
- Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?
- DGNN: Decoupled Graph Neural Networks With Structural Consistency Between Attribute and Graph Embedding Representations
- Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
- Heterophily-informed Message Passing
- Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks
- Measuring and Improving the Use of Graph Information in Graph Neural Networks
- Multi-Track Message Passing: Tackling Oversmoothing and Oversquashing in Graph Learning via Preventing Heterophily Mixing