Graph neural networks (GNNs) update each node by averaging over its neighbors. Stacked many times, this averaging makes nodes look alike, a failure called over-smoothing. Separately, people now pre-train GNNs on some graphs and reuse them on unrelated ones, such as citation networks reused on social or airport networks. The hope is that what carries over is a node's structural role: whether it is a hub, a bridge, or a peripheral leaf, regardless of what its features mean. This note argues that over-smoothing should be measured by what it destroys in transfer, namely these roles, and not by global statistics of the representation. We propose a cheap, label-free measure of role collapse. It is computed on the target graph by checking how well a frozen encoder's node representations recover analytic role descriptors (GraphWave's diffusion signatures). We test whether it predicts cross-domain transfer better than the current best over-smoothing diagnostics.
In two sentences, the idea is this. Existing over-smoothing metrics ask whether representations have collapsed globally, but a pre-trained encoder can keep a high-rank representation while erasing exactly the hub/bridge/leaf distinctions that make it transferable. We measure that erasure directly against a smoothing-immune reference. We use the measure to audit and repair the low-pass objectives now built into graph foundation models.
Core Claim
The claim has three parts. First, there is a failure mode we call role collapse: encoder depth and smoothness-promoting objectives make structurally different nodes indistinguishable even when standard diagnostics report a healthy encoder. Second, a role-recoverability score computed on an unlabeled target graph predicts few-shot cross-domain accuracy better than effective rank or Dirichlet energy, and the gap is largest on heterophilic targets (graphs where linked nodes tend to have different labels). Third, the losses that recent transfer methods add to encourage smoothness across edges are a measurable cause of role collapse.
The motivation comes from Are we measuring oversmoothing in graph neural networks correctly?. That paper shows Dirichlet energy (the summed squared feature difference across edges) misses over-smoothing in trained networks. It proposes effective rank (an entropy over singular values) instead, reporting an average Pearson correlation of 0.91 with accuracy. The authors also concede that high rank is necessary but not sufficient, since random noise has full rank. Their analysis is entirely single-domain. We supply the missing half. Rank says whether any information survives; role recoverability says whether the transferable information survives.
The Measure
For a target graph we first compute GraphWave descriptors $\Phi \in \mathbb{R}^{n \times d}$ for its $n$ nodes. GraphWave (Learning Structural Node Embeddings via Diffusion Wavelets) summarizes the heat-diffusion pattern around each node with an empirical characteristic function. It is analytic, costs time linear in the number of edges, and gives identical vectors to nodes with isomorphic neighborhoods. Because it never runs learned message passing, it cannot over-smooth, which makes it a stable reference.
Given a frozen encoder's layer-$\ell$ representations $H_\ell$, we fit a ridge map $W$ on half the nodes. On the other half we report
$$\rho_\ell = 1 - \frac{\|\Phi - H_\ell W\|_F^2}{\|\Phi - \bar{\Phi}\|_F^2},$$
where $\bar{\Phi}$ is the column mean of $\Phi$ and $\|\cdot\|_F$ is the Frobenius norm. A score $\rho_\ell$ near 1 means roles are linearly recoverable from the encoder; a score near 0 means they have been washed out.
We also report a coarse variant. We cluster $\Phi$ into role classes and take the effective rank of the class-centroid matrix of $H_\ell$. This is Zhang et al.'s rank restricted to role structure. Neither score needs downstream labels, so both can be computed on any target graph before adaptation.
The reference is imperfect. GraphWave encodes one definition of role, and an encoder might carry a different, equally useful notion that $\rho$ undercounts. We therefore also compute $\rho$ against struc2vec-style degree-sequence descriptors (struc2vec) and against degree and eigenvector centrality, the targets used by CenPre. We report whether conclusions agree across the three references.
Research Program
The program has three stages: build a population of encoders, audit it, and then intervene.
Encoder population. We pre-train GCN, GAT and GIN backbones at depths 2, 4, 8 and 16. We cross these with five pre-training objectives:
- Deep Graph Infomax (DGI), which maximizes agreement between local patches and a global summary.
- Centrality regression in the style of CenPre, which predicts degree and eigenvector centrality.
- A computation-tree reconstruction objective following GFT, which tokenizes unrolled message-passing trees.
- Each of the above with GP2F's topology-consistent fusion loss added (GP2F). This loss penalizes $\|z_u - z_v\|^2$ over edges and is therefore a Dirichlet penalty.
- A filterbank variant with a high-pass "diversification" channel (FB-GNN), which keeps each node's difference from its neighborhood average.
We also run the released or reimplemented GAUGE model (Are Common Substructures Transferable?) as an external point.
Pre-training sources follow GAUGE's protocol: ogbn-arxiv, Reddit and Amazon-Ratings. This lets our numbers be compared with its reported results.
Audit. On every target graph we compute $\rho_\ell$, effective rank, numerical rank and Dirichlet energy for every encoder and layer. We then measure few-shot transfer accuracy and ask which diagnostic best ranks the encoders.
Intervention. If role collapse predicts transfer failure, the obvious repair is to make role descriptors a pre-training target. We add an auxiliary loss requiring the final layer to regress GraphWave descriptors of the source graph. This extends CenPre from two centrality scalars to full diffusion signatures, and we pair it with a high-pass channel.
The intervention is deliberately simple. Its purpose is to test causally whether preserving $\rho$ preserves transfer. Its main competitor is attaching GraphWave features after the encoder ("late fusion"). We treat that comparison as a central result, not a nuisance.
GAUGE. GAUGE presents a tension we address directly. It treats low Dirichlet energy as a sign of transferability, while Zhang et al. show that low energy accompanies collapse. GAUGE's energy is measured on fibers after its learned parallel transport, not on raw features, so the two claims are not formally contradictory. Our audit settles the matter empirically: we check whether GAUGE's representations keep high $\rho$ on heterophilic targets, where equivalent nodes sit in high-energy boundary regions.
Baselines
Naive reference. A structure-only predictor uses GraphWave descriptors alone with a linear classifier and no learned encoder. This bounds how much any pre-training adds beyond analytic roles. It is run on the same few-shot splits with the same classifier tuning budget. On air-traffic networks, where labels are activity levels and closely track role, it may be very hard to beat. Showing that is itself informative.
Diagnostic competitor. The strongest competing diagnostic is effective rank from Zhang et al., which reports an average Pearson correlation of 0.91 with accuracy in single-domain settings. We compute it on exactly the same encoders, layers and target graphs as $\rho$. Role recoverability beats it if its rank correlation with transfer accuracy across the encoder population is higher, with a bootstrap confidence interval that excludes zero difference. A second test is whether $\rho$ keeps significant partial correlation with accuracy after controlling for effective rank. Dirichlet energy is included as the weaker, previously discredited reference.
Strongest transfer method. This is GAUGE. It reports 61.26% 1-shot accuracy on PubMed against 56.82% for RiemannGFM, and 93.88% zero-shot link-prediction AUC on Facebook against about 87% for the best baseline. We use its pre-training sources and target protocol so these figures stay comparable. For a second anchor on ogbn-arxiv, GFT reports 26.49% 1-shot accuracy on the 40-way task against 21.34% for the nearest competitor. For heterophilic in-domain context, GSAN (Geometric Scattering Attention Networks) reports 61.2% on Chameleon against 28.2% for GCN. Our cross-domain encoders are not expected to reach this, but the gap quantifies what transfer costs. For air-traffic transfer and for absolute Texas/Cornell accuracies, no reported result was found on Lacuna for those settings. struc2vec reports only a relative 50% gain over node2vec on Brazilian air traffic, and FB-GNN reports only a roughly 10% gain over GCN on Texas/Cornell.
Ablations isolating the key ingredient. The descriptor-distillation encoder is compared against four alternatives at matched parameters, pre-training steps and hyperparameter search budget (the same number of trials for each). The alternatives are the identical encoder without the role loss; the same encoder with GraphWave concatenated at the input; GraphWave concatenated to the frozen output (late fusion); and the encoder with GP2F's fusion loss added. The intervention counts as a win only if it beats late fusion on targets where labels depend jointly on role and features. Matching GAUGE on PubMed 1-shot while being a plain Euclidean GNN would also count as a win.
Why not just…?
Why not just concatenate GraphWave features and skip the learned encoder's role problem? This is the most serious threat, and the proposal is built so that it can win. GraphWave features are already domain-agnostic, so late fusion may match any learned role-preserving encoder on role-driven targets like air traffic. The ablation above settles this. The case for learning roles rests on targets where meaning comes from a role interacting with features, such as a hub whose label depends on its topic. A linear head on concatenated features cannot express that interaction, and CenPre needed an alignment module to stop structural priors from overwriting semantics. We also test late fusion with a two-layer MLP head, so the comparison is not stacked against it. If late fusion still wins, we will report that role information should be supplied analytically rather than learned. The diagnostic contribution stands either way.
Why not just use effective rank, which already tracks accuracy well? Effective rank measures whether the representation is spread across many directions, not whether those directions encode anything transferable. Its own authors note that high rank does not imply utility. The audit tests a specific prediction: adding GP2F's fusion loss, or deepening a GCN, should lower $\rho$ while leaving effective rank roughly unchanged, and transfer should follow $\rho$. If effective rank explains transfer equally well, our measure is redundant and we will say so.
Why not just use shallow encoders and linear probing? GP2F reports that linear probing often beats elaborate graph prompting under strong domain shift, and shallow models avoid deep smoothing. But the 2-layer encoders in our population test whether shallowness alone suffices. Shallow encoders see little context, and GFT's argument for deeper computation trees is that wider structure matters. If 2-layer encoders dominate at every target, the practical advice is simply "stay shallow," and role collapse matters only as an explanation of why.
Why not just use GAUGE? GAUGE is the strongest reported transfer model and serves as our main baseline. However, its Riemannian machinery (QR decompositions, transport between fibers) is costly. Its Dirichlet-based transfer criterion also offers no guarantee that distant, structurally equivalent nodes receive similar representations. If GAUGE shows high $\rho$ and wins outright, that supports our measure as an explanation of its success. If it shows low $\rho$ on heterophilic targets, that locates where it should fail.
Proposed Experiments and Decisive Checks
The first check uses synthetic planted roles. Following GraphWave's setup, we plant house, star and bridge motifs on base graphs of differing density and degree distribution. We pre-train on one family and evaluate on another, where true roles are known exactly. We plot $\rho_\ell$, effective rank and role-classification accuracy against depth. Role collapse is supported if role accuracy and $\rho$ fall together while effective rank stays flat.
The second check is real role transfer on air-traffic networks. We pre-train on two of Brazil, USA and Europe and run few-shot classification on the third. These labels are close proxies for structural role and share no features, so this is the cleanest real test.
The third check is the main audit. Using GAUGE's pre-training sources, we evaluate 1-shot and 5-shot node classification on Cora, CiteSeer, PubMed, Chameleon, Squirrel, Texas and Cornell, plus zero-shot link prediction on Facebook. We report the rank correlation of each diagnostic with accuracy, separately for homophilic and heterophilic targets, with bootstrap intervals over random seeds and splits.
The fourth check is causal. Holding backbone and depth fixed, we toggle the fusion loss and the role-distillation loss. If $\rho$ is a meaningful mechanism and not merely a correlate, changes in $\rho$ should predict changes in transfer accuracy in the direction the hypothesis states. The fusion loss should hurt most on heterophilic targets.
The program fails cleanly if either of two things happens. One is that $\rho$ adds nothing over effective rank in partial correlation. The other is that toggling the fusion loss moves accuracy without moving $\rho$. Either outcome would show that role collapse is not the channel through which over-smoothing harms transfer.
References
- Are we measuring oversmoothing in graph neural networks correctly?
- Learning Structural Node Embeddings via Diffusion Wavelets
- struc2vec: Learning Node Representations from Structural Identity
- Deep Graph Infomax
- Centrality-guided Pre-training for Graph
- GFT: Graph Foundation Model with Transferable Tree Vocabulary
- GP2F: Cross-Domain Graph Prompting with Adaptive Fusion of Pre-trained Graph Neural Networks
- Complete the Missing Half: Augmenting Aggregation Filtering with Diversification for Graph Convolutional Networks
- Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles
- Geometric Scattering Attention Networks