Addressing Feature Homogenization in Deep Graph Neural Networks

This direction is merged into another direction: graph over-smoothing (over-smoothing). Its last version stays readable and citable here.

Version 1 of 1127 papers128 conceptsfrozen 2026-10-04

In standard convolutional neural networks, depth is a virtue. Adding layers allows the model to learn increasingly abstract features, moving from simple edges to complex objects. In Graph Neural Networks (GNNs), however, depth has historically been a curse. As you stack more layers to capture long-range dependencies, a phenomenon known as over-smoothing occurs: node representations converge toward a uniform state, becoming indistinguishable from one another.

A reader should understand that over-smoothing is not just a training difficulty; it is a fundamental mathematical consequence of how GNNs aggregate information.

The Low-Pass Filter Problem

At its core, a standard GNN layer performs a weighted average of a node’s neighbors. Mathematically, this is equivalent to a Laplacian smoothing operation. While this helps capture local context, Xupeng Miao and colleagues in DeGNN provide an information-theoretic proof that this information decays exponentially with depth.

Think of it as a low-pass filter. Each layer washes away high-frequency signals (the unique differences between nodes) and preserves only the low-frequency signals (the global average). By the time a signal passes through ten or twenty layers, every node in a connected component essentially holds the same "blurry" representation. This loss of distinctiveness, as noted by Hao Cheng 0016, prevents the model from performing fine-grained classification.

Breaking the Convergence

To build deeper GNNs, researchers have moved beyond simple averaging toward mechanisms that "anchor" node identities or introduce "repulsive" forces to prevent feature collapse.

1. Topological and Curvature-Aware Rewiring

If the graph structure itself causes the smoothing, one solution is to change the graph. Dai Shi suggests using discrete graph Ricci curvature to identify edges that bridge different communities. By re-weighting or dropping these edges, the model can prevent information from bleeding across class boundaries. Similarly, Jiaqi WANG introduces Structural Balanced Propagation (SBP), which treats the graph as a signed network. By explicitly using "negative" edges to repel dissimilar nodes, SBP maintains class distinction even at depths of 300 layers.

2. Physics-Inspired Dynamics

Another powerful intuition treats the GNN as a dynamical system. Tuan Nguyen proposes KuramotoGNN, which models nodes as coupled oscillators. By assigning each node a "natural frequency," the system resists collapsing into a single synchronized state.

Taking this further, Cristian Bodnar and Pietro Lio introduce Neural Sheaf Diffusion (NSD). In standard GNNs, the "geometry" of the graph is fixed. NSD learns a custom algebraic structure (a cellular sheaf) on the edges. This allows the model to decide, per edge, whether to smooth features (diffusion) or sharpen them (anti-diffusion), effectively giving the network a "focus" knob to preserve local detail.

3. Geometric and Manifold Constraints

Standard aggregations are "contractive"—they pull points together in Euclidean space. Dongzhuoran Zhou argues that by shifting aggregation to compact manifolds, such as the surface of a sphere, we can prevent features from collapsing to a single point. Because the manifold has a fixed "volume" or curvature, node representations are forced to remain distributed, preserving diversity across hundreds of layers.

Decoupling Depth from Scope

A common misconception is that depth and receptive field must be linked. In a standard GCN, $L$ layers mean a node sees $L$ hops away. Hanqing Zeng and Muhan Zhang challenge this with SHADOW-GNN. They decouple these properties by extracting a local subgraph first and then applying a deep GNN only to that localized context. This allows for deep feature extraction without the "neighborhood explosion" or global smoothing that typically ruins large-scale graph learning.

Why These Approaches Work

The transition from shallow to deep GNNs relies on a shift in perspective:

  • From Passive to Active: Instead of passively averaging neighbors, models like Attentive GNN use triple-attention to selectively remember important features.
  • From Euclidean to Topological: By using sheaves or fractional Laplacians (Aras Bacho), models gain the mathematical flexibility to resist the "gravity" of the global mean.
  • From Heuristic to Principled: Frameworks like TWIRLS, proposed by Yongyi Yang, anchor GNN layers to energy functions derived from classical optimization. This ensures that even as the network gets deeper, it remains tethered to the original input features, preventing the signal from drifting into uniformity.

By treating over-smoothing as a problem of signal preservation and dynamical stability, these researchers have transformed GNNs from shallow local filters into deep, expressive architectures capable of capturing the complex, long-range structures of the real world.

Papers in this version

Showing 60 of 127 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_addressing_feature_homogenization_in_dee_c42cf016,
  title = {Addressing Feature Homogenization in Deep Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/addressing-feature-homogenization-in-deep-graph-neural-networks/rdr_c42cf0165b9c4a6a9588d214c5f6761d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_addressing_feature_homogenization_in_dee_c42cf016_v1,
  title = {Addressing Feature Homogenization in Deep Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/addressing-feature-homogenization-in-deep-graph-neural-networks/rdr_c42cf0165b9c4a6a9588d214c5f6761d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}