Counteracting Feature Homogenization in Deep Graph Neural Networks

This direction is merged into another direction: graph over-smoothing (over-smoothing). Its last version stays readable and citable here.

Version 1 of 1248 papers250 conceptsfrozen 2026-10-04

Standard Graph Neural Networks (GNNs) face a frustrating paradox: while deeper architectures in computer vision or natural language processing lead to richer representations, increasing the depth of a GNN often causes performance to plummet. After just a few layers of message passing, node representations tend to converge toward a uniform value, becoming statistically indistinguishable. This phenomenon, known as over-smoothing, prevents models from capturing the long-range dependencies necessary for complex graph reasoning.

Research in this area focuses on breaking this "shallow bottleneck" by rethinking how information propagates across a network.

The Geometry of Feature Collapse

To understand why over-smoothing happens, we must view message passing as a form of Laplacian smoothing. In a typical GNN layer, a node updates its state by averaging the features of its neighbors. Mathematically, this acts as a low-pass filter on the graph signal. As Weichen Zhao and colleagues demonstrate in Understanding Oversmoothing in Diffusion-Based GNNs, this process is fundamentally driven by the ergodicity of the diffusion operator. Just as a drop of ink eventually colors an entire glass of water uniformly, repeated aggregation washes out the unique "high-frequency" local features of individual nodes.

To quantify this fade, Deli Chen and Yankai Lin introduced metrics like MAD (Mean Average Distance) in Measuring and Relieving the Over-smoothing Problem. They observed that as depth increases, the distance between node embeddings shrinks toward zero, signaling a loss of discriminative power.

Adaptive Propagation and Reweighting

One way to prevent collapse is to make the message-passing process more selective. If every neighbor is treated equally, noise dominates. Jianpeng Chen and Ming Zeng proposed CustomGNN, which uses an LSTM-based path reweighting mechanism to highlight task-relevant connections while filtering out the structural noise that accelerates smoothing.

Similarly, Taiji Suzuki and Kishan Wimalawarne developed AdaGPR, which applies Generalized PageRank coefficients to each layer. By learning these coefficients, the model can adaptively decide how much influence the local neighborhood versus the global structure should have at different depths, effectively balancing local distinctiveness with long-range context.

From Discrete Layers to Continuous Flows

A significant shift in the field involves moving away from discrete message-passing steps toward continuous dynamical systems. By modeling node evolution through Neural Ordinary Differential Equations (ODEs), researchers can treat the "depth" of a network as a continuous time variable.

In Continuous Graph Neural Networks, Jian Tang and Meng Qu introduced a "restart" mechanism that constantly anchors the evolving node features to their original values. This prevents the system from drifting into a uniform state, allowing for effectively infinite-depth networks. This continuous perspective is further extended by Francesco Di Giovanni in Understanding convolution on graphs via energies, where GNNs are framed as physical systems. By adjusting channel-mixing weights, the model can induce "repulsion" between nodes—essentially acting as a high-pass filter—to counteract over-smoothing.

Structural Rewiring and Highways

If the graph's original topology is the bottleneck, why not change the graph? Over-squashing is a related problem where information from a large neighborhood is compressed into a single fixed-size vector, leading to bottlenecks. To solve this, Benjamin Gutteridge and Michael M. Bronstein proposed DRew (Dynamically Rewired Message Passing). DRew gradually adds long-range "highway" edges as the network gets deeper, allowing distant nodes to communicate directly without passing through dozens of intermediate, smoothing-heavy layers.

Deli Chen also explored this in HighwayGraph, which adds edges between distant but semantically similar nodes. This "shortcut" approach ensures that long-range dependencies are captured by the graph's structure itself, rather than relying on the model to survive the signal degradation of deep propagation.

The Path to Graph Foundation Models

As the field matures, these techniques are converging toward more robust, scalable architectures. Fuzhen Zhuang and Yu Zhao suggest in their comprehensive survey that solving over-smoothing is a prerequisite for "Graph Foundation Models." By decoupling feature propagation from the learning phase—as seen in the training-free hypergraph methods of Bohan Tang—researchers are finding ways to integrate global structural knowledge without the representational collapse that once defined deep graph learning.

Papers in this version

Showing 60 of 248 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_counteracting_feature_homogenization_in__75c09b4b,
  title = {Counteracting Feature Homogenization in Deep Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/counteracting-feature-homogenization-in-deep-graph-neural-networks/rdr_75c09b4b93414f75bae9555e6394eef9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_counteracting_feature_homogenization_in__75c09b4b_v1,
  title = {Counteracting Feature Homogenization in Deep Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/counteracting-feature-homogenization-in-deep-graph-neural-networks/rdr_75c09b4b93414f75bae9555e6394eef9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}