In computer vision, stacking hundreds of layers is the standard path to high performance. In Graph Neural Networks (GNNs), however, depth has historically been a liability. As layers are added, node representations tend to converge toward a single, uniform vector—a phenomenon known as over-smoothing.
Recent research has revealed that this collapse is not uniform across the graph. Instead, it is a topological process where the rate of information loss is dictated by the connectivity of individual nodes and the spectral properties of the graph itself.
The Geometry of Information Loss
The fundamental reason GNNs smooth is that graph convolutions act as low-pass filters. Kenta Oono demonstrated in Graph Neural Networks Exponentially Lose Expressive Power for Node Classification that the rate of this feature collapse is tied to the spectral gap of the graph's normalized Laplacian. In dense graphs with high connectivity, this gap is larger, causing the signal to disappear exponentially faster than in sparse networks.
This relationship between density and smoothing manifests at the local level as well. Ming Chen and Zhewei Wei observed in Simple and Deep Graph Convolutional Networks that high-degree nodes are the first to lose their distinctiveness. Because these "hub" nodes aggregate information from a vast number of neighbors, their receptive fields expand and overlap with the rest of the graph almost immediately. Jingbo Zhou and Yixuan Du further clarified this in Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptive Residual Module, noting that the rapid expansion of neighborhood subgraphs for well-connected nodes causes their identities to blend into a stationary state long before sparsely connected nodes.
Preserving Identity through Residuals
To counter this collapse, researchers have moved away from uniform architectures toward "topology-adaptive" mechanisms. The goal is to allow the network to go deep while ensuring that each node retains its unique "anchor" to the original input features.
The GCNII model, introduced by Ming Chen and colleagues, uses two primary techniques:
- Initial Residual Connections: Unlike standard ResNets that connect layer $l$ to $l+1$, GCNII connects every layer back to the initial input features. This ensures that the raw signal is never more than one step away, regardless of depth.
- Identity Mapping: By adding an identity matrix to the weight matrix at each layer, the model ensures that it can learn to pass information through without transforming it, effectively allowing the network to choose its own optimal depth.
Building on this, Jingbo Zhou proposed the PSNR module. Recognizing that high-degree nodes need more "protection" from smoothing than low-degree nodes, PSNR learns node-specific residual coefficients. By sampling these coefficients from a posterior distribution, the model adaptively decides how much of the original identity to preserve for each node based on its local neighborhood density.
Dynamic Depth and Frequency Control
Another approach to the over-smoothing problem involves explicitly controlling the "mixing" of information. Ye Tang and Xuesong Yang developed NDGGNET, which uses node-independent gates. In NDGGNET-A Node Independent Gate based Graph Neural Networks, they show that by tailoring the aggregation gate to a node's specific connectivity, the network can prevent high-degree nodes from being overwhelmed by their neighbors' features.
While GCNs are low-pass filters, Yimeng Min and Guy Wolf argue in Scattering GCN that we should also incorporate band-pass filters. By using geometric scattering transforms, they capture high-frequency structural information that standard GCNs smooth away. This multiscale approach is particularly effective in dense graphs where the low-pass signal collapses almost instantly.
Finally, Xuan Jiang and Zhiyong Yang 0001 explored the use of higher-order patterns in A Sparse-Motif Ensemble Graph Convolutional Network against Over-smoothing. They caution that simply adding more connections (like motifs) can inadvertently accelerate smoothing by increasing graph density. Their solution, SMEGCN, uses a hierarchical aggregation that maintains structural diversity without the "shortcut" effect that leads to representation collapse.
Summary of the Shift
The evolution of deep GNNs is a shift from treating the graph as a uniform grid to treating it as a complex, non-uniform landscape. By acknowledging that connectivity density drives the rate of smoothing, researchers have developed models that adapt their depth, residual strength, and frequency response to the local topology of every node. This allows GNNs to scale to 64 layers and beyond, capturing long-range dependencies without sacrificing the distinct identities of the nodes within the graph.