Introduction
Graph Neural Networks (GNNs) have revolutionized representation learning on non-Euclidean data. However, a persistent challenge in the field is the "shallow" nature of most successful GNN architectures, which typically utilize only 2 or 3 layers. While sufficient for simple graphs, real-world data often requires capturing long-range dependencies. Deepening GNNs traditionally leads to vanishing gradients, over-smoothing (where node representations become indistinguishable), and the "shading neighbors" effect, where distant information inappropriately drowns out local context.
The Deeper-GXX Framework
To overcome these limitations, we introduce Deeper-GXX, a model-agnostic framework designed to scale GNNs to dozens of layers. The framework integrates two primary innovations into arbitrary GNN backbones:
- Weight-Decaying Graph Residual Connection (WDG-ResNet): Standard residual connections can amplify distant noise. WDG-ResNet employs a decaying factor ($e^{-l/\lambda}$) to preserve the hierarchical importance of local neighborhoods while still allowing access to deep information.
- Topology-Guided Graph Contrastive Loss (TGCL): This contrastive objective mitigates over-smoothing by pulling 1-hop neighbors together in the feature space and pushing non-neighbors apart, ensuring discriminative features are maintained at depth.
Experimental Evaluation
Deeper-GXX was evaluated on standard benchmarks including citation networks (Cora, CiteSeer, PubMed), social networks (Reddit), and large-scale graphs (OGB-Arxiv).
Key Results
- Stability at Depth: On the Cora dataset, a 60-layer Deeper-GXX model achieved 80.59% accuracy, whereas a vanilla GCN collapsed to 29.62% due to over-smoothing.
- Competitive Performance: Deeper-GXX consistently outperformed state-of-the-art deep GNN baselines, including GCNII, DGN, and PairNorm.
- Robustness: The model demonstrates significant resilience to missing data. By leveraging its depth, it can "repair" features by borrowing information from distant, similar nodes when local features are masked.
Conclusion and Future Work
Deeper-GXX successfully addresses the classic pitfalls of deep GNNs. A key insight from this work is the connection between the optimal decay hyperparameter and the graph diameter, providing a physical heuristic for model tuning. Future research will focus on optimizing the computational cost of the contrastive loss for massive-scale graphs and exploring multi-hop community-aware sampling strategies.
Summary updated