Introduction
Graph Neural Networks (GNNs) have revolutionized the processing of graph-structured data by aggregating information from local neighborhoods. However, capturing high-order information—neighbors multiple hops away—remains a challenge. Traditional approaches often rely on stacking multiple layers, which can lead to over-smoothing, where node representations become indistinguishable. While methods like Personalized PageRank (PPR) allow for deeper propagation without stacking, they typically use fixed, hand-tuned coefficients that do not adapt to specific dataset characteristics.
To address these limitations, the authors introduce Contrastive Adaptive Propagation Graph Neural Networks (CAPGNN). This framework makes high-order neighbor propagation both learnable and efficient by combining an attention-based coefficient model with a self-supervised, "negative-free" contrastive loss.
Methodology
The CAPGNN framework is built upon three primary technical pillars designed to balance computational efficiency with model adaptability:
- Adaptive Extended Propagation: Unlike models with fixed propagation schemes, CAPGNN utilizes a coefficient-attention model. While it initializes weights based on PPR, it allows the network to learn and dynamically adjust the influence of local versus high-order neighbors during the training process.
- Efficient Power Iterations: To maintain scalability, propagation is implemented as a series of sparse matrix-vector multiplications (power iterations). This approach avoids the memory overhead of dense matrix operations while enabling the model to reach distant neighbors (e.g., $K=10$ hops).
- Negative-free Entropy-aware Contrastive Loss: To leverage unlabeled data, CAPGNN employs self-supervised learning. It generates two augmented views of the graph via dropout and encourages representation consistency across them. This "negative-free" approach eliminates the need for large memory banks of negative samples and includes a sharpening technique to promote confident, low-entropy predictions.
The authors offer two variants: CAPGCN, which utilizes static graph structures, and CAPGAT, which incorporates dynamic, attention-based structures.
Key Results and Insights
CAPGNN was evaluated on standard benchmarks, including citation networks (Cora, CiteSeer, Pubmed) and e-commerce networks (Amazon-Computers, Amazon-Photo). The model consistently matched or outperformed state-of-the-art baselines such as GCN, GAT, APPNP, and GRAND.
Beyond Fixed Heuristics
A central finding is that learnable coefficients allow the model to discover optimal propagation patterns for different graphs. For instance, visualization of learned coefficients showed that while traditional methods like APPNP may over-weight distant hops, CAPGNN learns a smoother, decaying distribution tailored to the data. This adaptability is key to its superior performance.
Efficiency and Robustness
By decomposing the propagation polynomial into power iterations, CAPGNN achieves the representational power of deep GNNs with the computational footprint of shallow ones. Furthermore, the model demonstrated steady performance once the number of iterations $K$ reached 10, indicating lower sensitivity to hyperparameter tuning compared to stacked-layer architectures.
Conclusion
CAPGNN demonstrates that making propagation schemes learnable and adaptive significantly enhances GNN performance without sacrificing efficiency. While the low-entropy assumption for unlabeled data may vary in effectiveness across datasets (e.g., requiring tuning on Pubmed), the framework provides a robust foundation for high-order graph learning. Future research will explore applying this adaptive propagation to recommendation systems and large-scale social network analysis.
Summary updated