Abstract
Graph Neural Networks (GNNs), due to their capability to learn complex relations among attributed objects within graph datasets, have already been widely used in various graph mining tasks. In this paper, we introduce a novel GNN learning framework, called AKE-GNN (Adaptive-Knowledge-Exchange GNN), which adaptively exchanges diverse knowledge learned from multiple graph views generated by graph augmentations. Specifically, AKE-GNN iteratively exchanges redundant channels in the weight matrix of one GNN by informative channels of another GNN in a layer-wise manner. Extensive experiments on node classification, graph classification, and edge prediction demonstrate the effectiveness of AKE-GNN. On the Cora semi-supervised node classification dataset, our framework achieves new state-of-the-art results.
Introduction: From Pruning to Reactivation
Recent research has highlighted a significant characteristic of well-trained Graph Neural Networks (GNNs): high parameter redundancy. Many channels within the weight matrices are highly correlated, capturing nearly identical information. Traditionally, this redundancy has been addressed through pruning—removing underutilized neurons to compress the model.
However, AKE-GNN (Adaptive Knowledge Exchange GNN) proposes a paradigm shift. Rather than discarding redundant space, AKE-GNN treats it as "empty shelf space" that can be filled with complementary knowledge. By "reactivating" these redundant channels with informative features learned from different views of the same graph, the framework enhances model performance without increasing inference-time costs.
Methodology: The AKE-GNN Framework
The AKE-GNN framework operates through a symbiotic learning process involving two main phases: individual learning and knowledge exchange.
Phase 1: Individual Learning via Multi-View Augmentation
The process begins by generating multiple "views" of the input graph using stochastic augmentations. These include dropping edges, masking node features, or extracting subgraphs. A separate GNN is trained on each view. Because each augmentation emphasizes different structural or feature-based aspects, each GNN develops a unique set of "informative" channels.
Phase 2: Adaptive Knowledge Exchange
The core innovation of AKE-GNN is its "parameter surgery." The framework identifies and replaces redundant channels in a target GNN with high-information channels from a source GNN.
- Identifying Redundancy: Pearson correlation is used to detect pairs of output channels in the target GNN that are highly similar.
- Measuring Informativeness: Shannon Entropy is employed to measure the information density of channels in the source GNN. Channels with high entropy are considered to capture more diverse and useful features.
- The Exchange: Redundant channels in the target are replaced by high-entropy channels from the source. This exchange is performed layer-wise to maintain structural consistency, as different GNN layers correspond to different neighborhood subtree heights.
Following the exchange, the models undergo a brief re-training phase to integrate the newly "grafted" knowledge. Crucially, the final architecture remains unchanged, ensuring zero extra overhead during inference.
Experimental Results
AKE-GNN was evaluated across 15 public benchmark datasets, including citation networks (Cora, CiteSeer), chemical compounds (DD, PROTEINS), and large-scale benchmarks like OGBn-Arxiv.
- Consistent Performance Gains: AKE-GNN consistently outperformed eight popular GNN backbones (e.g., GCN, GAT, GCNII) across all tasks. On node classification, it provided an average absolute accuracy boost of 1.9% to 3.9%.
- New State-of-the-Art: The framework achieved a top-tier accuracy of 85.9% on the Cora dataset, surpassing existing multi-view and contrastive learning baselines.
- Mitigating Over-smoothing: By injecting diverse information, AKE-GNN acts as a regularizer, allowing GNNs to reach 50% to 66% greater depth than standard models before performance degrades.
- Efficiency: Unlike ensemble methods that multiply test-time computation, AKE-GNN delivers superior results using only a single model for prediction.
Conclusion
AKE-GNN demonstrates that model redundancy is not merely an efficiency hurdle but an opportunity for knowledge enrichment. By adaptively exchanging parameters between models trained on diverse graph views, it achieves state-of-the-art performance across a wide array of graph mining tasks. Future work will explore optimal view generation strategies and more automated exchange mechanisms to further refine this symbiotic learning approach.
Summary updated