Abstract & TL;DR
L2DGCN tackles the "degree bias" problem in Graph Neural Networks (GNNs), where nodes with few connections are often misclassified due to insufficient information flow. By combining a teacher-student framework with dynamic graph restructuring and a high-confidence pseudo-label selector, the model significantly outperforms state-of-the-art methods in "sparse labeling" scenarios (e.g., where as few as 3 labels per class are available).
The "Degree Bias" Problem
Most GNNs rely on a message-passing mechanism: nodes learn representations by aggregating information from their neighbors. This creates a fundamental "degree bias." High-degree nodes (hubs) are flooded with information, while low-degree nodes (the "long tail" of the power-law distribution) remain starved of context.
This problem is exacerbated in semi-supervised learning when labeled data is scarce. If a low-degree node is several hops away from a labeled node, the message-passing process often fails to deliver useful signals, leading to poor classification. Existing solutions often struggle to balance global structure with local noise or fail to handle the unreliability of pseudo-labels used to fill the gaps.
Proposed Methodology: L2DGCN
To address these challenges, the authors propose the Learnable Enhancement and Label Selection Dynamic Graph Convolutional Network (L2DGCN), which utilizes a two-pronged teacher-student architecture.
- The Teacher Model: It employs an improved soft label propagation mechanism. Unlike standard propagation that resets labels every iteration, this version allows label information to disseminate over long ranges, creating a global "label view" for all nodes, including isolated ones.
- The Student Model: This acts as the "active learner," using a decoupled GCN backbone that separates feature transformation from propagation. It features two key innovations:
- Dynamic Structural Optimization: The model prunes edges from high-degree nodes to reduce noise and uses a K-Nearest Neighbor (KNN) approach based on feature similarity to add "virtual" edges to low-degree nodes. This ensures a more balanced information flow.
- Pseudo-Label Selector: To avoid "garbage in, garbage out," a selector identifies only the most confident pseudo-labels from the teacher to update the student's training set.
- Contrastive Learning: The model uses an InfoNCE contrastive loss to ensure that the representations learned by the student remain consistent with the labels provided by the teacher, helping the model capture deeper feature associations.
Datasets and Evaluation
The model was evaluated on six standard homogeneous graph benchmarks, focusing specifically on "low labeling rates" (testing performance with only 3, 5, or 10 labeled nodes per category).
The datasets span various domains:
- Citation Networks: Cora, Citeseer, and Pubmed.
- Academic Collaboration: Coauthor-CS and Coauthor-Physics.
- E-commerce Metadata: Amazon-Photo.
Key Insights
- Pruning is as Important as Adding: Most methods focus solely on adding edges to help low-degree nodes. L2DGCN demonstrates that pruning edges from high-degree nodes is equally vital to reduce structural noise and prevent "over-smoothing" (where all node embeddings become indistinguishable).
- Decoupling for Depth: By using a decoupled GCN, the model can propagate information over many steps without the massive computational overhead or vanishing gradient issues typical of traditional deep GCNs.
- The Power of Contrastive Loss: Ablation studies revealed that the contrastive loss is a massive performance driver. In extreme cases (like Cora 3-shot), removing it caused a performance drop of over 20 percentage points, proving its essential role in aligning features with sparse labels.
Key Results
L2DGCN consistently outperformed classical GNNs (GCN, GAT) and specialized few-shot models (AGST, CGPN).
- Extreme Scarcity (3-shot): On the Cora dataset with only 3 labels per class, L2DGCN achieved 78.50% accuracy, a 6.62% improvement over the next best baseline.
- Consistency: The model showed superior performance across almost all datasets in 3-shot, 5-shot, and 10-shot settings. For instance, on Coauthor-CS, it maintained a lead of ~2.5% even as more labels were added.
- Mitigating Bias: Qualitative analysis showed that the performance gap between high-degree and low-degree nodes narrowed significantly compared to standard GCNs.
Limitations and Future Work
The authors identify a few areas for improvement. Primarily, the graph enhancement strategy relies heavily on node features to create new edges. If the initial features are noisy or the graph is "low-homogeneous" (heterophilic, where connected nodes are often from different classes), this could introduce undesirable edges. Future work will explore more robust augmentation techniques capable of handling heterophilic graphs and developing models less sensitive to initial node feature quality.
Summary updated