L2DGCN: Learnable Enhancement and Label Selection Dynamic Graph Convolutional Networks for Mitigating Degree Bias

jingxiao zhang, Shifei Ding, Jian Zhang, Lili Guo, Xuan LiNeurIPS 2025 · OpenReview

Abstract & TL;DR

L2DGCN tackles the "degree bias" problem in Graph Neural Networks (GNNs), where nodes with few connections are often misclassified due to insufficient information flow. By combining a teacher-student framework with dynamic graph restructuring and a high-confidence pseudo-label selector, the model significantly outperforms state-of-the-art methods in "sparse labeling" scenarios (e.g., where as few as 3 labels per class are available).

The "Degree Bias" Problem

Most GNNs rely on a message-passing mechanism: nodes learn representations by aggregating information from their neighbors. This creates a fundamental "degree bias." High-degree nodes (hubs) are flooded with information, while low-degree nodes (the "long tail" of the power-law distribution) remain starved of context.

This problem is exacerbated in semi-supervised learning when labeled data is scarce. If a low-degree node is several hops away from a labeled node, the message-passing process often fails to deliver useful signals, leading to poor classification. Existing solutions often struggle to balance global structure with local noise or fail to handle the unreliability of pseudo-labels used to fill the gaps.

Proposed Methodology: L2DGCN

To address these challenges, the authors propose the Learnable Enhancement and Label Selection Dynamic Graph Convolutional Network (L2DGCN), which utilizes a two-pronged teacher-student architecture.

L2DGCN Architecture
  1. The Teacher Model: It employs an improved soft label propagation mechanism. Unlike standard propagation that resets labels every iteration, this version allows label information to disseminate over long ranges, creating a global "label view" for all nodes, including isolated ones.
  2. The Student Model: This acts as the "active learner," using a decoupled GCN backbone that separates feature transformation from propagation. It features two key innovations:
  3. Dynamic Structural Optimization: The model prunes edges from high-degree nodes to reduce noise and uses a K-Nearest Neighbor (KNN) approach based on feature similarity to add "virtual" edges to low-degree nodes. This ensures a more balanced information flow.
  4. Pseudo-Label Selector: To avoid "garbage in, garbage out," a selector identifies only the most confident pseudo-labels from the teacher to update the student's training set.
  5. Contrastive Learning: The model uses an InfoNCE contrastive loss to ensure that the representations learned by the student remain consistent with the labels provided by the teacher, helping the model capture deeper feature associations.

Datasets and Evaluation

The model was evaluated on six standard homogeneous graph benchmarks, focusing specifically on "low labeling rates" (testing performance with only 3, 5, or 10 labeled nodes per category).

Dataset Statistics and Hyperparameters

The datasets span various domains:

  • Citation Networks: Cora, Citeseer, and Pubmed.
  • Academic Collaboration: Coauthor-CS and Coauthor-Physics.
  • E-commerce Metadata: Amazon-Photo.

Key Insights

  • Pruning is as Important as Adding: Most methods focus solely on adding edges to help low-degree nodes. L2DGCN demonstrates that pruning edges from high-degree nodes is equally vital to reduce structural noise and prevent "over-smoothing" (where all node embeddings become indistinguishable).
  • Decoupling for Depth: By using a decoupled GCN, the model can propagate information over many steps without the massive computational overhead or vanishing gradient issues typical of traditional deep GCNs.
  • The Power of Contrastive Loss: Ablation studies revealed that the contrastive loss is a massive performance driver. In extreme cases (like Cora 3-shot), removing it caused a performance drop of over 20 percentage points, proving its essential role in aligning features with sparse labels.

Key Results

L2DGCN consistently outperformed classical GNNs (GCN, GAT) and specialized few-shot models (AGST, CGPN).

  • Extreme Scarcity (3-shot): On the Cora dataset with only 3 labels per class, L2DGCN achieved 78.50% accuracy, a 6.62% improvement over the next best baseline.
  • Consistency: The model showed superior performance across almost all datasets in 3-shot, 5-shot, and 10-shot settings. For instance, on Coauthor-CS, it maintained a lead of ~2.5% even as more labels were added.
  • Mitigating Bias: Qualitative analysis showed that the performance gap between high-degree and low-degree nodes narrowed significantly compared to standard GCNs.

Limitations and Future Work

The authors identify a few areas for improvement. Primarily, the graph enhancement strategy relies heavily on node features to create new edges. If the initial features are noisy or the graph is "low-homogeneous" (heterophilic, where connected nodes are often from different classes), this could introduce undesirable edges. Future work will explore more robust augmentation techniques capable of handling heterophilic graphs and developing models less sensitive to initial node feature quality.

Summary updated

Abstract

Graph Neural Networks (GNNs) are powerful models for node classification, but their performance is heavily reliant on manually labeled data, which is often costly and results in insufficient labeling. Recent studies have shown that message-passing neural networks struggle to propagate information in low-degree nodes, negatively affecting overall performance. To address the information bias caused by degree imbalance, we propose a Learnable Enhancement and Label Selection Dynamic Graph Convolutional Network (L2DGCN). L2DGCN consists of a teacher model and a student model. The teacher model employs an improved label propagation mechanism that enables remote label information dissemination among all nodes. The student model introduces a dynamically learnable graph enhancement strategy, perturbing edges to facilitate information exchange among low-degree nodes. This approach maintains the global graph structure while learning graph representations. Additionally, we have designed a label selector to mitigate the impact of unreliable pseudo-labels on model learning. To validate the effectiveness of our proposed model with limited labeled data, we conducted comprehensive evaluations of semi-supervised node classification across various scenarios with a limited number of annotated nodes. Experimental results demonstrate that our data enhancement model significantly contributes to node classification tasks under sparse labeling conditions.
BibTeX
@inproceedings{zhang2025l,
  author = {jingxiao zhang and Shifei Ding and Jian Zhang and Lili Guo and Xuan Li},
  title = {{L2DGCN: Learnable Enhancement and Label Selection Dynamic Graph Convolutional Networks for Mitigating Degree Bias}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=PoIhCjqzn0},
}
Figure