Abstract
Graph Neural Networks (GNNs) typically assume that both the graph structure and node labels are accurate. However, real-world data is frequently "noisy," containing structural errors and incorrect annotations. This paper presents RNCGLN (Robust Node Classification under Graph and Label Noise), a framework designed to simultaneously address these challenges. RNCGLN utilizes a multi-head self-attention backbone for global learning, complemented by a graph self-improvement module for local structural consistency and a label self-improvement module to rectify noisy annotations. Our evaluations on standard benchmarks show that RNCGLN significantly outperforms existing robust methods, maintaining stability even under severe noise conditions.
Introduction
Standard GNNs rely on message-passing mechanisms that are highly sensitive to data quality. Graph noise, such as missing or spurious edges, leads to the aggregation of irrelevant information. Simultaneously, label noise—incorrect node categories—spreads misinformation through the network during training. While previous research has addressed these issues in isolation, RNCGLN tackles them jointly, solving the "chicken and egg" problem where clean graphs are needed for accurate labels and vice versa.
Methodology: The RNCGLN Framework
The RNCGLN framework consists of three integrated modules designed to foster mutual reinforcement between graph structure and label accuracy.
1. Multi-Head Self-Attention (Global Learning)
To bypass potentially noisy graph structures, RNCGLN employs self-attention as its backbone. This treats the graph as fully connected but learns task-relevant weights for each connection. This mechanism allows the model to naturally ignore noisy edges and discover latent relationships between nodes.
2. Graph Self-Improvement (Local Learning)
To preserve essential local structures, the model incorporates a Graph Contrastive Loss ($L_{gsi}$). This loss ensures that representations of multi-hop neighbors remain similar. The module iteratively updates the graph structure using pseudo-graphs generated from high-confidence predictions, effectively pruning noise and reinforcing reliable connections.
3. Label Self-Improvement
To mitigate incorrect annotations, RNCGLN uses pseudo-labeling. When the model achieves high confidence that an original label is erroneous, it replaces it with its own prediction for subsequent training iterations. This iterative process helps the model converge toward a self-consistent state.
Experiments and Results
The model was evaluated on four benchmark datasets: Cora, Citeseer, Pubmed, and Amazon Photo. Noise was manually injected into both the graph structure and labels to test robustness.
Key Findings
- Superior Performance: RNCGLN consistently achieved the highest accuracy across all noise levels. It improved accuracy by an average of 3.50% over the best existing robust methods (e.g., LAFAK) and 7.32% over standard GAT.
- Resilience to High Noise: Unlike baseline models such as IDGL or NRGNN, which often fail when both graph and label noise are present, RNCGLN remains stable.
- Ablation Insights: Removing the graph contrastive loss resulted in performance drops of up to 20%, highlighting the necessity of local structural information even in a global attention-based model.
Discussion and Conclusion
RNCGLN demonstrates that global self-attention and local contrastive learning are complementary strategies for robustness. By cleaning the graph and labels in tandem, the model achieves a state of mutual reinforcement.
While the model is highly effective, it introduces sensitivity to the hyperparameter $\alpha$, which balances graph and label information. Future work will focus on automated methods for noise balancing and improving the scalability of self-attention for extremely large-scale graphs.
Summary updated