Abstract
The BO-NNC framework addresses the challenge of training Graph Neural Networks (GNNs) on datasets with noisy labels. By integrating diverse self-supervised teacher models with a bi-level optimization strategy, the framework dynamically adjusts teacher weights to filter label noise. Experimental results across five benchmark datasets demonstrate that BO-NNC significantly outperforms state-of-the-art methods, maintaining high accuracy even under extreme noise conditions.
Introduction
Graph Neural Networks (GNNs) typically rely on the assumption of clean, ground-truth labels for node classification. However, real-world graph data often contains "noisy" labels—incorrectly categorized nodes that can severely degrade model performance. Existing Noisy Node Classification (NNC) methods often suffer from confirmation bias when using a single model to clean data or rely on static, hand-tuned weights in multi-teacher distillation setups. To overcome these limitations, we propose BO-NNC, a framework that leverages diverse self-supervised teachers and an adaptive bi-level optimization strategy to robustly classify nodes in the presence of noise.
Methodology
The BO-NNC framework consists of three integrated components: Multi-teacher Construction, Multi-teacher Distillation via Bi-level Optimization, and a Label Improvement Module.
- Multi-teacher Construction: Instead of training teachers on noisy labels, BO-NNC employs multiple self-supervised learning (SSL) methods (e.g., DGI, GCA, SUGRL). These methods learn high-quality node representations from graph structure and features alone, ensuring the teachers' "opinions" are not biased by label noise.
- Bi-level Optimization: This core component manages the distillation process through two nested loops. In the lower level, the student model learns to predict soft labels, which are a weighted combination of the teachers' outputs. In the upper level, the system evaluates the student on a set of automatically identified "clean nodes." The resulting feedback is used to update the Teacher Weight Matrix, allowing the model to dynamically prioritize the most reliable teachers.
- Label Improvement Module: This module filters out likely noisy labels based on high loss values and replaces them with high-confidence pseudo-labels generated by the ensemble of student and teacher models.
Experiments and Results
The framework was evaluated on five benchmark datasets: Cora, Citeseer, DBLP (citation networks), and Amazon Computers and Photo (business networks). Noise was simulated using Uniform and Pair noise at rates from 20% to 60%.
BO-NNC consistently outperformed both standard GNNs and specialized NNC baselines. For instance, on the Cora dataset with 60% uniform noise, BO-NNC achieved 73.16% accuracy, compared to just 41.76% for a standard GCN. Compared to the best-performing unsupervised baseline (DGI), BO-NNC showed average improvements between 2.93% and 13.44% across different noise levels.
Key insights from the evaluation include:
- Diversity as Defense: Self-supervised teachers provide a robust alternative to noise-poisoned supervised models.
- Dynamic Trust: Bi-level optimization effectively "learns how to learn" from teachers, adapting to their varying reliability during training.
- Clean Node Heuristic: Leveraging the observation that models learn clean patterns before memorizing noise allows for effective validation even in noisy environments.
Conclusion
BO-NNC provides a robust solution for node classification on noisy graphs by combining self-supervised diversity with adaptive bi-level optimization. While the current implementation faces computational challenges due to second-order gradients—resulting in $O(n^2)$ complexity—future work will focus on gradient approximation techniques to scale the framework to massive graphs.
Summary updated