Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation

Yujing Liu, Zongqian Wu, Zhengyu Lu, Ci Nie, Guoqiu Wen, Ping Hu, Xiaofeng Zhu

Abstract

The BO-NNC framework addresses the challenge of training Graph Neural Networks (GNNs) on datasets with noisy labels. By integrating diverse self-supervised teacher models with a bi-level optimization strategy, the framework dynamically adjusts teacher weights to filter label noise. Experimental results across five benchmark datasets demonstrate that BO-NNC significantly outperforms state-of-the-art methods, maintaining high accuracy even under extreme noise conditions.

Introduction

Graph Neural Networks (GNNs) typically rely on the assumption of clean, ground-truth labels for node classification. However, real-world graph data often contains "noisy" labels—incorrectly categorized nodes that can severely degrade model performance. Existing Noisy Node Classification (NNC) methods often suffer from confirmation bias when using a single model to clean data or rely on static, hand-tuned weights in multi-teacher distillation setups. To overcome these limitations, we propose BO-NNC, a framework that leverages diverse self-supervised teachers and an adaptive bi-level optimization strategy to robustly classify nodes in the presence of noise.

Methodology

The BO-NNC framework consists of three integrated components: Multi-teacher Construction, Multi-teacher Distillation via Bi-level Optimization, and a Label Improvement Module.

The BO-NNC framework architecture, illustrating the multi-teacher construction, bi-level distillation process, and label improvement module.
  1. Multi-teacher Construction: Instead of training teachers on noisy labels, BO-NNC employs multiple self-supervised learning (SSL) methods (e.g., DGI, GCA, SUGRL). These methods learn high-quality node representations from graph structure and features alone, ensuring the teachers' "opinions" are not biased by label noise.
  2. Bi-level Optimization: This core component manages the distillation process through two nested loops. In the lower level, the student model learns to predict soft labels, which are a weighted combination of the teachers' outputs. In the upper level, the system evaluates the student on a set of automatically identified "clean nodes." The resulting feedback is used to update the Teacher Weight Matrix, allowing the model to dynamically prioritize the most reliable teachers.
  3. Label Improvement Module: This module filters out likely noisy labels based on high loss values and replaces them with high-confidence pseudo-labels generated by the ensemble of student and teacher models.

Experiments and Results

The framework was evaluated on five benchmark datasets: Cora, Citeseer, DBLP (citation networks), and Amazon Computers and Photo (business networks). Noise was simulated using Uniform and Pair noise at rates from 20% to 60%.

BO-NNC consistently outperformed both standard GNNs and specialized NNC baselines. For instance, on the Cora dataset with 60% uniform noise, BO-NNC achieved 73.16% accuracy, compared to just 41.76% for a standard GCN. Compared to the best-performing unsupervised baseline (DGI), BO-NNC showed average improvements between 2.93% and 13.44% across different noise levels.

Sensitivity analysis across five datasets showing the stability of accuracy across different parameter settings.

Key insights from the evaluation include:

  • Diversity as Defense: Self-supervised teachers provide a robust alternative to noise-poisoned supervised models.
  • Dynamic Trust: Bi-level optimization effectively "learns how to learn" from teachers, adapting to their varying reliability during training.
  • Clean Node Heuristic: Leveraging the observation that models learn clean patterns before memorizing noise allows for effective validation even in noisy environments.

Conclusion

BO-NNC provides a robust solution for node classification on noisy graphs by combining self-supervised diversity with adaptive bi-level optimization. While the current implementation faces computational challenges due to second-order gradients—resulting in $O(n^2)$ complexity—future work will focus on gradient approximation techniques to scale the framework to massive graphs.

Summary updated

BibTeX
@inproceedings{liu2025noisy,
  author = {Yujing Liu and Zongqian Wu and Zhengyu Lu and Ci Nie and Guoqiu Wen and Ping Hu and Xiaofeng Zhu},
  title = {{Noisy Node Classification by Bi-level Optimization based Multi-teacher Distillation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i18.34095},
}
Figure