Cleaning Noisy Labels by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation

Waqar Ahmed, Pietro Morerio, Vittorio Murino

Introduction

In the field of machine learning, Unsupervised Domain Adaptation (UDA) is a critical technique for deploying models in new environments. It involves adapting a model trained on a labeled "source" domain (e.g., synthetic images) to an unlabeled "target" domain (e.g., real-world photos). However, real-world constraints such as data privacy (medical records) or proprietary restrictions often make the original source data inaccessible during the adaptation phase. This challenge is known as Source-Free UDA.

The standard approach to Source-Free UDA involves using the source model to generate "pseudo-labels" for the target data. Unfortunately, the "domain shift" between source and target often leads to inaccurate guesses. Training a model on these noisy pseudo-labels typically causes it to memorize errors, leading to poor performance. This paper introduces Negative Ensemble Learning (NEL), a framework designed to clean these noisy labels without requiring access to the original source data.

Methodology

The core of the proposed framework is a "pseudo-label refinery" that leverages an ensemble of experts to progressively filter out noise. The architecture is built upon three primary pillars:

1. Negative Learning (NL)

Unlike standard "Positive Learning," which forces a model to map an input to a specific class (e.g., "This is a dog"), Negative Learning (NL) uses a complementary approach: "This image is not a cat." NL is inherently more robust to noise; even if the initial pseudo-label is incorrect, the probability that a randomly selected "negative" label is actually the true label is statistically very low. This "gentle" approach prevents the model from aggressively overfitting to incorrect labels.

2. Disjoint Residual Labels (DRL)

To ensure diversity within the ensemble, the authors introduce Disjoint Residual Labels. If a dataset has 10 classes and one is predicted as the pseudo-label, the remaining 9 "residual" labels are partitioned among the ensemble members. This forces different experts to learn from different negative constraints, preventing the ensemble from converging on a single incorrect consensus.

Figure 1: The NEL architecture showing the ensemble of experts, data augmentation branches, and the Negative Ensemble Learning loss feedback loop.

3. Adaptive Pseudo-Label Refinement

The system maintains a moving average of ensemble predictions to calculate confidence scores. It employs a dynamic threshold ($\gamma$) that starts strictly and relaxes as the model's accuracy improves. High-confidence samples are used for standard supervised training, while low-confidence samples are reassigned new labels based on the emerging ensemble consensus.

Experimental Setup

The authors evaluated NEL on four major UDA benchmarks:

  • Digit5: A collection of five digit datasets (MNIST, SVHN, USPS, MNIST-M, and Synthetic Digits).
  • PACS: A challenging benchmark featuring four domains: Photo, Art-Painting, Cartoon, and Sketch.
  • VisDA-C: A large-scale dataset focused on the synthetic-to-real domain gap.
  • DomainNet: The largest UDA benchmark, containing 6 domains and 345 categories.

Key Insights and Results

Progressive Cleaning

A key observation of the study is the natural separation of clean and noisy samples during training. Initially, most samples exhibit low confidence. As training progresses, "clean" samples (those where the pseudo-label matches the ground truth) migrate toward the high-confidence zone. Conversely, "noisy" samples remain in the low-confidence zone, where they can be safely identified and refined.

Figure 5: Histogram of sample probabilities at epoch 25, demonstrating the clear separation between clean and noisy samples in the high-probability region.

Performance Benchmarks

NEL achieved state-of-the-art results across all tested benchmarks, often outperforming methods that retained access to source data:

  • PACS: Achieved a significant jump in accuracy, outperforming previous methods by up to 21.8% in specific tasks.
  • Digit5: Reached an average accuracy of 94.4% in multi-source scenarios.
  • Noise Resilience: The method proved effective even when initial pseudo-labels were extremely noisy (e.g., only ~17% correct).

Conclusion

Negative Ensemble Learning provides a robust, GAN-free solution for Source-Free UDA. By focusing on what a sample is not and utilizing a diverse ensemble, the framework effectively cleans noisy pseudo-labels. While the ensemble approach introduces some computational overhead, the significant gains in accuracy and noise resilience make it a powerful tool for real-world domain adaptation challenges.

Summary updated

Abstract

Conventional Unsupervised Domain Adaptation (UDA) methods presume source and target domain data to be simultaneously available during training. Such an assumption may not hold in practice, as source data is often inaccessible (e.g., due to privacy reasons). On the contrary, a pre-trained source model is usually available, which performs poorly on target due to the well-known domain shift problem. This translates into a significant amount of misclassifications, which can be interpreted as structured noise affecting the inferred target pseudo-labels. In this work, we cast UDA as a pseudo-label refinery problem in the challenging source-free scenario. We propose Negative Ensemble Learning (NEL) technique, a unified method for adaptive noise filtering and progressive pseudo-label refinement. NEL is devised to tackle noisy pseudo-labels by enhancing diversity in ensemble members with different stochastic (i) input augmentation and (ii) feedback. The latter is achieved by leveraging the novel concept of Disjoint Residual Labels, which allow propagating diverse information to the different members. Eventually, a single model is trained with the refined pseudo-labels, which leads to a robust performance on the target domain. Extensive experiments show that the proposed method achieves state-of-the-art performance on major UDA benchmarks, such as Digit5, PACS, Visda-C, and DomainNet, without using source data samples at all.
BibTeX
@inproceedings{ahmed2022cleaning,
  author = {Waqar Ahmed and Pietro Morerio and Vittorio Murino},
  title = {{Cleaning Noisy Labels by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  doi = {10.1109/wacv51458.2022.00043},
}
Figure