PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive Representation Learning

Xiaoyu Liu, Beitong Zhou, Cheng Cheng

Abstract

Recently, the usage of Contrastive Representation Learning (CRL) as a pre-training technique improves the performance of learning with noisy labels (LNL) methods. However, instead of pre-training, when trivially combining CRL loss with LNL methods as an end-to-end framework, the empirical experiments show severe degeneration of the performance. We verify through experiments that this issue is caused by optimization conflicts of losses and propose an end-to-end PLReMix framework by introducing a Pseudo-Label Relaxed (PLR) contrastive loss. This PLR loss constructs a reliable negative set of each sample by filtering out its inappropriate negative pairs, alleviating the loss conflicts by trivially combining these losses. The proposed PLR loss is pluggable and we have integrated it into other LNL methods, observing their improved performance. Furthermore, a two-dimensional Gaussian Mixture Model is adopted to distinguish clean and noisy samples by leveraging semantic information and model outputs simultaneously. Experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method. Code is available at https://github.com/lxysl/PLReMix.

Introduction

In real-world machine learning, datasets are rarely perfectly labeled. Labels often come from web-scraping or automated tagging, leading to "noisy labels"—for instance, a picture of a cat erroneously labeled as a dog. To handle this, researchers often employ Contrastive Representation Learning (CRL), a technique that teaches models to group similar images together in a feature space without relying on labels.

While CRL is highly effective as a standalone pre-training step, a significant challenge arises when attempting to train CRL and supervised learning objectives simultaneously in an end-to-end fashion. Performance often degrades significantly. The authors of this paper identify this phenomenon as an optimization conflict. Standard CRL (such as SimCLR) treats every other sample in a batch as a "negative," pushing them away from the target sample. If the target and another sample actually belong to the same class, CRL still pushes them apart, creating a "gradient tug-of-war" with the supervised loss that tries to pull them together.

The PLReMix Framework

To resolve these conflicts, the authors propose PLReMix, an end-to-end framework featuring two primary innovations: the Pseudo-Label Relaxed (PLR) Contrastive Loss and a Joint Sample Selection mechanism using a 2D Gaussian Mixture Model (GMM).

The PLReMix architecture (i-iii) and its end-to-end training pipeline (d) compared to traditional multi-stage approaches.

Pseudo-Label Relaxed (PLR) Contrastive Loss

Instead of blindly pushing all other samples away, the PLR loss uses the model's current predictions (pseudo-labels) to "relax" the contrastive constraint. If two samples have overlapping "top-$\kappa$" predicted classes, the model assumes they might belong to the same category and removes them from the negative set. This eliminates the gradient interference between the contrastive and supervised loss functions.

Joint Sample Selection (2D GMM)

Most existing methods separate "clean" from "noisy" data using a 1D GMM based solely on classification loss. PLReMix upgrades this to a 2D GMM that considers both classification loss and a "prototypical" loss—a measure of how well a sample matches the average features of its assigned class. This approach is significantly more effective at identifying samples that might have low classification loss but are semantically inconsistent with their class.

Experimental Results and Discussion

The effectiveness of PLReMix is most evident in extreme noise scenarios. On the CIFAR-10 benchmark with a 90% noise ratio, PLReMix achieved 92.43% accuracy, vastly outperforming the popular DivideMix algorithm (75.4%).

Test accuracy comparison (a) shows the performance gain of PLR over standard CRL, while (b) illustrates the filtering of inappropriate negative pairs.

As shown in Figure 1, a "trivial" combination of SimCLR and DivideMix (red line) leads to a substantial drop in test accuracy compared to the baseline DivideMix (blue line). By introducing the PLR loss (green line), the framework not only recovers but exceeds the baseline performance. This is achieved by identifying "inappropriate negative pairs"—samples $x_i$ and $x_j$ that share similar prediction probabilities and thus should not be pushed apart in the feature space.

Furthermore, the PLR loss is "pluggable." Integrating it into other algorithms like JoCoR improved performance on CIFAR-10 (50% noise) from 80.94% to 83.46%, demonstrating its broad utility. On real-world datasets like Clothing1M, PLReMix reached 74.85% accuracy, setting a new state-of-the-art for end-to-end methods.

Conclusion

PLReMix successfully bridges the gap between contrastive representation learning and supervised learning with noisy labels. By addressing optimization conflicts through pseudo-label relaxation and improving sample selection with a 2D GMM, it achieves robust performance across synthetic and real-world benchmarks. Future work will explore refining class prototype initialization to better handle datasets with a high number of classes and complex noise patterns.

Summary updated

Abstract

Recently, the usage of Contrastive Representation Learning (CRL) as a pre-training technique improves the performance of learning with noisy labels (LNL) methods. However, instead of pre-training, when trivially combining CRL loss with LNL methods as an end-to-end framework, the empirical experiments show severe degeneration of the performance. We verify through experiments that this issue is caused by optimization conflicts of losses and propose an end-to-end \textbf{PLReMix} framework by introducing a Pseudo-Label Relaxed (PLR) contrastive loss. This PLR loss constructs a reliable negative set of each sample by filtering out its inappropriate negative pairs, alleviating the loss conflicts by trivially combining these losses. The proposed PLR loss is pluggable and we have integrated it into other LNL methods, observing their improved performance. Furthermore, a two-dimensional Gaussian Mixture Model is adopted to distinguish clean and noisy samples by leveraging semantic information and model outputs simultaneously. Experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method. Code is available at \url{https://github.com/lxysl/PLReMix}.
BibTeX
@inproceedings{liu2025plremix,
  author = {Xiaoyu Liu and Beitong Zhou and Cheng Cheng},
  title = {{PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive Representation Learning}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  url = {https://openreview.net/forum?id=SQjnHNUy85},
}
Figure