Confidence-based Reliable Learning under Dual Noises

Peng Cui, Yang Yue, Zhijie Deng, Jun Zhu

Abstract

Deep neural networks (DNNs) have achieved remarkable success in a variety of computer vision tasks, where massive labeled images are routinely required for model optimization. Yet, the data collected from the open world are unavoidably polluted by noise, which may significantly undermine the efficacy of the learned models. Various attempts have been made to reliably train DNNs under data noise, but they separately account for either the noise existing in the labels or that existing in the images. A naive combination of the two lines of works would suffer from the limitations in both sides, and miss the opportunities to handle the two kinds of noise in parallel. This work provides a first, unified framework for reliable learning under the joint (image, label)-noise. Technically, we develop a confidence-based sample filter to progressively filter out noisy data without the need of pre-specifying noise ratio. Then, we penalize the model uncertainty of the detected noisy data instead of letting the model continue over-fitting the misleading information in them. Experimental results on various challenging synthetic and real-world noisy datasets verify that the proposed method can outperform competing baselines in the aspect of classification performance.

Introduction

In an ideal research environment, deep learning datasets are perfectly curated. However, real-world data collection is inherently messy. Most existing research on "noisy data" focuses on one of two isolated problems: label noise ($y$-noise), where a clear image is incorrectly labeled, or image noise ($x$-noise), where the image itself is corrupted by blur, low resolution, or background clutter, making identification difficult even if the label is technically correct.

This paper argues that real-world data often suffers from dual noises—both problems occurring simultaneously. Existing solutions for label noise often fail when the image quality is poor, and image denoising techniques are often too computationally expensive or specific to certain types of artifacts. This work proposes a unified approach to handle both noise types simultaneously without requiring prior knowledge of the noise ratio.

Methodology

The authors leverage Uncertainty-based Deep Models, specifically Deep Ensembles, to distinguish between clean and noisy data. The framework operates in two primary stages: filtering and optimization.

1. The Confidence-based Sample Filter

Instead of complex statistical modeling, the framework utilizes two metrics derived from the ensemble's predictions to categorize data:

  • Label Confidence (L-Con): The average probability the model assigns to the given label. A low L-Con suggests the label is likely incorrect ($y$-noise).
  • Maximum Confidence (M-Con): The highest probability the model assigns to any class. If the model lacks confidence across all classes, the image itself is likely corrupted or ambiguous ($x$-noise).

By applying thresholds ($\epsilon_1, \epsilon_2$), the system categorizes every sample as either "clean" or "noisy."

The proposed pipeline: Input data is processed through a Deep Ensemble to calculate confidence levels, which then guide a filter to apply different optimization objectives.

2. Uncertainty Penalty on Noisy Data

Standard training forces a model to fit noisy data, leading to over-fitting on misleading information. To counter this, the authors propose a dual-loss objective:

  • For Clean Data: Minimize standard Cross-Entropy loss ($\ell_{CE}$).
  • For Noisy Data: Minimize Model Uncertainty (measured as Mutual Information, $\ell_{uta}$).

By penalizing uncertainty on noisy samples, the model is encouraged to extract reliable features from the image without being misled by incorrect labels or heavy corruptions. This approach treats M-Con as a proxy for data quality; when an image is blurry, the model's output distribution becomes "flat," naturally lowering the M-Con and triggering the uncertainty penalty.

Experimental Results

The method was evaluated on both synthetic and real-world benchmarks:

  • CIFAR-100 & TinyImageNet: Used for controlled tests where label flipping ($y$-noise) and image corruptions like Gaussian blur ($x$-noise) were manually injected.
  • WebVision: A massive real-world dataset (2.4 million images) containing natural mislabeling and poor-quality web images.

As shown in the performance table below, the proposed method (Proposed-LM) consistently outperforms standard Deep Ensembles (DE-CE) and specialized label-noise methods like DYR, particularly as the noise rate increases.

Experimental results on CIFAR-100 and TinyImageNet showing the proposed method's superiority across various noise rates compared to existing baselines.

Key Findings

  • Superiority under Dual Noise: On CIFAR-100 with 40% label noise and 30% image noise, Proposed-LM achieved 66.61% accuracy, significantly higher than the 54.90% of standard Deep Ensembles.
  • Real-World Efficacy: On the WebVision benchmark, the method achieved a Top-1 accuracy of 76.68%, outperforming the best baseline by nearly 3%.
  • Information Retention: Unlike filters that simply discard noisy samples, this method demonstrates that even noisy images contain valuable information. Penalizing uncertainty allows the model to learn robust features without over-fitting the noise.

Conclusion

This work provides a robust, unified framework for handling joint (image, label)-noise. By moving away from simple data exclusion and toward uncertainty-based optimization, the proposed method achieves state-of-the-art results on challenging datasets. While the computational cost of training ensembles remains a consideration for future work, the simplicity and effectiveness of the confidence-based filter make it a practical solution for real-world "messy" data.

Summary updated

Abstract

Deep neural networks (DNNs) have achieved remarkable success in a variety of computer vision tasks, where massive labeled images are routinely required for model optimization. Yet, the data collected from the open world are unavoidably polluted by noise, which may significantly undermine the efficacy of the learned models. Various attempts have been made to reliably train DNNs under data noise, but they separately account for either the noise existing in the labels or that existing in the images. A naive combination of the two lines of works would suffer from the limitations in both sides, and miss the opportunities to handle the two kinds of noise in parallel. This work provides a first, unified framework for reliable learning under the joint (image, label)-noise. Technically, we develop a confidence-based sample filter to progressively filter out noisy data without the need of pre-specifying noise ratio. Then, we penalize the model uncertainty of the detected noisy data instead of letting the model continue over-fitting the misleading information in them. Experimental results on various challenging synthetic and real-world noisy datasets verify that the proposed method can outperform competing baselines in the aspect of classification performance.
BibTeX
@inproceedings{cui2022confidence,
  author = {Peng Cui and Yang Yue and Zhijie Deng and Jun Zhu},
  title = {{Confidence-based Reliable Learning under Dual Noises}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=7fGIR2oIHTl},
}
Figure