Joint Asymmetric Loss for Learning with Noisy Labels

TL;DR

This paper introduces the Joint Asymmetric Loss (JAL), a novel framework that enhances deep learning models' robustness to noisy labels. By proposing the Asymmetric Mean Square Error (AMSE) as a new passive asymmetric loss function and integrating it into the Active Passive Loss (APL) framework, JAL achieves superior performance in mitigating label noise compared to existing methods.

What Problem Does This Paper Address?

Training deep neural networks (DNNs) on real-world datasets often involves dealing with "noisy labels" – incorrect labels assigned to data samples. This noise can significantly degrade model performance and hinder generalization. While various robust loss functions have been proposed, including symmetric and asymmetric types, existing methods face limitations. Symmetric losses can lead to underfitting, while current asymmetric losses are not easily compatible with advanced frameworks like APL, which combine "active" and "passive" loss components to improve fitting and robustness. This paper aims to bridge this gap by developing a novel asymmetric passive loss function that can be effectively integrated into APL, leading to a more robust and better-fitting learning framework.

Methods

The core of this work lies in two main contributions:

• Asymmetric Mean Square Error (AMSE): The authors propose AMSE as a new passive asymmetric loss function. They rigorously establish the necessary and sufficient conditions for AMSE to satisfy the asymmetric property, which is crucial for noise tolerance. AMSE is an extension of the Mean Squared Error (MSE) loss, parameterized by 'a', which allows it to adapt to different noise levels and types.
• Joint Asymmetric Loss (JAL): JAL is a novel framework that integrates the proposed AMSE into the Active Passive Loss (APL) framework. APL combines an "active" loss (which focuses on maximizing the probability of the correct class) and a "passive" loss (which minimizes probabilities of incorrect classes). By replacing the traditional symmetric passive loss in APL with the new asymmetric AMSE, JAL aims to leverage the benefits of both asymmetric properties and the APL structure for improved noise tolerance and fitting ability. The authors propose two variants: JAL-CE (based on Cross Entropy) and JAL-FL (based on Focal Loss).

Datasets

The proposed methods were evaluated on several standard image classification datasets:

• CIFAR-10 and CIFAR-100: These are widely used benchmark datasets for evaluating image classification models.
• CIFAR-10N and CIFAR-100N: These are versions of CIFAR-10 and CIFAR-100 with human-annotated label noise, providing a more realistic noise scenario.
• WebVision, ILSVRC12, and Clothing1M: These are large-scale, real-world datasets used to assess the method's performance in practical, less controlled environments.

Key Insights and Ideas

• Asymmetric losses are more than just "active": The paper highlights that existing asymmetric loss functions were primarily "active" losses. The key insight is that extending asymmetry to "passive" losses is crucial for improving robustness and fitting simultaneously, especially within frameworks like APL.
• The limitations of simply replacing symmetric losses: Experiments showed that directly replacing a symmetric loss in APL with an existing asymmetric loss (like AGCE) did not yield significant improvements. This underscores the need for a specifically designed passive asymmetric loss.
• AMSE's tunable parameter 'a' is critical: The parameter 'a' in AMSE allows for flexible adaptation to different noise levels. The paper demonstrates that selecting an appropriate 'a' is important, with larger values offering more robustness to symmetric noise but potentially leading to underfitting with asymmetric noise.
• JAL combines the best of both worlds: By integrating AMSE into APL, JAL successfully combines the noise-tolerant properties of asymmetric losses with the enhanced fitting capabilities of the APL framework, leading to state-of-the-art performance.

Key Results

• Superior performance across various noise types: JAL consistently achieved top-2 performance across symmetric, asymmetric, and instance-dependent noise scenarios on CIFAR-10 and CIFAR-100.
• Significant improvements on challenging noise levels: On CIFAR-10 with 0.8 symmetric noise, JAL improved accuracy by approximately 3%. For CIFAR-100 with 0.4 asymmetric noise, the improvement reached up to 10% compared to previous state-of-the-art methods.
• Robustness on real-world datasets: JAL outperformed other methods on large-scale datasets like WebVision, ILSVRC12, and Clothing1M, demonstrating its effectiveness in practical applications.
• Effective mitigation of overfitting: Qualitative analysis (histogram visualizations) showed that JAL models were less prone to overfitting noisy labels compared to standard Cross Entropy, maintaining better performance throughout training.

Limitations and Future Work

• Parameter sensitivity: While JAL demonstrates robustness, the performance of AMSE can be sensitive to the choice of the hyperparameter 'a', especially under asymmetric noise. The paper suggests selecting a moderate 'a' to balance robustness and fitting.
• Theoretical analysis of 'a' for different noise types: While the paper provides a theoretical condition for AMSE to be asymmetric, further exploration into the optimal selection of 'a' for various noise distributions could be beneficial.
• Extending to other tasks: The authors hope that AMSE and JAL will be useful for other tasks beyond image classification that involve noisy labels. Future work could explore their application in areas like natural language processing or object detection.