TL;DR
This paper introduces ENCOFA, a novel framework for medical image classification that tackles the challenging problem of mixed open-set and closed-set label noise. By treating open-set noisy samples as a distinct category and augmenting their features, ENCOFA significantly improves classification accuracy compared to existing methods.
What Problem Does This Paper Address?
Medical image classification relies heavily on accurate labels, but real-world datasets often suffer from label noise. This noise can be categorized into two types: closed-set noise, where an image is mislabeled as another known class, and open-set noise, where an image from an unknown class is mislabeled as a known one. While existing methods have addressed closed-set noise, open-set noise in medical imaging remains a significant challenge due to the high inter-class similarity, making it difficult to distinguish between true classes and out-of-distribution samples. Furthermore, many methods either discard open-set noisy samples or treat them uniformly with closed-set noisy samples, failing to leverage their potential for improving model robustness.
Methods
ENCOFA introduces several key innovations to address mixed label noise:
- Noise Type Identifier: A two-step process that first uses a Gaussian Mixture Model (GMM) to separate clean and noisy samples based on their loss values, and then employs a K-Nearest Neighbors (KNN) approach to identify open-set noisy samples within the noisy group by measuring their distance in feature space from known classes.
- Extended Noise-robust Supervised Contrastive (ENSC) Loss: This loss function extends the concept of supervised contrastive learning to include open-set noisy samples as a separate class. It aims to improve the separation between different classes, including distinguishing in-distribution from out-of-distribution samples, by adjusting contrastive pair weights based on label reliability.
- Open-set Feature Augmentation (OSFeatAug) Module: Instead of discarding open-set noisy samples, this module augments their features. It identifies class-specific and class-general channels within the feature representations and substitutes class-general channels with those from other samples to increase feature diversity. These augmented features are then assigned dynamic labels.
- Classification Loss: A combined loss function that includes cross-entropy losses for clean, closed-set noisy (using pseudo-labels), and open-set noisy (using dynamic labels) samples, along with the ENSC loss.
Datasets
The proposed ENCOFA framework was evaluated on two datasets:
- Synthetic Noisy Dataset (NoisyKather5k): This dataset is derived from the Kather-5k dataset, which contains pathological image patches. For this study, eight classes were used, with the first five designated as closed-set and the last three as open-set. Instance-dependent label noise was simulated for closed-set noise, and images were used for open-set noise.
- Real-World Noisy Dataset (Kaggle DR): This dataset consists of fundus images for diabetic retinopathy classification. The label noise in this dataset arises from observer variability (closed-set) and mislabeling of other retinal diseases as DR (open-set).
Key Insights and Ideas
- Open-set noisy samples are valuable: Contrary to common practice, this paper demonstrates that open-set noisy samples, when handled appropriately, can significantly contribute to mitigating label noise and improving model performance.
- Distinguishing noise types is crucial: The high inter-class similarity in medical images necessitates specialized strategies for closed-set and open-set noisy samples, rather than a one-size-fits-all approach.
- Feature augmentation for OOD samples: Augmenting the features of open-set noisy samples can enrich their representations and help the model learn more robust decision boundaries.
- Contrastive learning can be extended: The ENSC loss shows that supervised contrastive learning can be adapted to handle open-set noise by treating OOD samples as a distinct class.
Key Results
- On the NoisyKather5k dataset, ENCOFA achieved state-of-the-art performance, outperforming competing methods by a significant margin. For instance, in the challenging scenario of α=0.4 and β=0.25, ENCOFA achieved an accuracy of 91.73%, compared to the best competitor's 87.47%.
- ENCOFA also demonstrated superior performance on the Kaggle DR dataset, achieving a 2.26% higher accuracy on the binary classification task and 1.19% on the five-class task compared to the best competitor.
- The noise type classification accuracy of ENCOFA was significantly higher than other methods, indicating its effectiveness in correctly identifying open-set noisy samples.
Limitations and Future Work
- Hyperparameter Sensitivity: The authors acknowledge that ENCOFA's hyperparameter selection is sensitive to the noise level in the training data, which can lead to a substantial workload when adapting the framework to new datasets or tasks.
- Clustering's Limited Impact: Applying K-Means clustering to identified open-set noisy samples did not enhance performance, suggesting that the current clustering approach did not significantly improve the detection or utilization of these samples.
Future work will focus on mitigating the hyperparameter sensitivity challenge by refining the adaptability and robustness of ENCOFA, aiming to make it a more dependable solution for managing label noise across diverse medical imaging tasks.
Summary updated