TL;DR
This paper introduces a new way to simulate realistic "noisy labels" in medical images, called Typicality- and Instance-Dependent Label Noise (TIDN), which better reflects how human errors occur in practice. They then propose a novel framework, TIDN-attention, that effectively combats this simulated and real-world label noise, leading to significantly improved polyp classification accuracy.
What Problem Does This Paper Address?
Training deep learning models requires large amounts of labeled data. However, in specialized fields like medical imaging, obtaining accurate labels is challenging and time-consuming, often leading to incorrect annotations (noisy labels). While existing methods handle "instance-independent" label noise (where errors are random), they struggle with "instance-dependent" label noise (IDN), where the likelihood of an error depends on the specific image content. This paper argues that current IDN approaches don't account for "typicality" – the idea that more unusual or atypical samples are more prone to mislabeling. This gap limits their effectiveness in real-world scenarios, especially in medical image classification where subtle features can be easily misidentified.
Methods
The core of this work lies in two main contributions:
- Simulating Typicality- and Instance-Dependent Label Noise (TIDN): The authors propose a novel way to generate noisy labels that mimics real-world scenarios. They use the distance of a sample from the decision boundary of a Support Vector Machine (SVM) classifier to represent its "typicality." Samples closer to the boundary are considered less typical and thus more likely to be mislabeled. This distance is then used to determine the probability of flipping a label, creating TIDN.
- TIDN-combating Framework: To address the simulated TIDN, they introduce a framework that includes a "TIDN-attention module." This module learns a per-sample noisy transition matrix, effectively mapping features to the probability of a label being incorrect. They also propose a recursive algorithm, inspired by Expectation-Maximization (EM), to iteratively learn this transition matrix and improve the model's predictions. This framework aims to learn the underlying true labels rather than just fitting the noisy ones.
Datasets
The paper uses two datasets for colonoscopy image polyp classification:
- Kvasir V2: A public dataset containing 8000 images across eight categories.
- A colonoscopy video classification dataset: This dataset, derived from videos, includes labels from both expert doctors and beginners, providing a source of real-world noisy labels.
The datasets were split into training, validation, and testing sets, with the training set containing noisy labels and the validation/test sets containing accurate ground truth labels.
Key Insights and Ideas
- Typicality is Crucial for Realistic Noise Simulation: The paper highlights that simply considering instance-specific features isn't enough; the "typicality" of a sample (how close it is to decision boundaries) is a key factor in how likely it is to be mislabeled in real-world scenarios.
- TIDN Better Mimics Real-World Noise: The proposed TIDN simulation method shows a lower "mean total distance" to real-world noisy labels compared to existing IIN and IDN simulation techniques, indicating it's a more faithful representation.
- Attention for Noise Modeling: The TIDN-attention module effectively learns to model the per-sample noise transition, allowing the model to adapt to the specific noise characteristics of each instance.
- Recursive Algorithm for Robust Learning: The recursive algorithm helps the model iteratively refine its understanding of the noisy transition matrix and the true labels, leading to more robust predictions.
Key Results
- Superior Noise Simulation: The TIDN simulation method demonstrated the lowest mean total distance to real-world noisy labels compared to IIN and IDN methods.
- Improved Classification Accuracy: The TIDN-combating framework achieved significantly higher test accuracy (ACC) on datasets with both simulated TIDN and real-world noisy labels. For instance, on the Kvasir V2 dataset with a 15% noise ratio, their method achieved 92.44% ACC, outperforming baselines and other state-of-the-art methods.
- Effective on Real-World Noise: The proposed method also showed strong performance when trained on real-world noisy labels from human experts and beginners, achieving 86.67% ACC, matching the upper bound performance.
Limitations and Future Work
- Initialization Dependency: The paper acknowledges that the performance of the TIDN-combating framework can be influenced by the initialization of the noise transition matrix. Future work could focus on developing better initialization strategies or methods to theoretically address the "freedom problems" associated with instance-dependent noisy transition matrices.
- Broader Applicability: The authors suggest that their method could be applied to more advanced classification techniques, such as diffusion models, to further mitigate the impact of incorrect labels in various domains.
Summary updated