TL;DR
This paper introduces "Self-Relaxed Joint Training," a novel framework for training deep learning models to estimate severity levels from medical images when the provided labels are noisy and have an ordinal relationship (e.g., mild, moderate, severe). By leveraging "self-relaxed" soft labels derived from noisy hard labels, the method significantly improves sample selection and model training, outperforming existing techniques on multiple medical imaging datasets.
What Problem Does This Paper Address?
Accurately assessing the severity of medical conditions from images is crucial for diagnosis and treatment. However, manually assigning these severity labels is expensive and time-consuming, often leading to noisy labels due to the inherent ambiguity in visual assessment. This is particularly challenging for ordinal labels, where the "correct" label might fall between discrete categories. Traditional methods for handling noisy labels often struggle with this ordinal nature, and their sample selection mechanisms can be imperfect, leading to the propagation of incorrect labels. The paper aims to address this by developing a robust training framework that can effectively handle ordinal noisy labels in medical image severity estimation.
Methods
The core of the proposed method is "Self-Relaxed Joint Training," which builds upon the established "joint training" paradigm for noisy labels. This paradigm typically involves two key components:
- Sample Selection: Identifying and discarding samples likely to have incorrect labels.
- Dual-Network Architecture: Training two networks collaboratively, where they learn from each other's predictions.
The novelty of this paper lies in how it adapts these components for ordinal noisy labels:
- Self-Relaxed Soft Labels: Instead of solely relying on the noisy hard labels (e.g., one-hot encoded labels), the framework derives "soft labels." These soft labels represent a probability distribution over the ordinal classes, estimated based on the assumption that noisy labels are likely to be close to the true label (e.g., a "mild" label is more likely to be mistaken for "moderate" than "severe"). This is mathematically formulated as $y_i \sim [\mathcal{N}(\tilde{y}_i, \sigma^2)]$, where $\tilde{y}_i$ is the noisy label and $y_i$ is the true label.
- Hybrid Loss Usage: The framework strategically uses different loss functions for different parts of the training process.
- Hard Labels ($L_h$) for Sample Selection: The traditional method of using the loss calculated from the noisy hard labels is retained for selecting potentially clean samples. This is because hard labels, despite being noisy, can still provide a strong signal for identifying samples with very high losses, indicating likely mislabeling.
- Soft Labels ($L_s$) for Network Updates: The derived soft labels are used to update the network parameters. This is the crucial step that mitigates the negative impact of noisy labels. By using a relaxed distribution, the model is less penalized for making small errors between adjacent ordinal classes, leading to more robust learning.
- Versatility: The proposed framework is designed to be plug-and-play, meaning it can be integrated with existing joint-training methods like Co-teaching, CoDis, and JoCor, enhancing their performance.
Datasets
The paper evaluates its method on three datasets:
- LIMUC Dataset: A publicly available dataset of endoscopic ulcerative colitis (UC) images.
- Private UC Dataset: A dataset collected from Kyoto Second Red Cross Hospital, also containing UC endoscopic images.
- Diabetic Retinopathy (DR) Dataset: A real-world noisy dataset of retinal images used to assess performance on naturally occurring label noise.
For the UC datasets, the authors created noisy versions by applying two types of simulated noise: Quasi-Gaussian and Truncated-Gaussian. The DR dataset was used as a real-world noisy dataset.
Key Insights and Ideas
- Ordinality Matters: The paper strongly emphasizes that simply treating ordinal labels as independent classes misses valuable information. Leveraging the ordered nature of severity levels is key to improving robustness against noise.
- Soft Labels as a "Relaxation" Mechanism: The "self-relaxed" soft labels act as a buffer, preventing the model from being overly penalized by minor misclassifications between adjacent ordinal classes, which are common in noisy ordinal data.
- Strategic Loss Usage is Crucial: The combination of using hard labels for sample selection (to effectively prune highly erroneous samples) and soft labels for model updates (to learn robustly from less erroneous samples) is more effective than using only soft labels for both tasks.
- Improved Sample Selection: The proposed method leads to higher "label precision" (the proportion of correctly labeled samples among those selected as clean) compared to traditional methods, especially as training progresses. This is attributed to the more robust learning signal provided by soft labels.
Key Results
- The proposed "Self-Relaxed Joint Training" framework, when applied to existing methods like Co-teaching, CoDis, and JoCor, consistently outperformed their original versions and other state-of-the-art methods across all metrics (accuracy, MAE, macro F1) on both UC datasets and the DR dataset.
- For example, on the LIMUC dataset with a noise rate of $\epsilon=0.4$, "Co-teaching + Ours" achieved an accuracy of 0.689±0.012, outperforming the original Co-teaching's 0.647±0.009.
- The method demonstrated a more stable test accuracy curve, avoiding the "memorization effect" (where test accuracy degrades after initial gains due to overfitting noisy samples) observed in some traditional methods.
- Label precision curves showed that "Co-teaching + Ours" maintained higher and more stable label precision throughout training compared to standard Co-teaching.
Limitations and Future Work
- Class Imbalance: The authors acknowledge that medical image datasets often suffer from class imbalance, which can exacerbate the problem of learning with noisy labels. They suggest that future work should focus on integrating techniques to mitigate class imbalance.
- Computational Cost: While not explicitly detailed as a limitation, the use of dual networks inherently increases computational requirements compared to single-network approaches.
The paper's future work direction is clearly stated as addressing class imbalance issues in ordinal noisy label learning.
Summary updated