Joint Dermatological Lesion Classification and Confidence Modeling with Uncertainty Estimation

Gun-Hee Lee, Han-Bin Ko, Seong-Whan Lee

TL;DR

This paper introduces a deep learning framework that treats skin lesion features as probabilistic distributions rather than fixed points. By estimating the "uncertainty" of individual features, the model can ignore noisy data (like hair or artifacts) and focus on confident indicators, significantly improving diagnostic accuracy and allowing the system to flag ambiguous cases for human review.

What Problem Does This Paper Address?

Deep learning has become incredibly effective at identifying skin cancer from dermoscopic images, sometimes rivaling expert dermatologists. However, standard models suffer from two major flaws in a clinical setting:

  1. Overconfidence: They often produce high-probability predictions even when the input is noisy, blurry, or contains "distractors" like hair, skin folds, or medical markers.
  2. Lack of Nuance: Standard models represent an image as a single "point" in a high-dimensional space. If that point is shifted by noise, the classification fails.

In medicine, a doctor doesn't just give a diagnosis; they provide a level of confidence. If a specific feature of a mole is blurry, they ignore it and look at the clearer parts. This paper aims to give AI models that same ability to "know what they don't know."

Methods

The authors propose a two-stage framework that moves away from deterministic "point" vectors to Probabilistic Latent Representations.

  • The Architecture: They use a standard "backbone" network (like EfficientNet or ResNet) to extract features (the mean, $\mu$). They then add a secondary Uncertainty Estimation Network that predicts the variance ($\sigma^2$) for every feature.
  • Confidence Pooling: This is the core innovation. Instead of treating all extracted features equally, the model calculates a confidence value ($1/\sigma^2$) for each. It then performs "Confidence Pooling," which essentially weights the features so the classifier "listens" to the confident ones and ignores the uncertain ones.
  • Two-Stage Training:
    1. Train the backbone to classify lesions using standard cross-entropy.
  • Freeze the backbone and train the uncertainty network using a specialized log-likelihood loss that encourages images of the same class to have overlapping Gaussian distributions in the latent space.
  • Confidence Modeling: Finally, the model doesn't just output a class; it outputs a distribution. This allows the system to calculate an overall confidence score for the entire prediction.

Datasets

The model was evaluated on two industry-standard benchmarks:

  • ISIC 2018 (HAM10000): 10,015 images across 7 categories (e.g., Melanoma, Basal cell carcinoma).
  • ISIC 2019: A larger, more diverse set of 25,331 images across 8 categories. These datasets are notoriously difficult because they are "imbalanced"—some common benign conditions have thousands of images, while rare cancers have very few.

Key Insights and Ideas

  • Uncertainty as a Filter: The most striking insight is that modeling uncertainty isn't just a safety feature; it’s a performance booster. By penalizing uncertain features during the decision-making process, the model becomes more robust to the "noise" common in medical imaging.
  • Different from Attention: While it sounds like an "Attention Mechanism," the authors clarify the distinction: standard attention looks for important parts of an image, while confidence pooling looks for reliable features.
  • The "Reject" Option: The framework allows for a "rejection" strategy. In a real-world clinic, a model could automatically pass high-uncertainty images to a human expert rather than guessing, which drastically increases the overall reliability of the diagnostic pipeline.

Key Results

The authors tested their method across various backbones (ResNet, DenseNet, EfficientNet). In every case, adding Confidence Pooling (CP) improved performance:

  • Accuracy Boost: On ISIC 2018, using EfficientNet-B4 with CP improved Balanced Accuracy (BACC) from 74.2% to 78.7%.
  • Robustness: Qualitative results showed that the model correctly identified images with hair or poor lighting as "high uncertainty," yet was still able to classify them correctly by focusing on the remaining clear features.
  • Performance via Rejection: The most impressive results appeared when the model was allowed to "reject" its most uncertain predictions. By discarding the bottom 20% of low-confidence samples on the ISIC 2019 dataset, the Balanced Accuracy jumped from 75.5% to 86.7%.

Limitations and Future Work

  • Epistemic vs. Aleatoric: The paper focuses primarily on epistemic uncertainty (uncertainty in the model's parameters/knowledge). It does not deeply explore aleatoric uncertainty (noise inherent in the data itself), though the two are often linked in medical imaging.
  • Two-Stage Training: The current approach requires freezing the backbone. A fully end-to-end joint training session might yield even better feature representations but could be more unstable to train.
  • Clinical Integration: While the "rejection" metric is mathematically sound, the paper does not define the specific threshold a hospital would need to use to balance safety with the workload of human dermatologists.

Summary updated

Abstract

Deep learning has played a major role in the interpretation of dermoscopic images for detecting skin defects and abnormalities. However, current deep learning solutions for dermatological lesion analysis are typically limited in providing probabilistic predictions which highlights the importance of concerning uncertainties. This concept of uncertainty can provide a confidence level for each feature which prevents overconfident predictions with poor generalization on unseen data. In this paper, we propose an overall framework that jointly considers dermatological classification and uncertainty estimation together. The estimated confidence of each feature to avoid uncertain feature and undesirable shift, which are caused by environmental difference of input image, in the latent space is pooled from confidence network. Our qualitative results show that modeling uncertainties not only helps to quantify model confidence for each prediction but also helps classification layers to focus on confident features, therefore, improving the accuracy for dermatological lesion classification. We demonstrate the potential of the proposed approach in two state-of-the-art dermoscopic datasets (ISIC 2018 and ISIC 2019).
BibTeX
@inproceedings{lee2021joint,
  author = {Gun-Hee Lee and Han-Bin Ko and Seong-Whan Lee},
  title = {{Joint Dermatological Lesion Classification and Confidence Modeling with Uncertainty Estimation}},
  booktitle = {ACPR},
  year = {2021},
  url = {https://openreview.net/forum?id=Nz7ORH0y5ve},
}

Loading related content...

Figure