Interpreting Fine-Grained Dermatological Classification by Deep Learning

Sourav Mishra, Hideaki Imaizumi, Toshihiko Yamasaki

Introduction

Dermatological expertise is a scarce resource globally, with approximately 3.6 dermatologists per 10,000 people in the United States. While general practitioners often manage skin conditions, their diagnostic accuracy is estimated at only 57%. Deep learning offers a potential solution, yet "fine-grained" classification—distinguishing between many visually similar skin diseases—remains a significant hurdle. This research presents a deep learning pipeline designed to classify ten common East Asian skin conditions, achieving nearly 90% accuracy while providing insights into model decision-making through interpretability tools.

Methodology

The researchers employed a transfer learning approach using the ResNet architecture (variants 34, 50, 101, and 152) pretrained on ImageNet. To optimize performance, several advanced training strategies were implemented:

  • Learning Rate (LR) Range Test: This technique identifies the optimal starting learning rate by observing loss behavior over a range of values.
  • Stochastic Gradient Descent with Restarts (SGD-R): A cosine annealing schedule was used to periodically reset the learning rate, helping the model escape local minima during optimization.
  • Differential Learning Rates: Lower learning rates were assigned to early layers (capturing general features) and higher rates to final layers (capturing task-specific features).
  • Interpretability Tools: Grad-CAM (Class Activation Maps) and Guided Backpropagation (GBP) were used to visualize the specific pixels driving the model's predictions.
Figure 3: Learning rate range test showing validation loss vs. learning rate on a log scale to identify the optimal training starting point.

Dataset and Training

The study utilized a custom-compiled dermoscopic dataset of 7,264 images focused on East Asian skin types. The images were categorized into ten conditions: Acne, Alopecia, Blister, Crust, Erythema, Leukoderma, Pigmented Macula, Tumor, Ulcer, and Wheal. Data was sourced from clinical specimens and volunteer submissions, split into a 5:1 ratio for training and validation.

Key Results and Interpretability

The ResNet-152 model achieved a peak Top-1 accuracy of 89.8% on the balanced validation set. Interestingly, increasing model depth from ResNet-34 to ResNet-152 only yielded a ~1% gain, suggesting that data quality and training regimen are more critical than raw parameter count for this task.

Figure 6: Confusion matrix for the 10 dermatological classes, illustrating high diagonal accuracy and specific confusion pairs like Ulcer vs. Tumor.

Error analysis revealed that model mistakes often mirror biological realities. For instance, "Ulcer" and "Crust" were frequently confused because a crust often forms during the healing phase of an ulcer. The most common confusion pair was Ulcer vs. Tumor (29 total errors). Grad-CAM visualizations showed that the model sometimes incorrectly fixated on surrounding inflammation to classify a lesion as a "Tumor" when it was actually a benign "Ulcer."

Practical Insights for Clinical AI

The study identified several factors that significantly impact model performance:

  • Centering and Field of View (FOV): Misclassifications often occurred when lesions were not centered. Reducing the FOV and centering the object improved prediction clarity.
  • Dataset Balancing: The authors caution that high accuracy on unbalanced datasets can be misleading, as models may simply learn to guess the most frequent class.
  • Environmental Sensitivity: Models remain sensitive to real-world noise like image blur, poor lighting, and specular reflections from camera flashes.

Conclusion and Future Work

While the model shows high performance, it is currently limited by its focus on East Asian skin types and its sensitivity to environmental factors. The researchers emphasize that AI should complement rather than replace clinical context, as a true diagnosis often requires patient history that a single image cannot provide. Future developments may include mobile applications that assist users in capturing optimal FOV and use techniques like Gamma Correction to normalize lighting conditions.

Summary updated

Abstract

This paper analyzes a deep learning based classification process for common East Asian dermatological conditions. We have chosen ten common categories based on prevalence. With more than 85% accuracy in our experiments, we have tried to investigate why current models are yet to reach accuracy benchmarks seen in object identification tasks. Our current attempt sheds light on how deep learning based dermoscopic identification and dataset creation could be improved.
BibTeX
@inproceedings{mishra2019interpreting,
  author = {Sourav Mishra and Hideaki Imaizumi and Toshihiko Yamasaki},
  title = {{Interpreting Fine-Grained Dermatological Classification by Deep Learning}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  doi = {10.1109/cvprw.2019.00331},
}
Figure