Deep Learning for Diagnostic Binary Classification of Multiple-Lesion Skin Diseases

Kenneth Thomsen, Anja Liljedahl Christensen, Lars Iversen, Hans B. Lomholt, Ole WintherFrontiers in Medicine 2020 · DOI

TL;DR

This paper explores the use of deep learning, specifically a VGG-16 convolutional neural network (CNN), for the binary classification of common and rare multiple-lesion skin diseases. The developed model shows promising diagnostic accuracy, comparable to or exceeding that of general practitioners with dermatological training, suggesting its potential as a valuable tool in clinical settings.

What Problem Does This Paper Address?

Diagnosing skin diseases is a significant challenge, especially for general practitioners (GPs), due to the vast number of possible conditions and the often subtle differences between them. This difficulty is compounded by the fact that many skin diseases present with multiple lesions, making differential diagnosis complex. The paper highlights that GPs' diagnostic accuracy in dermatology can range from 48-77%, and the shortage of dermatologists in some areas further emphasizes the need for accessible, accurate diagnostic aids. While computer-aided diagnostic (CAD) models have shown success with single-lesion skin diseases (like melanoma), there's a gap in robust CAD tools for multiple-lesion skin diseases, particularly those encountered in real-world clinical settings with non-standardized imagery.

Methods

The researchers employed a multi-task learning approach using a VGG-16 convolutional neural network (CNN) for binary classification. They experimented with four variations of the VGG-16 model:

  • VGG-16P: VGG-16 with pre-trained layers (from ImageNet) frozen, and a new fully connected/sigmoid output layer.
  • VGG-16N: VGG-16 trained from scratch (no pre-trained layers, all layers trainable).
  • VGG-16PS: VGG-16P with an added Spatial Transformer Network (STN) for improved region of interest selection.
  • VGG-16NS: VGG-16N with an added STN.

The models were trained on a dataset of 16,543 non-standardized skin images. Data augmentation techniques were applied to the training set to increase sample diversity. Images were pre-processed using K-means clustering to segment skin from non-skin areas. The primary outcome measures were sensitivity and specificity, with AUC and accuracy serving as secondary metrics.

Datasets

The study utilized a dataset of 19,641 images from the local skin image database of the Department of Dermatology at Aarhus University Hospital, Denmark. After data cleansing (removing non-skin images), 16,543 skin images remained. These images were collected from 2,342 patients and primarily represented individuals with Fitzpatrick skin types II and III. The dataset included images categorized under ICD-10 codes related to acne, rosacea, psoriasis, eczema, and cutaneous T-cell lymphoma (CTCL). The data was split into an 80% training set, 10% validation set, and 10% test set.

Key Insights and Ideas

  • Real-world data is crucial: The study emphasizes the importance of using non-standardized images from clinical settings, mimicking how GPs would encounter these conditions, rather than relying solely on highly curated dermatoscopic images.
  • Pre-trained models offer an advantage: The VGG-16 model with pre-trained ImageNet weights (VGG-16P) consistently outperformed models trained from scratch (VGG-16N), highlighting the benefit of leveraging knowledge from large-scale image datasets.
  • STN's limited impact: The addition of a Spatial Transformer Network (STN) did not significantly improve performance compared to models without it, suggesting that for this specific task and dataset, the VGG-16 architecture was sufficient for localization.
  • Binary classification is effective: The multi-task learning approach, focusing on binary distinctions between specific disease pairs, yielded clinically relevant accuracy.
  • Potential for rare diseases: The model's performance in distinguishing CTCL from eczema suggests that CAD tools could be valuable for identifying rare but serious conditions.

Key Results

The best-performing model, VGG-16P, achieved the following results on the test set:

  • Acne vs. Rosacea: Sensitivity of 85.42% and specificity of 89.53%.
  • CTCL vs. Eczema: Sensitivity of 74.29% and specificity of 84.09%.
  • Psoriasis vs. Eczema: Sensitivity of 81.79% and specificity of 73.57%.

Overall, the VGG-16P model demonstrated accuracy above 77% across all three binary tasks, which is considered clinically relevant and comparable to or better than the reported diagnostic accuracy of GPs (48-77%).

Limitations and Future Work

  • Overfitting concerns: The authors acknowledge potential overfitting due to small datasets for acne, rosacea, and CTCL, although balancing the data only slightly improved performance. Selection bias is also a concern given the retrospective nature of the study.
  • Racial bias: The dataset primarily consisted of Fitzpatrick skin types II-III, limiting generalizability to other skin types.
  • Healthy skin images: The presence of healthy skin images (estimated at 3.7%) may have underestimated the true performance of the models.
  • Grouping of ICD-10 codes: While beneficial for data volume, grouping ICD-10 codes into broader categories might obscure finer distinctions between subtypes.

Future work suggested includes:

  • Head-to-head testing: Conducting prospective clinical trials to compare the CAD model's performance against trained physicians.
  • Multinomial classification: Extending the model to classify multiple diseases simultaneously, rather than just binary pairs.
  • External validation: Testing the model on an independent, external dataset to assess its generalizability and mitigate potential biases.
  • Making models accessible: The authors have made their classification algorithm available on GitHub to facilitate further research and validation.

Summary updated

Abstract

<b>Background:</b> Diagnosis of skin diseases is often challenging and computer-aided diagnostic tools are urgently needed to underpin decision making. <b>Objective:</b> To develop a convolutional neural network model to classify clinically relevant selected multiple-lesion skin diseases, this in accordance to the STARD guidelines. <b>Methods:</b> This was an image-based retrospective study using multi-task learning for binary classification. A VGG-16 model was trained on 16,543 non-standardized images. Image data was distributed in training set (80%), validation set (10%), and test set (10%). All images were collected from a clinical database of a Danish population attending one dermatological department. Included was patients categorized with ICD-10 codes related to acne, rosacea, psoriasis, eczema, and cutaneous t-cell lymphoma. <b>Results:</b> Acne was distinguished from rosacea with a sensitivity of 85.42% CI 72.24-93.93% and a specificity of 89.53% CI 83.97-93.68%, cutaneous t-cell lymphoma was distinguished from eczema with a sensitivity of 74.29% CI 67.82-80.05% and a specificity of 84.09% CI 80.83-86.99%, and psoriasis from eczema with a sensitivity of 81.79% CI 78.51-84.76% and a specificity of 73.57% CI 69.76-77.13%. All results were based on the test set. <b>Conclusion:</b> The performance rates reported were equal or superior to those reported for general practitioners with dermatological training, indicating that computer-aided diagnostic models based on convolutional neural network may potentially be employed for diagnosing multiple-lesion skin diseases.
BibTeX
@inproceedings{thomsen2020deep,
  author = {Kenneth Thomsen and Anja Liljedahl Christensen and Lars Iversen and Hans B. Lomholt and Ole Winther},
  title = {{Deep Learning for Diagnostic Binary Classification of Multiple-Lesion Skin Diseases}},
  booktitle = {Frontiers in Medicine},
  year = {2020},
  doi = {10.3389/fmed.2020.574329},
}
Figure