Knowledge transfer for melanoma screening with deep learning

Afonso Menegola, Michel Fornaciali, Ramon Pires, Flávia Vasques Bittencourt, Sandra Avila, Eduardo Valle

Introduction

Melanoma is the most lethal form of skin cancer, yet early detection remains the most effective way to improve patient survival rates. While Deep Neural Networks (DNNs) have revolutionized image classification, their performance is heavily dependent on large-scale labeled datasets. In medical imaging, datasets are often limited to hundreds or thousands of images, making training from scratch impractical. This paper investigates transfer learning—the process of leveraging knowledge from a pre-trained model—as a solution for melanoma screening, specifically evaluating the impact of source datasets and fine-tuning strategies.

Methodology

The researchers evaluated several transfer learning strategies using VGG-M and VGG-16 architectures across four datasets:

  • Target: Atlas (clinical cases) and ISIC 2016 Challenge data.
  • Source: ImageNet (general objects) and a Diabetic Retinopathy dataset (eye images).

The experimental design compared:

  1. No Transfer: Training from scratch.
  2. Feature Extraction (No FT): Using pre-trained weights as fixed features for an SVM.
  3. Fine-Tuning (FT): Updating pre-trained weights on the target skin lesion data.
  4. Double Transfer: Sequential transfer from ImageNet to Retinopathy, then to Melanoma.

The Impact of Transfer Sources

A primary finding of the study is that the choice of source data significantly affects performance. Contrary to the intuition that "medical-to-medical" transfer would be superior, models pre-trained on the general ImageNet dataset outperformed those pre-trained on the specialized Diabetic Retinopathy dataset.

Performance comparison across different transfer learning strategies.

As shown in the results, fine-tuning on ImageNet-based models yielded the highest AUC (up to 83.6% for multi-class tasks). Transferring from Retinopathy without fine-tuning actually degraded performance compared to the baseline. This suggests that specialized models may overfit to domain-specific features (like retinal vasculature) that do not generalize to dermatological structures.

Fine-Tuning and Architecture

The study confirms that fine-tuning is essential. Static feature extraction lacks the nuance required for medical diagnosis. Furthermore, model depth remains a factor even in data-constrained environments; the deeper VGG-16 architecture consistently outperformed the shallower VGG-M, achieving a peak AUC of 84.5% on the Atlas dataset.

The Difficulty Gap in AI Diagnosis

While the models achieved competitive overall results, performance varied drastically based on the inherent difficulty of the cases. When stratified by human-rated difficulty, a significant performance gap emerged.

Model performance (AUC) stratified by diagnosis difficulty.

The models performed exceptionally well on "Low" difficulty cases (93%+ AUC) but struggled on "High" difficulty cases, where AUC dropped to approximately 56-58%. This indicates that current AI models are highly effective for "referability"—identifying obvious malignancies—but cannot yet replace expert clinical judgment for subtle or borderline lesions.

Conclusion

Transfer learning from general-purpose datasets like ImageNet, combined with fine-tuning, provides a robust baseline for melanoma detection. However, the "medical-to-medical" transfer trap and the performance drop on difficult cases highlight the need for more nuanced transfer strategies and larger, more diverse medical datasets. Future work should explore more visually similar medical domains and the integration of AI as a triage tool to assist, rather than replace, human experts.

Summary updated

BibTeX
@inproceedings{menegola2017knowledge,
  author = {Afonso Menegola and Michel Fornaciali and Ramon Pires and Flávia Vasques Bittencourt and Sandra Avila and Eduardo Valle},
  title = {{Knowledge transfer for melanoma screening with deep learning}},
  booktitle = {ISBI},
  year = {2017},
  doi = {10.1109/isbi.2017.7950523},
}
Figure