Transfer Learning with Ensembles of Deep Neural Networks for Skin Cancer Detection in Imbalanced Data Sets

Aqsa Saeed Qureshi, Teemu RoosNeural Process. Lett 2023 · OpenReview

Abstract

Early diagnosis is critical for the effective treatment of skin cancer. While machine learning techniques, particularly convolutional neural networks (CNNs), have shown promise in medical image analysis, their performance is often hampered by the extreme scarcity of malignant samples in clinical datasets. This paper proposes a novel ensemble-based CNN architecture that integrates multiple models trained on various image scales, hybrid transfer learning, and patient metadata. By utilizing a Support Vector Machine (SVM) as a meta-learner, the proposed approach significantly improves classification performance on highly imbalanced data, outperforming several state-of-the-art benchmarks.

Introduction

Skin cancer detection presents a significant challenge for automated systems due to the extreme class imbalance found in real-world datasets. In the ISIC 2020 archive, for instance, malignant lesions represent less than 2% of the total images. Standard CNNs often struggle with this imbalance, frequently defaulting to the majority (benign) class to maintain high overall accuracy while failing to identify the clinically critical minority (malignant) class. While transfer learning from general-purpose datasets like ImageNet provides a starting point, it often lacks the specificity required for nuanced dermatological features.

Proposed Methodology

The core of the proposed system is a multi-stage ensemble architecture designed to maximize the diversity of information extracted from dermoscopic images. The architecture consists of three primary components:

  1. Multi-Scale Base-Learners: The system employs six CNN base-learners. Four are trained from scratch on the target dataset using varying input resolutions (32×32, 64×64, 128×128, and 256×256). This multi-scale approach allows the ensemble to simultaneously capture fine-grained textures and broader structural patterns.
  2. Hybrid Transfer Learning: To inject prior dermatological knowledge, two additional base-learners are pre-trained on a separate, balanced skin cancer dataset from the ISIC archive. This helps mitigate the bias introduced by the primary dataset's imbalance.
  3. SVM Meta-Learner: Rather than simple averaging, the predicted probabilities from all six CNNs are combined with auxiliary patient metadata (age, sex, and anatomical site). These inputs are fed into an SVM with a Radial Basis Function (RBF) kernel, which learns to optimally weight the various inputs for a final diagnosis.
Flowchart of the proposed multi-scale ensemble architecture.

Dataset and Experimental Setup

The primary evaluation was conducted using the ISIC 2020 Challenge dataset, which contains 33,126 dermoscopic images from 2,056 patients. The dataset is characterized by extreme imbalance, with only 584 malignant images. For the pre-trained components, a balanced archive from 2019 was utilized to prevent data leakage.

A sample dermoscopic image from the ISIC 2020 dataset.

Experimental Results

The proposed ensemble method was compared against seven benchmarks, including traditional classifiers and recent deep learning techniques. The results demonstrate the superiority of the ensemble approach in imbalanced contexts:

  • F1-Measure: The proposed method achieved an F1-score of 0.23, nearly doubling the performance of some recent CNN-based approaches (e.g., Mahbod et al. at 0.12).
  • AUC-PR: In the presence of high imbalance, the Area Under the Precision-Recall curve is a critical metric. The proposed method reached 0.16, significantly higher than the 0.12–0.13 range of top benchmarks.
  • AUC-ROC: The model maintained a strong overall discriminative power with an AUC-ROC of 0.87.

A key insight from the results is that the ensemble outperforms its individual components. Even if base-learners have lower individual F1-scores, their diverse errors allow the meta-learner to filter noise and improve the final classification.

Conclusion

This research highlights the power of ensemble learning and multi-scale feature extraction in addressing the challenges of imbalanced medical data. By combining specialized CNNs with clinical metadata, the proposed architecture offers a more robust tool for early skin cancer detection. Future work will explore the use of Generative Adversarial Networks (GANs) to synthesize minority class samples and the integration of richer hyperspectral data to further refine diagnostic accuracy.

Summary updated

Abstract

Early diagnosis plays a key role in prevention and treatment of skin cancer. Several machine learning techniques for accurate detection of skin cancer from medical images have been reported. Many of these techniques are based on pre-trained convolutional neural networks (CNNs), which enable training the models based on limited amounts of training data. However, the classification accuracy of these models still tends to be severely limited by the scarcity of representative images from malignant tumours. We propose a novel ensemble-based convolutional neural network (CNN) architecture where multiple CNN models, some of which are pre-trained and some are trained only on the data at hand, along with auxiliary data in the form of metadata associated with the input images, are combined using a meta-learner. The proposed approach improves the model’s ability to handle limited and imbalanced data. We demonstrate the benefits of the proposed technique using a dataset with 33,126 dermoscopic images from 2056 patients. We evaluate the performance of the proposed technique in terms of the F1-measure, area under the ROC curve (AUC-ROC), and area under the PR-curve (AUC-PR), and compare it with that of seven different benchmark methods, including two recent CNN-based techniques. The proposed technique compares favourably in terms of all the evaluation metrics.
BibTeX
@inproceedings{qureshi2023transfer,
  author = {Aqsa Saeed Qureshi and Teemu Roos},
  title = {{Transfer Learning with Ensembles of Deep Neural Networks for Skin Cancer Detection in Imbalanced Data Sets}},
  booktitle = {Neural Process. Lett},
  year = {2023},
  url = {https://openreview.net/forum?id=D6cGsBQrlk},
}
Figure