Abstract
Deep learning models for melanoma detection often struggle with robustness and calibration in real-world clinical settings. This study investigates whether using multiple real-world dermoscopic photographs of a single lesion (MV-Real) improves performance compared to single-view or standard test-time augmentation (MV-Artificial) approaches. Evaluating a ConvNeXT-based classifier on a prospective dataset of 656 lesions, we find that the MV-Real approach significantly enhances AUROC, reduces calibration error, and improves robustness. These findings suggest that genuine visual diversity from multiple physical perspectives is superior to digital transformations for enhancing the reliability of AI in dermatology.
Introduction
Convolutional Neural Networks (CNNs) have reached a milestone in dermatology, often matching or exceeding the diagnostic performance of expert clinicians in controlled environments. Despite this success, translating these models to the clinic reveals significant hurdles: brittleness and overconfidence. A model's prediction may fluctuate based on minor variations in image acquisition, such as angle or lighting, indicating a lack of robustness. Furthermore, these networks are often "confident but wrong," yielding high-probability scores for incorrect classifications, which points to poor calibration.
Standard practice often involves "Test-Time Augmentation" (TTA), where a single image is digitally manipulated (e.g., rotated or brightened) to give the model multiple "looks." However, this paper argues that TTA is a mathematical proxy that does not add new information. This study explores whether taking multiple actual photos from different perspectives can solve these reliability issues.
Methods
The researchers employed a state-of-the-art ConvNeXT architecture trained on a large-scale dataset of 29,562 images from the ISIC archive (HAM10000 and BCN20000). To evaluate the impact of multiple views, they utilized the SCP2 dataset, a prospective multicenter collection of 656 suspected melanoma lesions from eight German university hospitals. Each lesion in SCP2 was documented with six distinct real-world photographs, varying in angle, position, and dermatoscope mode (polarized vs. non-polarized).
The study compared three primary diagnostic workflows:
- Single-View: A baseline using only the primary "original" image.
- MV-Artificial (TTA): Generating five additional views from the original image using digital transforms like rotation, zooming, and brightness adjustments.
- MV-Real: Utilizing the six distinct physical photographs taken by the physician.
In the multiview approaches, individual predictions for each image were averaged to produce a final diagnostic score. Performance was measured using AUROC for accuracy, Expected Calibration Error (ECE) for probability reliability, and Maximum Confidence Change (MCC) for robustness.
Key Results
The "MV-Real" approach demonstrated superior performance across all evaluated metrics. The diagnostic accuracy (AUROC) improved from 0.905 in the single-view baseline to 0.930. More importantly, the model's reliability saw significant gains: the ECE dropped from 0.131 to 0.072, indicating that the model's confidence scores more accurately reflected the true likelihood of disease. Robustness also improved, with the MCC decreasing from 0.149 to 0.115.
A critical finding was the "Robustness Paradox": while MV-Artificial (TTA) provided a minor boost to accuracy, it actually decreased the model's robustness compared to the single-view baseline. This suggests that digital transformations can introduce noise that makes model predictions more erratic, whereas real-world photographs provide complementary information—such as different perspectives on lesion borders or the mitigation of hair occlusions—that stabilizes the model.
Discussion and Limitations
The results underscore that "more is better" only when the additional data provides genuinely new visual information. The synergy between polarized and non-polarized imaging modes in the MV-Real set likely contributed to this performance leap, as different modes highlight different pathological features.
However, the practical application of this method faces challenges regarding clinical workflow efficiency. Requiring six photos per lesion increases the time burden on dermatologists. Future research should focus on determining the optimal number of images—perhaps two or three—needed to capture the majority of the benefits without significantly slowing down examinations. Additionally, while this study focused on binary classification (melanoma vs. nevus), extending this multiview approach to complex, multiclass diagnostic environments remains a vital next step for clinical integration.
Summary updated