Abstract
Computer-aided skin cancer detection systems built with deep neural networks yield overconfident predictions on out-of-distribution examples. Motivated by the importance of out-of-distribution detection in these systems and the lack of relevant benchmarks targeted for skin cancer classification, we introduce a rich collection of out-of-distribution datasets -- designed to comprehensively evaluate state-of-the-art out-of-distribution algorithms with skin cancer classifiers. In addition, we propose an adaptation in the Gram-Matrix algorithm for out-of-distribution detection that generally performs better and faster than the original algorithm for the considered skin cancer classification task. We also include a detailed discussion comparing the various state-of-the-art out-of-distribution detection algorithms and identify avenues for future research.
Introduction
In clinical practice, a skin cancer classifier should ideally only process images of the specific diseases it was trained to recognize. However, real-world deployment often involves irrelevant inputs, such as blurred fingers, histology slides, or even non-medical images. Standard Deep Neural Networks (DNNs) are notoriously overconfident, frequently assigning high-probability scores to these out-of-distribution (OOD) samples. This lack of a "reject" option poses a significant safety risk and undermines clinical trust. While OOD detection is a mature field in general computer vision, dermatology lacks specialized benchmarks that account for "near-OOD" cases—where healthy skin may closely resemble a lesion.
Methods: The Gram-OOD* Algorithm
The authors evaluate several state-of-the-art OOD detection strategies, such as ODIN and Mahalanobis distance. Their primary contribution is Gram-OOD*, an optimized version of the Gram Matrix approach. This method detects anomalies by analyzing pairwise correlations between feature maps across different network layers.
Gram-OOD* introduces three critical improvements over the original algorithm:
- Additional Normalization: Gram Matrix values are mapped to a [0, 1] range, ensuring deviations are comparable across different layers.
- Layer Pruning: By focusing only on activation function layers and skipping convolutional layers, the algorithm maintains performance while doubling processing speed.
- Order Reduction: Unlike the original algorithm that used correlations up to the 10th power, the authors found that "Order-1" (simple correlations) is more robust for the texture-heavy images found in dermatology.
Experimental Setup and Datasets
Models were trained on the ISIC 2019 dataset, covering eight skin disease classes. To evaluate OOD detection, the authors curated six diverse datasets to test the model's ability to identify "unknown" samples:
- Derm-Skin & Clin-Skin: Images of healthy skin (dermoscopic and clinical), representing challenging "near-OOD" cases.
- ImageNet: Natural images (e.g., animals, vehicles) representing "far-OOD" samples.
- B-box & B-box-70: ISIC images with lesions partially obscured by black boxes to test reliance on silhouettes.
- NCT: Histology images of colorectal cancer.
Key Insights and Results
A major finding is the "biased trap" in OOD research: many methods report high accuracy because they tune hyperparameters using the OOD data they intend to detect. The authors advocate for unbiased evaluation, where the detector is tuned on one OOD set and tested on entirely different, unseen data.
In these unbiased tests, Gram-OOD significantly outperformed Mahalanobis-based approaches. On the DenseNet-121 architecture, Gram-OOD showed consistent performance across datasets, whereas Mahalanobis performance fluctuated when encountering novel OOD types. On the official ISIC challenge "unknown" task, the method achieved an AUC of ~70% without external data or specific tuning. Furthermore, by pruning layers and reducing orders, the detector is optimized for real-time use in mobile medical applications.
Conclusion
While Gram-OOD* offers a robust and efficient solution for OOD detection in dermatology, challenges remain. The gap in detecting "near-OOD" cases like healthy skin versus flat lesions persists. Future research should explore more sophisticated ensembling techniques to combine information across network layers, as simple summation sometimes degrades performance. Achieving higher AUCs in unbiased settings is essential before these systems can be fully integrated into autonomous clinical workflows.
Summary updated