Abstract
Deep learning has the potential to automate many clinically useful tasks in medical imaging. However, translation into clinical practice has been hindered by a lack of transparency and interpretability in these "black box" algorithms. Specifically, many models lack robust techniques for conveying certainty, limiting their appeal for medical decision-making. Furthermore, algorithmic bias has increased hesitancy towards clinical deployment. We explore how conformal predictions (CP) can provide an intuitive way of expressing uncertainty while facilitating transparency. Through field interviews with radiologists, we identified two clinical use-cases—"rule-in" and "rule-out"—and evaluated CP methods on a dermatology dataset. We demonstrate how to modify CP to be more adaptive to subgroup differences in patient skin tones through equalized coverage and compare CP against traditional measures of epistemic uncertainty.
Introduction
The adoption of deep learning in clinical settings is often stalled by the "black box" nature of these models. Clinicians require more than just a probability score; they need a rigorous way to understand the model's certainty. Moreover, the risk of disparate performance across demographic groups—such as different skin tones in dermatology—poses a significant ethical and clinical challenge. This research addresses these issues by applying Conformal Prediction (CP) to medical imaging, offering a framework that provides statistically guaranteed uncertainty sets while ensuring fairness across patient subgroups.
Methods and Clinical Use-Cases
Based on interviews with radiologists, the authors framed the AI's output as a "differential diagnosis" set. Two primary use-cases were defined:
- Rule-in: Identifying all possible critical conditions to ensure high sensitivity.
- Rule-out: Confidently identifying normal or benign cases to expedite patient discharge.
The study utilized the Fitzpatrick17k dataset, containing 16,577 clinical images of skin lesions categorized as non-neoplastic, benign, or malignant. A key challenge identified in the data is the significant imbalance across skin types, which can lead to biased performance in standard models.
To address this, the researchers introduced Group Adaptive Prediction Sets (GAPS) and Group Regularized Adaptive Prediction Sets (GRAPS). Unlike standard CP, which uses a global threshold, these methods calculate separate thresholds for each Fitzpatrick skin type, ensuring that the target confidence level (coverage) is met for every subgroup individually.
Results: Achieving Equalized Coverage
The core technical finding is that group-calibrated CP methods successfully achieve equalized coverage. In standard models (like APS), error rates often fluctuate wildly between subgroups due to data scarcity or inherent difficulty. By using GAPS and GRAPS, the researchers forced the model to maintain the desired miscoverage level ((\alpha)) across all skin tones.
The size of the resulting prediction sets acts as a natural proxy for uncertainty. Larger sets were produced for skin types with less training data (Type 6) or for more complex diagnostic categories (Type 1 malignancies). This provides clinicians with an intuitive signal: a larger set indicates the model is less certain and the case requires closer human inspection. The study found a strong correlation (0.66 to 0.87) between these set sizes and traditional epistemic uncertainty measures.
Conclusion
Conformal prediction bridges the gap between complex deep learning models and clinical needs. By providing sets of possible diagnoses with formal guarantees, it aligns with medical intuition. Crucially, the introduction of group-adaptive thresholds allows for "fair" uncertainty quantification, ensuring that the AI's reliability is consistent regardless of a patient's skin tone. Future work should explore the integration of these sets into real-time clinical workflows to measure their impact on diagnostic accuracy and bias reduction in practice.
Summary updated