Fair Conformal Predictors for Applications in Medical Imaging

Charles Lu, Andréanne Lemay, Ken Chang, Katharina Hoebel, Jayashree Kalpathy-Cramer

Abstract

Deep learning has the potential to automate many clinically useful tasks in medical imaging. However, translation into clinical practice has been hindered by a lack of transparency and interpretability in these "black box" algorithms. Specifically, many models lack robust techniques for conveying certainty, limiting their appeal for medical decision-making. Furthermore, algorithmic bias has increased hesitancy towards clinical deployment. We explore how conformal predictions (CP) can provide an intuitive way of expressing uncertainty while facilitating transparency. Through field interviews with radiologists, we identified two clinical use-cases—"rule-in" and "rule-out"—and evaluated CP methods on a dermatology dataset. We demonstrate how to modify CP to be more adaptive to subgroup differences in patient skin tones through equalized coverage and compare CP against traditional measures of epistemic uncertainty.

Introduction

The adoption of deep learning in clinical settings is often stalled by the "black box" nature of these models. Clinicians require more than just a probability score; they need a rigorous way to understand the model's certainty. Moreover, the risk of disparate performance across demographic groups—such as different skin tones in dermatology—poses a significant ethical and clinical challenge. This research addresses these issues by applying Conformal Prediction (CP) to medical imaging, offering a framework that provides statistically guaranteed uncertainty sets while ensuring fairness across patient subgroups.

Methods and Clinical Use-Cases

Based on interviews with radiologists, the authors framed the AI's output as a "differential diagnosis" set. Two primary use-cases were defined:

  1. Rule-in: Identifying all possible critical conditions to ensure high sensitivity.
  2. Rule-out: Confidently identifying normal or benign cases to expedite patient discharge.

The study utilized the Fitzpatrick17k dataset, containing 16,577 clinical images of skin lesions categorized as non-neoplastic, benign, or malignant. A key challenge identified in the data is the significant imbalance across skin types, which can lead to biased performance in standard models.

Distribution of Fitzpatrick skin types across diagnostic categories in the Fitzpatrick17k dataset, highlighting the underrepresentation of darker skin tones (Type 6).

To address this, the researchers introduced Group Adaptive Prediction Sets (GAPS) and Group Regularized Adaptive Prediction Sets (GRAPS). Unlike standard CP, which uses a global threshold, these methods calculate separate thresholds for each Fitzpatrick skin type, ensuring that the target confidence level (coverage) is met for every subgroup individually.

Results: Achieving Equalized Coverage

The core technical finding is that group-calibrated CP methods successfully achieve equalized coverage. In standard models (like APS), error rates often fluctuate wildly between subgroups due to data scarcity or inherent difficulty. By using GAPS and GRAPS, the researchers forced the model to maintain the desired miscoverage level ((\alpha)) across all skin tones.

Comparison of coverage and prediction set size across skin types for various CP methods. GAPS and GRAPS (right columns) show much tighter clustering of coverage lines, indicating successful equalization across subgroups.

The size of the resulting prediction sets acts as a natural proxy for uncertainty. Larger sets were produced for skin types with less training data (Type 6) or for more complex diagnostic categories (Type 1 malignancies). This provides clinicians with an intuitive signal: a larger set indicates the model is less certain and the case requires closer human inspection. The study found a strong correlation (0.66 to 0.87) between these set sizes and traditional epistemic uncertainty measures.

Conclusion

Conformal prediction bridges the gap between complex deep learning models and clinical needs. By providing sets of possible diagnoses with formal guarantees, it aligns with medical intuition. Crucially, the introduction of group-adaptive thresholds allows for "fair" uncertainty quantification, ensuring that the AI's reliability is consistent regardless of a patient's skin tone. Future work should explore the integration of these sets into real-time clinical workflows to measure their impact on diagnostic accuracy and bias reduction in practice.

Summary updated

Abstract

Deep learning has the potential to automate many clinically useful tasks in medical imaging. However translation of deep learning into clinical practice has been hindered by issues such as lack of the transparency and interpretability in these "black box" algorithms compared to traditional statistical methods. Specifically, many clinical deep learning models lack rigorous and robust techniques for conveying certainty (or lack thereof) in their predictions -- ultimately limiting their appeal for extensive use in medical decision-making. Furthermore, numerous demonstrations of algorithmic bias have increased hesitancy towards deployment of deep learning for clinical applications. To this end, we explore how conformal predictions can complement existing deep learning approaches by providing an intuitive way of expressing uncertainty while facilitating greater transparency to clinical users. In this paper, we conduct field interviews with radiologists to assess possible use-cases for conformal predictors. Using insights gathered from these interviews, we devise two clinical use-cases and empirically evaluate several methods of conformal predictions on a dermatology photography dataset for skin lesion classification. We show how to modify conformal predictions to be more adaptive to subgroup differences in patient skin tones through equalized coverage. Finally, we compare conformal prediction against measures of epistemic uncertainty.
BibTeX
@inproceedings{lu2022fair,
  author = {Charles Lu and Andréanne Lemay and Ken Chang and Katharina Hoebel and Jayashree Kalpathy-Cramer},
  title = {{Fair Conformal Predictors for Applications in Medical Imaging}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21459},
}
Figure