Mitigating calibration bias without fixed attribute grouping for improved fairness in medical imaging analysis

Changjian Shui, Justin Szeto, Raghav Mehta, Douglas Lorne Arnold, Tal Arbel

Introduction

In the deployment of artificial intelligence for medical diagnostics, high classification accuracy is often insufficient to ensure clinical safety. A critical yet frequently overlooked metric is calibration—the degree to which a model’s predicted probability reflects the actual likelihood of a specific outcome. For instance, if a diagnostic tool identifies a skin lesion as malignant with 90% confidence, that lesion should indeed be malignant in 90% of similar cases.

However, even models that appear well-calibrated on average can exhibit "calibration bias," where the model is significantly overconfident or underconfident for specific patient subgroups (e.g., based on age, sex, or disease severity). This paper introduces Cluster-Focal, a novel two-stage training strategy designed to mitigate these biases without requiring explicit knowledge of sensitive patient attributes during the training phase.

The Problem: Hidden Calibration Bias

Traditional fairness interventions in machine learning typically rely on pre-defined labels for sensitive attributes, such as race or gender, to balance performance across groups. In clinical practice, these labels are often missing, incomplete, or inaccurate. Furthermore, bias can emerge from "image-derived" factors that researchers may not have considered, such as the total volume of lesions in a scan or specific hardware artifacts from different imaging centers.

When a model exhibits calibration bias, it poses a significant risk to patient safety. A clinician might trust an overconfident but incorrect prediction for a specific demographic, leading to missed diagnoses or unnecessary interventions. Addressing this requires a "blind" debiasing approach that can identify and correct for performance disparities across latent groups.

The Cluster-Focal Framework

The researchers propose a two-stage framework called Cluster-Focal to address these challenges:

Stage 1: Identifying the Calibration Gap

The process begins by training a standard "vanilla" model on the dataset. After initial training, the researchers calculate the calibration gap for every training sample—defined as the absolute difference between the model's confidence level and its actual correctness. Using these gap values, the system employs K-means clustering to group samples based solely on their calibration difficulty. This allows the model to identify which types of data it is struggling to calibrate correctly, regardless of whether those samples share a known demographic label.

Stage 2: Group-wise Focal Loss

In the second stage, a new prediction model is trained using a modified Focal Loss function. While standard focal loss is designed to down-weight "easy" samples and focus on "hard" ones to improve overall accuracy, Cluster-Focal applies this logic "group-wise" across the clusters identified in Stage 1. By prioritizing the calibration of the worst-performing clusters, the model implicitly learns to be more reliable for vulnerable subgroups without ever being explicitly told which patients belong to which demographic.

Experimental Evaluation

The effectiveness of Cluster-Focal was evaluated on two distinct medical imaging tasks:

  1. HAM10000: A public dataset of 10,000 skin lesion images for multi-class classification, with subgroups defined by age (over/under 60) and sex.
  2. Proprietary MS Dataset: A large-scale, multi-center dataset of MRI scans from 602 Multiple Sclerosis (MS) patients. The task involved predicting future lesional activity (new or enlarging brain lesions) two years post-scan. Subgroups included age, sex, and image-derived attributes like "lesion load" (total volume of existing lesions).

Key Results and Insights

The study compared Cluster-Focal against established debiasing methods, including Just Train Twice (JTT) and Adversarially Reweighted Learning (ARL).

  • Improved Calibration in MS Prediction: On the MS dataset, Cluster-Focal improved the calibration error (measured by Q-ECE) by approximately 8% for the worst-performing sex subgroup. This improvement was achieved with only a negligible 2% decrease in the F1-score, demonstrating a favorable trade-off between fairness and raw performance.
  • Superiority Over Baselines: In predicting specific clinical markers like "Gad-enhancing lesion count," Cluster-Focal outperformed the JTT baseline by over 20% in calibration error while maintaining competitive prediction accuracy.
  • Qualitative Reliability: Reliability diagrams indicated that while standard models remained dangerously overconfident for older patients, the Cluster-Focal method successfully adjusted confidence levels to align more closely with actual accuracy rates.

Discussion and Future Work

The primary advantage of Cluster-Focal is its flexibility; it can discover and mitigate biases against groups that researchers might not have anticipated, such as patients with specific lesion volumes. However, the authors note a current limitation: while the method does not require subgroup labels during training, these labels are still necessary at the evaluation stage to verify that the bias has been successfully mitigated.

Future research directions include moving toward dynamic clustering, where the identification of calibration gaps and the focal loss training occur simultaneously rather than in two distinct stages. Additionally, exploring how the method handles intersectional subgroups (e.g., the combination of age, sex, and lesion load) will be vital for ensuring robust performance in diverse clinical environments.

Conclusion

Cluster-Focal represents a significant step toward more equitable medical AI. By focusing on the calibration gap rather than fixed demographic categories, it provides a scalable way to ensure that diagnostic models are not just accurate on average, but consistently reliable for every patient, regardless of their individual characteristics.

Summary updated

Abstract

Trustworthy deployment of deep learning medical imaging models into real-world clinical practice requires that they be calibrated. However, models that are well calibrated overall can still be poorly calibrated for a sub-population, potentially resulting in a clinician unwittingly making poor decisions for this group based on the recommendations of the model. Although methods have been shown to successfully mitigate biases across subgroups in terms of model accuracy, this work focuses on the open problem of mitigating calibration biases in the context of medical image analysis. Our method does not require subgroup attributes during training, permitting the flexibility to mitigate biases for different choices of sensitive attributes without re-training. To this end, we propose a novel two-stage method: Cluster-Focal to first identify poorly calibrated samples, cluster them into groups, and then introduce group-wise focal loss to improve calibration bias. We evaluate our method on skin lesion classification with the public HAM10000 dataset, and on predicting future lesional activity for multiple sclerosis (MS) patients. In addition to considering traditional sensitive attributes (e.g. age, sex) with demographic subgroups, we also consider biases among groups with different image-derived attributes, such as lesion load, which are required in medical image analysis. Our results demonstrate that our method effectively controls calibration error in the worst-performing subgroups while preserving prediction performance, and outperforming recent baselines.
BibTeX
@inproceedings{shui2023mitigating,
  author = {Changjian Shui and Justin Szeto and Raghav Mehta and Douglas Lorne Arnold and Tal Arbel},
  title = {{Mitigating calibration bias without fixed attribute grouping for improved fairness in medical imaging analysis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=VFxrxs1OGT},
}
Figure