Abstract
Deep learning is becoming increasingly ubiquitous in medical research and applications, often involving sensitive information and critical diagnostic decisions. However, researchers have observed a significant performance disparity among subgroups with different demographic attributes—a phenomenon known as model unfairness. While existing efforts to address this often involve complex architectures that increase training burdens or degrade generalization, we propose FairAdaBN. This framework makes batch normalization adaptive to sensitive attributes, allowing for a simple yet effective design that can be integrated into various classification backbones. Additionally, we introduce a novel Statistical Disparity Loss ($L_{SD}$) to restrain statistical parity between subgroups and a new metric, Fairness-Accuracy Trade-off Efficiency (FATE), to evaluate the balance between model performance and fairness. Experiments on two dermatological datasets demonstrate that FairAdaBN outperforms existing methods in both fairness criteria and trade-off efficiency.
Introduction
As deep learning models are deployed in clinical settings, ensuring equitable performance across diverse patient populations is paramount. In dermatology, models trained on datasets lacking diversity often exhibit "model unfairness," performing significantly better for privileged groups (e.g., light-skinned patients) than for unprivileged groups (e.g., dark-skinned patients).
Existing mitigation strategies generally fall into two categories: fairness through unawareness, which fails because neural networks often learn correlated demographic features implicitly, and independent modeling, which is computationally inefficient and prevents the model from learning shared features across the entire dataset. FairAdaBN seeks a middle ground: a single architecture that shares core knowledge while maintaining specialized adapters for demographic subgroups.
Methodology
The FairAdaBN framework consists of three primary innovations designed to balance diagnostic accuracy with demographic equity.
1. Fair Adaptive Batch Normalization (FairAdaBN)
Standard Batch Normalization (BN) layers utilize a single set of parameters ($\gamma, \beta$) and running statistics ($\mu, \sigma$) for the entire training set. FairAdaBN replaces these with subgroup-specific versions. While the convolutional layers—which represent the core feature-extraction "knowledge"—are shared across all groups, the normalization layers are parallelized. This allows the model to dynamically align feature maps for different subgroups, effectively filtering out demographic-specific statistical shifts before they propagate through the network.
2. Statistical Disparity Loss ($L_{SD}$)
To ensure the model treats different groups fairly beyond mere architectural adjustments, we introduce a novel loss function, $L_{SD}$. This term penalizes the model if the distribution of its predictions differs significantly between subgroups within a mini-batch. By minimizing this disparity, the model is encouraged to converge toward a state of statistical parity, ensuring that the probability of a specific diagnosis is independent of the sensitive attribute.
3. The FATE Metric
Improving fairness often incurs an "accuracy tax." To quantify the efficiency of this trade-off, we propose the Fairness-Accuracy Trade-off Efficiency (FATE) metric. FATE computes the normalized fairness improvement relative to the drop in accuracy. A higher FATE score indicates a more efficient mitigation strategy, identifying models that achieve significant gains in equity with minimal loss in overall performance.
Experimental Setup
We evaluated FairAdaBN on two major dermatological benchmarks:
- Fitzpatrick-17k: Comprising 16,577 images across 9 diagnostic categories, using skin tone (categorized into "light" and "dark") as the sensitive attribute.
- ISIC 2019: Comprising 25,331 images across 9 categories, using gender (male vs. female) as the sensitive attribute.
In both datasets, subgroups were identified based on historical performance disparities to test the framework's ability to protect unprivileged populations.
Results and Discussion
Superior Fairness and Efficiency
FairAdaBN achieved the lowest scores across multiple fairness metrics, including Equalized Odds (EOdd) and Equal Opportunity (EOpp0/1). Notably, on the Fitzpatrick-17k dataset, FairAdaBN achieved a FATE score of $48.79 \times 10^{-2}$, more than double the efficiency of the next best method (Independent modeling at $22.63 \times 10^{-2}$).
Architecture Portability
A key strength of FairAdaBN is its versatility. We successfully integrated the framework into several popular backbones, including ResNet-152, VGG-19, and DenseNet-121. In all cases, the addition of FairAdaBN improved fairness metrics, proving that the method is architecture-agnostic and can be applied to existing clinical models with minimal modification.
Ablation Studies
Ablation experiments confirmed that both the architectural change (FairAdaBN layers) and the optimization constraint ($L_{SD}$) are vital. Removing either component resulted in a significant degradation of fairness, suggesting that the combination of specialized normalization and distribution-aware loss is necessary to reach the Pareto frontier of fairness and accuracy.
Conclusion
FairAdaBN provides a robust and efficient framework for mitigating demographic bias in medical AI. By treating normalization as a bias filter and optimizing for statistical parity, we can create models that are both accurate and equitable.
Limitations: The primary requirement for FairAdaBN is that the sensitive attribute (e.g., skin tone or gender) must be known at the time of inference to select the appropriate normalization branch. Future work will explore methods to automate this selection and extend the framework to other medical imaging modalities, such as chest X-rays and MRIs, as well as tasks beyond classification, such as lesion segmentation.
Summary updated