Abstract: Ensuring fairness across diverse patient populations is a fundamental challenge for clinical AI systems, yet current fairness evaluation approaches create critical blind spots. Group-level metrics capture systemic disparities but miss patient-level variations, while individual fairness frameworks ensure consistency but potentially obscure structural biases. In this paper, we propose EquiLense, a post-hoc, model-agnostic framework that bridges these perspectives through clinical similarity matching and comprehensive fairness auditing. Our method introduces the Mean Predicted Probability Difference (MPPD), which quantifies prediction inconsistencies between clinically similar patients across demographic groups, integrating both individual-level consistency and group-level equity assessment. Moreover, we provide flexible similarity matching using clinical features and comprehensive visualization tools that support practical deployment in healthcare settings. Applied to electronic health record data from over 59,000 surgical patients, our framework revealed disparities in prediction consistency even when overall model performance appeared strong. EquiLense identified differences in predicted probabilities between clinically similar patients from different racial groups, disparities that were substantially reduced when sensitive attributes were excluded from model training. Our method provides a clinically relevant and interpretable approach to fairness auditing that enables healthcare practitioners to identify, understand, and address algorithmic disparities in real-world deployment settings.
Introduction
In the deployment of clinical AI, a persistent tension exists between group fairness—ensuring balanced outcomes across broad demographic categories—and individual fairness—ensuring that similar individuals receive similar outcomes. Group-level metrics can often mask significant variations within a group, while individual metrics might miss systemic, structural biases that affect entire demographics. In clinical settings, defining "similarity" is complex, involving multifaceted medical data that existing tools often fail to handle in a way that is interpretable for clinicians.
We present EquiLense, a post-hoc, model-agnostic framework designed to bridge this gap. By focusing on "clinically similar" patients, EquiLense identifies cases where patients receive different predictions primarily due to their demographic group membership, revealing hidden biases even in high-performing models.
Methodology: The EquiLense Framework
The core of the EquiLense framework is the Mean Predicted Probability Difference (MPPD). This metric quantifies the inconsistency in model predictions for patients who are clinically similar but belong to different sensitive attribute groups. The workflow consists of four primary steps:
- Similarity Mapping: Identify "clinically similar" patients using relevant features such as age, BMI, and comorbidity indices (e.g., via k-nearest neighbors).
- Cross-Group Matching: Match these similar patients across different sensitive attribute groups (e.g., comparing a White patient to a clinically similar Asian patient).
- Probability Comparison: Calculate the absolute difference in the model’s predicted probabilities for these matched pairs.
- Aggregation: Average these differences to produce the final MPPD score.
EquiLense also incorporates visualization tools, such as radar plots for group metrics and heatmaps for pairwise MPPD comparisons, to help auditors pinpoint specific model failures.
Experimental Evaluation
The framework was evaluated using Electronic Health Record (EHR) data from Stanford Medicine Partners, covering 59,047 surgical patients and 75,179 total surgeries between 2012 and 2022. The models were designed to predict two critical outcomes: post-surgical delirium and 30-day hospital readmission. Features included both sensitive attributes (sex, race/ethnicity, insurance type) and clinical attributes (age, BMI, Charlson Comorbidity Index, anesthesia type, etc.).
Key Results and Insights
Our analysis demonstrated that standard performance metrics are not a proxy for fairness. An XGBoost model for delirium achieved a solid AUROC of 0.78, yet EquiLense revealed significant underlying fairness issues.
- Disparity Detection: The largest disparity in the delirium model was found between White and Asian patients. Clinically similar patients in these groups had an MPPD of 0.119, significantly higher than the internal variation within the White group (0.074).
- The "Stealth" Impact of Sensitive Attributes: Interestingly, removing race or sex from the training data had almost no impact on model accuracy or AUROC. However, it significantly reduced the MPPD, suggesting that these attributes contributed to inconsistent predictions for similar patients without providing meaningful predictive signal.
- Contextual Similarity: The definition of "clinical similarity" is subjective and context-dependent. EquiLense allows users to define which clinical features determine similarity, making the audit relevant to specific medical tasks.
Conclusion and Future Work
EquiLense provides a robust diagnostic tool for identifying algorithmic inconsistencies in clinical settings. While it identifies disparities, it does not currently provide automated mitigation or prove causal links for the observed biases. Future work will focus on establishing standardized guidelines for similarity thresholds across different clinical domains and exploring how these biased predictions impact real-world patient care and health outcomes.
Summary updated