Introduction
Predictive models in healthcare are increasingly used to guide critical decisions, such as identifying patients for Advance Care Planning (ACP). While guidelines like TRIPOD-AI and PROBAST-AI emphasize the need for reliability and fairness audits, a significant "implementation gap" exists between these theoretical principles and actual clinical practice. This study provides a practical roadmap for auditing healthcare AI by conducting a head-to-head assessment of two mortality prediction models, revealing the substantial operational effort required and the critical importance of intersectional subgroup analysis.
Methodology
The researchers evaluated two models designed to identify patients at high risk of 12-month mortality: the commercial Epic End-of-Life (EOL) Index (logistic regression) and an internally developed Stanford Hospital Medicine (HM) ACP model (gradient-boosted tree).
To assess these models, the team utilized the "Surprise Question" (SQ) as a ground truth surrogate: "Would you be surprised if [patient X] passed away in [Y years]?" Performance was measured through Positive Predictive Value (PPV), sensitivity, specificity, and calibration (Observed/Expected ratios). Crucially, the audit employed an intersectional fairness framework, examining performance across combined demographic categories (e.g., Hispanic Males) rather than isolated variables. Finally, the researchers surveyed 10 clinicians to understand how audit metrics influenced their trust in the AI tools.
Data and Settings
The study leveraged Electronic Health Record (EHR) data from Stanford Health Care, mapped to the OMOP Common Data Model. The evaluation set comprised 1,085 clinician-generated labels across Primary Care, Inpatient Oncology, and Hospital Medicine. Demographic data included age, sex, and race/ethnicity, with the latter treated as a social construct to acknowledge the limitations of EHR-recorded data.
Key Findings: Performance and Fairness
The audit revealed significant disparities between the two models. While both maintained high precision (PPV ≥ 0.76), the Stanford model demonstrated superior sensitivity and calibration. In Hospital Medicine, the Stanford model achieved a sensitivity of 0.69, catching nearly 70% of high-risk patients, whereas the Epic model caught only 20% (sensitivity 0.20).
Calibration was a major differentiator. The Epic model consistently underpredicted risk, with O/E ratios between 2.5 and 3.0, indicating that actual "surprising" deaths occurred far more frequently than predicted. In contrast, the Stanford model's O/E ratios (1.5 to 1.7) were much closer to the ideal 1.0. Clinician surveys confirmed that subgroup calibration is a vital metric for trust, with 90% stating it would influence their decision to use the tool.
The intersectional analysis uncovered critical "blind spots." For instance, the Epic model showed 0% sensitivity for Hispanic/Latino males in Inpatient Oncology—a failure that would have been hidden by aggregate statistics. Both models generally exhibited lower sensitivity for Hispanic/Latino male patients compared to the overall population.
The Operational "Tax"
A key takeaway for healthcare organizations is the massive operational effort required for a robust audit. The study documented a "tax" of 115 person-hours over 8–10 months.
The majority of this time was consumed by "data cascades"—complex tasks such as cleaning messy demographic data, linking clinicians to patients for labeling, and navigating institutional data access hurdles. This suggests that auditing is as much an organizational and informatics challenge as it is a computational one.
Conclusion and Future Directions
The study highlights that local tuning or custom model architecture may be necessary for high-stakes clinical tasks, as the internal Stanford model outperformed the commercial alternative. However, the use of surrogate outcomes like the "Surprise Question" and the inherent inaccuracies in EHR demographic data remain limitations.
Future efforts should focus on empowering "model auditors" to bridge the gap between developers and clinical stakeholders. Informatics teams must also prioritize building better clinician-patient linkage data to streamline the auditing process, making it a routine part of the AI lifecycle in healthcare.
Summary updated