A framework for making predictive models useful in practice

Kenneth Jung, Sehj Kashyap, Anand Avati, Stephanie Harman, Heather Shaw, Ron C. Li, Margaret Smith, Kenny Shum, Jacob Javitz, Yohan Vetteth, Tina Seto, Steven C. Bagley, Nigam H. Shah

TL;DR

This paper argues that high-performing AI models often fail in healthcare because they ignore "last-mile" delivery constraints like staff capacity and patient discharge timing. By using a simulation-based cost-benefit framework, the authors demonstrate that investing in outpatient follow-up "rescue" pathways provides significantly more clinical value than simply increasing inpatient capacity or improving model accuracy.

What Problem Does This Paper Address?

There is a notorious "implementation gap" in healthcare AI: we have countless models with high accuracy (AUROC), but very few that actually improve patient outcomes or reduce costs. The authors posit that this is because standard evaluation metrics ignore the messy reality of clinical workflows.

Using Advance Care Planning (ACP)—the process of documenting a patient's end-of-life wishes—as a case study, the authors address a specific challenge: even if a model perfectly identifies patients at high risk of 12-month mortality, the hospital might not have enough staff to talk to them, or the patients might be discharged before a conversation can happen. The paper provides a framework to quantify how these real-world "failure modes" erode the theoretical benefit of predictive models.

Methods

The researchers developed a four-stage framework for model deployment:

  1. Problem Specification: Defining the target (12-month mortality) and the intervention (ACP).
  2. Technical Validation: Building and testing the model.
  3. Utility Analysis: Using simulations to see how workflow constraints impact "Net Utility."
  4. Monitoring: Ongoing evaluation of the running system.

The Model: They built a gradient-boosted tree model (LightGBM) using 63,043 features from Electronic Health Record (EHR) data. The Simulation: This is the core innovation. They didn't just look at model performance; they simulated 5,000 days of hospital operations. They varied parameters like:

  • Daily Capacity: How many ACPs can the staff actually perform per day?
  • Rejection Rate: How often do patients decline the conversation for non-clinical reasons?
  • Time to Completion: How many days does it take to coordinate an ACP?
  • Outpatient Rescue: If an ACP is missed in the hospital, can it be "rescued" in an outpatient setting?

Datasets

  • Training/Validation Set: A retrospective dataset of 97,683 admissions at Stanford Hospital between 2010 and 2017.
  • Prospective Evaluation Set: 178 patient charts reviewed by an experienced palliative care nurse in 2019. The nurse answered the "Surprise Question" ("Would you be surprised if this patient passed away in the next 12 months?") to provide a ground-truth label for ACP appropriateness.
  • Utility Values: Derived from a multi-site Randomized Controlled Trial (RCT) that quantified the cost savings of palliative care ($4,855 net savings per patient in 2003 dollars, adjusted for inflation).

Key Insights and Ideas

  • AUROC is a Distraction: A model can have an AUROC of 0.86 (as this one did), but if the hospital only has the capacity to act on one "hit" per day, more than half of the potential benefit is lost.
  • The "Crazy Day" Constraint: Clinical benefit is non-linear. Increasing staff capacity yields diminishing returns because there are only so many high-risk patients admitted on a given day.
  • Outpatient "Rescue" is the Hero: The most surprising finding was that building an outpatient pathway to catch patients missed during their hospital stay is more effective than hiring more inpatient staff. Even a 50% success rate in an outpatient "rescue" pathway outperformed significant increases in inpatient capacity.
  • Utility as a Decision Tool: By assigning dollar values (or "utility") to True Positives, False Positives, etc., the authors could identify the optimal "probability threshold" for triggering an intervention, balancing the workload against the expected benefit.

Key Results

  • Model Performance: The 12-month mortality model achieved an AUROC of 0.86.
  • Capacity Impact: Limiting capacity to 1 ACP per day captured less than 50% of the "best-case" utility.
  • Discharge Impact: If an ACP takes 2 days to complete on average, the net benefit drops to 75% of the best case due to patients being discharged too early.
  • The Tradeoff: An outpatient pathway with a 50% success rate resulted in greater total savings than increasing inpatient capacity by one additional intervention per day across all starting levels.

Limitations and Future Work

  • Proxy for Utility: The authors used "healthcare expenditures" as a stand-in for utility. They acknowledge that the "true goal" of ACP—patient comfort and goal-concordant care—is harder to measure than dollars saved.
  • Expert Fallibility: The ground truth relied on a single nurse's judgment; experts can be wrong or inconsistent.
  • Simulation Granularity: The simulations are simplified and may not capture every nuance of a complex hospital environment (e.g., the specific "rapport" built between a specific doctor and patient).
  • Future Directions: The authors suggest this framework should be used before deployment to guide resource allocation—deciding whether to spend a budget on a better GPU for the model or another nurse for the clinic.

Reference: Jung, K., et al. (2020). A framework for making predictive models useful in practice. medRxiv preprint.

Summary updated

Abstract

Abstract Objective To analyze the impact of factors in healthcare delivery on the net benefit of triggering an Advanced Care Planning (ACP) workflow based on predictions of 12-month mortality. Materials and Methods We built a predictive model of 12-month mortality using electronic health record data and evaluated the impact of healthcare delivery factors on the net benefit of triggering an ACP workflow based on the models’ predictions. Factors included non-clinical reasons that make ACP inappropriate, limited capacity for ACP, inability to follow up due to patient discharge, and availability of an outpatient workflow to follow up on missed cases. We also quantified the relative benefits of increasing capacity for inpatient ACP versus outpatient ACP. Results Work capacity constraints and discharge timing can significantly reduce the net benefit of triggering the ACP workflow based on a model’s predictions. However, the reduction can be mitigated by creating an outpatient ACP workflow. Given limited resources to either add capacity for inpatient ACP versus developing outpatient ACP capability, the latter is likely to provide more benefit to patient care. Discussion The benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model. We provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit. Conclusion An analysis of the sensitivity of the net benefit realized by a predictive model triggered clinical workflow to various healthcare delivery factors is necessary for making predictive models useful in practice.
BibTeX
@article{jung2021framework,
  author = {Kenneth Jung and Sehj Kashyap and Anand Avati and Stephanie Harman and Heather Shaw and Ron C. Li and Margaret Smith and Kenny Shum and Jacob Javitz and Yohan Vetteth and Tina Seto and Steven C. Bagley and Nigam H. Shah},
  title = {{A framework for making predictive models useful in practice}},
  journal = {J. Am. Medical Informatics Assoc.},
  year = {2021},
  doi = {10.1093/jamia/ocaa318},
}

Loading related content...

Figure