TL;DR
This paper introduces a novel score-based CUSUM monitoring procedure to reliably track the performance of machine learning (ML) risk prediction models in healthcare, even when clinical interventions (like prescribing medication) are influenced by the model's predictions. This is crucial for ensuring patient safety and model trustworthiness in real-world clinical settings.
What Problem Does This Paper Address?
When ML models are deployed in healthcare, their predictions can influence clinical decisions. For example, if a model predicts a patient is at high risk for a certain outcome, a doctor might intervene with treatment. This intervention, however, alters the very outcome the model was trying to predict, creating a "confounding medical intervention" (CMI) problem. This makes it difficult to monitor the model's true performance because the observed outcomes are no longer solely a reflection of the patient's underlying risk. Traditional monitoring methods often fail in these scenarios, leading to either too many false alarms or missed performance degradations. The authors tackle this challenge by developing a method that can account for these confounding interventions.
Methods
The core of the paper is a new score-based cumulative sum (CUSUM) monitoring procedure designed to handle confounding medical interventions (CMI). This procedure uses dynamic control limits (DCLs), which adapt over time, to detect performance deterioration. The key innovation lies in how it addresses CMI:
- Conditional Performance Monitoring: Instead of monitoring the overall outcome, the method focuses on the conditional performance of the model. This is achieved by analyzing data from patients who received standard-of-care (SOC) treatment, effectively isolating the model's prediction from the confounding intervention.
- Ignorability Assumptions: The validity of this approach relies on specific assumptions:
- Conditional Exchangeability: This assumes that treatment assignment is independent of the potential outcome, given the model's prediction.
- Time-Constant Selection Bias: This allows for unmeasured confounders to influence treatment decisions, but requires that the bias introduced by these decisions remains consistent over time.
- Score-Based CUSUM: The procedure uses a nonanticipative score-based chart statistic, which is robust to nonstationarity in predictor sequences and allows for the estimation of unknown pre-change parameters.
- Parametric Bootstrap: Dynamic control limits are constructed using a parametric bootstrap procedure, making the method computationally efficient.
Datasets
The paper uses simulated data for its extensive experiments. It also references a real-world dataset for a specific application:
- Multicenter Perioperative Outcomes Group (MPOG) registry: Used to illustrate the monitoring of an ML-based risk calculator for postoperative nausea and vomiting (PONV).
Key Insights and Ideas
- CMI is a fundamental challenge: The paper clearly articulates why confounding medical interventions are a significant hurdle for monitoring ML models in healthcare, impacting standard statistical assumptions.
- Conditional monitoring is key: Focusing on the outcomes of patients who didn't receive interventions (SOC-only data) is a viable strategy for valid inference, provided certain ignorability conditions hold.
- Clinician trust matters: The simulations reveal that "over-trust" in an ML model can significantly delay the detection of performance degradation, especially when shifts in risk are concentrated in high-risk patient groups.
- Model retraining is a double-edged sword: While continually retraining models can help them adapt to changing data distributions and maintain performance, it can also mask underlying issues if not monitored carefully. The monitoring procedure helps balance model updating with performance assurance.
- Risk scale is often preferred for monitoring: For certain models (like GBTs), monitoring shifts on the "risk scale" rather than the "logit scale" is recommended for better calibration and sensitivity to model misspecification.
Key Results
- False Alarm Rate Control: The proposed score-based CUSUM procedure effectively controls the false alarm rate, especially as the size of the non-contaminated dataset (used for initialization) increases.
- Detection of Calibration Decay: The method successfully detected calibration decay in an ML-based risk calculator for PONV during the COVID-19 pandemic.
- Impact of Clinician Trust: In simulations, increasing clinician trust led to substantially increased detection delay in high-risk scenarios, highlighting the need for robust monitoring systems that are not overly influenced by user behavior.
- Effectiveness of Retraining: Continual model retraining, when wrapped in a monitoring framework, can allow for steady improvement in model discrimination while maintaining calibration.
- Comparison to Bayesian Methods: The paper notes that while Bayesian methods can be powerful, their computational demands and sensitivity to model misspecification make them less practical for real-time, long-term monitoring compared to the proposed score-based CUSUM.
Limitations and Future Work
- Assumptions: The validity of the proposed method relies on the conditional exchangeability or time-constant selection bias assumptions. Violations of these assumptions can impact detection delay.
- Data Complexity: The current framework assumes clear delineation of treatment options and immediate observation of outcomes. Future work could explore more complex scenarios with delayed reporting or more nuanced treatment decisions.
- Extreme Treatment Propensities: The paper suggests exploring methods to address extreme treatment propensities, which can reduce the power of monitoring using SOC-only data.
- Model Misspecification: While the score-based CUSUM is shown to be somewhat robust to model misspecification, further improvements in robustness are desirable.
- Online Learning Integration: The paper highlights the potential of integrating online learning with monitoring for continually updated models, suggesting further research into optimizing this balance.
- Bayesian Monitoring Practicality: The authors point out limitations of current Bayesian monitoring tools, suggesting future work on developing more practical and efficient Bayesian approaches.
Summary updated