Deep Learning-Adjusted Monitoring of In-Hospital Mortality after Liver Transplantation

Nikolaus Börner, Markus Bo Schoenberg, Benedikt Pöllmann, Philipp Pöschke, Christian Böhm, Dominik Koch, Moritz Drefs, Dionysios Koliogiannis, Joachim Andrassy, Jens Werner, Markus GubaJournal of Clinical Medicine 2024 · DOI

TL;DR

This paper introduces DL-CUSUM, the first deep learning-adjusted system for real-time monitoring of in-hospital mortality after liver transplantation. By combining a novel deep learning model for accurate mortality prediction with CUSUM charts, it offers a more sensitive and less biased approach to quality control in complex surgical procedures.

What Problem Does This Paper Address?

Surgical complications are a significant concern, and their late recognition can burden healthcare systems and negatively impact patient outcomes. Liver transplantation, in particular, is a high-risk procedure where timely identification of adverse events is crucial. Current quality monitoring methods often rely on retrospective manual analyses, which are slow, lack proper risk adjustment, and can lead to biases, potentially discouraging the transplantation of higher-risk patients. The challenge is to develop a system that can continuously and accurately monitor outcomes in real-time, accounting for the inherent variability in patient risk profiles.

Methods

The core of this work is the development of a Deep Learning-Adjusted Cumulative Sum (DL-CUSUM) system. This involves two main components:

  1. Deep Learning (DL) Model for Mortality Prediction: A novel DL algorithm was trained to predict in-hospital mortality. This model utilizes a large number of preoperatively available patient and donor variables. The development process involved data imputation using a custom MMCI algorithm to handle missing data, followed by training and cross-validation of a neural network. Hyperparameters like learning rate, batch size, and epochs were tuned.
  2. Risk-Adjusted CUSUM Analysis: The predictions from the DL model were integrated into a custom CUSUM analyzer. CUSUM charts are a sequential analysis technique that tracks the cumulative sum of deviations from an expected outcome. By using the DL model's risk-adjusted predictions, the CUSUM charts become more sensitive to genuine excess mortality while minimizing false alarms, even with a lower control limit. This approach aims to overcome the limitations of traditional CUSUM charts that lack robust risk adjustment.

Datasets

The study utilized a dataset comprising 1066 individuals who underwent liver transplantation between 2004 and 2019. This dataset included 66,092 preoperatively available data point variables from both recipients and donors. The data was split into a training set (n=477) and a test set (n=52) for model development and evaluation, respectively. The authors also developed a custom imputation algorithm (MMCI) for this dataset.

Key Insights and Ideas

  • DL for Enhanced Prediction: Deep learning models can achieve significantly higher accuracy in predicting complex outcomes like in-hospital mortality compared to traditional risk scores. This improved predictive power is crucial for effective real-time monitoring.
  • CUSUM's Potential in Medicine: While CUSUM charts are established in industrial quality control, their application in medicine, especially for complex procedures like transplantation, is underutilized. This paper demonstrates their value when combined with accurate risk adjustment.
  • The Need for Dynamic Risk Adjustment: Liver transplantation involves a highly variable patient population. Static risk scores are insufficient; a dynamic and adaptive approach like DL-CUSUM is necessary to accurately assess quality and identify deviations.
  • Addressing Data Imputation Challenges: Handling missing data in complex medical datasets is critical. The development of a specialized imputation algorithm (MMCI) was essential for building a robust DL model.

Key Results

  • DL Model Performance: The DL model achieved a high area under the precision-recall curve (PRAUC) of 0.857 for predicting in-hospital mortality. This significantly outperformed traditional risk scores.
  • Comparison with Traditional Scores:
    • D-MELD (Donor age multiplied by recipient MELD) AUC: 0.618
    • BAR (Balance of Risk) score AUC: 0.648
  • CUSUM Performance:
    • BAR-adjusted CUSUM: Control Limit (CL) of 2.3, Average Run Length (ARL) of 326.31.
  • D-MELD adjusted CUSUM: CL of 2.4, ARL of 303.29.
  • DL-adjusted CUSUM: CL of 1.8, ARL of 332.67. The lower CL with a comparable ARL indicates higher sensitivity to detect deviations.
  • Qualitative Finding: The DL-CUSUM system successfully identified no false or real excess mortality within the test data set, suggesting its ability to provide accurate real-time monitoring.

Limitations and Future Work

  • Sample Size: The authors acknowledge that a larger sample size would be desirable, particularly for the test data set (n=52), which showed significant differences in some variables compared to the training data.
  • Generalizability: The study was conducted at a single large transplant center, limiting immediate generalizability to all patient populations. However, the authors believe the pipeline of methods can be adapted.
  • "Black Box" Nature of DL: Deep learning models are often considered "black boxes," making it difficult to fully interpret the decision-making process. While efforts were made to exclude discriminatory variables, this inherent opacity remains a limitation.
  • Future Work:
    • The authors propose that similar DL-CUSUM systems could be applied to monitor other surgical interventions and outcomes beyond mortality.
  • They suggest the development of a graphical user interface (already created and available online) to facilitate the adoption of such systems.
  • Further research into multi-marker approaches and frailty markers could enhance predictive models.
  • Addressing data privacy concerns and exploring secure anonymization techniques for real-time monitoring is crucial for clinical implementation.

Summary updated

Abstract

<b>Background</b>: Surgeries represent a mainstay of medical care globally. Patterns of complications are frequently recognized late and place a considerable burden on health care systems. The aim was to develop and test the first deep learning-adjusted CUSUM program (DL-CUSUM) to predict and monitor in-hospital mortality in real time after liver transplantation. <b>Methods</b>: Data from 1066 individuals with 66,092 preoperatively available data point variables from 2004 to 2019 were included. DL-CUSUM is an application to predict in-hospital mortality. The area under the curve for risk adjustment with Model of End-stage Liver Disease (D-MELD), Balance of Risk (BAR) score, and deep learning (DL), as well as the ARL (average run length) and control limit (CL) for an in-control process over 5 years, were calculated. <b>Results</b>: D-MELD AUC was 0.618, BAR AUC was 0.648 and DL model AUC was 0.857. CL with BAR adjustment was 2.3 with an ARL of 326.31. D-MELD reached an ARL of 303.29 with a CL of 2.4. DL prediction resulted in a CL of 1.8 to reach an ARL of 332.67. <b>Conclusions</b>: This work introduces the first use of an automated DL-CUSUM system to monitor postoperative in-hospital mortality after liver transplantation. It allows for the real-time risk-adjusted monitoring of process quality.
BibTeX
@article{borner2024deep,
  author = {Nikolaus Börner and Markus Bo Schoenberg and Benedikt Pöllmann and Philipp Pöschke and Christian Böhm and Dominik Koch and Moritz Drefs and Dionysios Koliogiannis and Joachim Andrassy and Jens Werner and Markus Guba},
  title = {{Deep Learning-Adjusted Monitoring of In-Hospital Mortality after Liver Transplantation}},
  journal = {Journal of Clinical Medicine},
  year = {2024},
  doi = {10.3390/jcm13206046},
}
Figure