Mortality prediction among ICU inpatients based on MIMIC-III database results from the conditional medical generative adversarial network

Wei Yang, Hong Zou, Meng Wang, Qin Zhang, Shadan Li, Hongyin LiangHeliyon 2023 · DOI

TL;DR

This paper introduces a novel conditional medical generative adversarial network (c-med GAN) for predicting in-hospital mortality in ICU patients. The c-med GAN demonstrates superior predictive performance compared to traditional methods and standard machine learning models, even when trained on smaller datasets, highlighting its potential for data augmentation and improved clinical decision-making.

What Problem Does This Paper Address?

Predicting mortality in intensive care units (ICUs) is crucial for assessing patient severity, guiding treatment, and evaluating new interventions. While traditional scoring systems like SAPS II exist, they often struggle with accuracy due to strict assumptions about variable relationships and limitations in capturing complex patient data. Modern machine learning and deep learning approaches show promise, but their effectiveness is frequently hampered by the scarcity of high-quality, labeled clinical data. This scarcity stems from the time-consuming nature of data labeling, the cost of detailed medical tests, interoperability issues between health information systems, and strict privacy regulations. Generative Adversarial Networks (GANs) offer a potential solution by generating synthetic data, but their application to mortality prediction in ICUs, especially with labeled data, has been underexplored.

Methods

The researchers developed a conditional medical GAN (c-med GAN), an enhanced version of existing GAN architectures. This model combines the strengths of:

  • Medical GAN (medGAN): This variant uses an autoencoder to better handle discrete medical data (like diagnoses or medications) and generate realistic synthetic electronic health records.
  • Conditional GAN (CGAN): This allows the GAN to generate data conditioned on specific labels, meaning it can produce synthetic data that mimics real data with associated outcomes (in this case, mortality).

The c-med GAN's architecture involves an autoencoder to extract features and labels, a generator that creates synthetic data based on random noise and these labels, and a discriminator that distinguishes between real and generated data. Crucially, the generated synthetic data (at a 1:1 ratio with real data) was then used to train a Multilayer Perceptron (MLP) for mortality prediction. This approach aims to augment the real dataset, thereby improving the MLP's predictive power. The performance of the c-med GAN was compared against traditional methods like the SAPS II score and other machine learning models, including Support Vector Machines (SVM) and a standard MLP.

Datasets

The study utilized the publicly available MIMIC-III (Medical Information Mart for Intensive Care III) database (v1.4). This dataset contains de-identified clinical data for over 60,000 ICU patients. For this study, data from 38,597 adult patients (aged 15 and older) collected between 2001 and 2012 were used. The dataset was preprocessed to include 136 features and death labels. To evaluate the model's robustness, the data was artificially divided into three sizes: the full dataset, a small dataset (approximately 10% of patients), and a medium dataset (approximately 50% of patients).

Key Insights and Ideas

  • GANs for Labeled Clinical Data: The paper demonstrates that GANs, specifically the c-med GAN, can be effectively adapted to generate labeled synthetic medical data, addressing a key limitation of previous GAN applications in healthcare.
  • Data Augmentation for Mortality Prediction: By augmenting the real MIMIC-III dataset with synthetic data generated by the c-med GAN, the researchers were able to improve the predictive performance of a standard MLP model.
  • Robustness to Data Scarcity: The c-med GAN showed remarkable resilience to data reduction. While both MLP and c-med GAN performance degraded with smaller datasets, the c-med GAN exhibited less degradation and maintained superior performance over MLP, suggesting its utility in data-limited scenarios.
  • Beyond Basic GANs: The study highlights the necessity of advanced GAN variants like medGAN and CGAN to handle the complexities of discrete medical data and conditional generation, moving beyond the limitations of basic GANs.

Key Results

  • Superior Performance on Full Dataset: On the complete MIMIC-III dataset, the c-med GAN achieved the highest PR-AUC (0.532), ROC-AUC (0.910), and F1 score (0.551), outperforming SAPS II, SVM, and MLP.
  • Significant Improvement Over Baselines: The c-med GAN showed approximately a 15% improvement in ROC-AUC and an 80% improvement in PR-AUC compared to SAPS II and SVM. It also improved ROC-AUC by 4.6% and PR-AUC by 20.5% over the MLP.
  • Performance on Smaller Datasets: Even when the dataset size was reduced, the c-med GAN consistently outperformed the MLP. For instance, in the small dataset, the PR-AUC for c-med GAN was 0.425 compared to 0.176 for MLP, and the ROC-AUC was 0.796 compared to 0.726 for MLP. The degradation in performance for c-med GAN was notably less severe than for MLP.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Simplicity: The MIMIC-III dataset, while large, might contain aberrant data, especially near the end of life, which could affect generalizability.
  • Model Complexity: The study primarily used MLP for prediction after data augmentation. Future work could explore more complex deep learning models like CNNs, RNNs, or LSTMs to further assess the c-med GAN's potential.
  • Class Imbalance: Despite using metrics like PR-AUC and F1 score that are less sensitive to class imbalance, it might still have influenced prediction effects.
  • External Validation: The study lacked external validation, which is crucial for confirming the model's real-world applicability.

Future research will focus on addressing these limitations, including external validation, exploring more advanced prediction models, and investigating the c-med GAN's potential for generating data with different labels to enhance its naturalization performance for various risk prediction tasks.

Summary updated

Abstract

The prediction of in-hospital mortality based on the c-med GAN for ICU patients showed better performance than the baseline models. Despite some inadequacies, this model may have a promising future in clinical applications which will be explored by further research.
BibTeX
@inproceedings{yang2023mortality,
  author = {Wei Yang and Hong Zou and Meng Wang and Qin Zhang and Shadan Li and Hongyin Liang},
  title = {{Mortality prediction among ICU inpatients based on MIMIC-III database results from the conditional medical generative adversarial network}},
  booktitle = {Heliyon},
  year = {2023},
  doi = {10.1016/j.heliyon.2023.e13200},
}
Figure