Utilizing imbalanced electronic health records to predict acute kidney injury by ensemble learning and time series model

Yuan Wang, Yake Wei, Hao Yang, Jingwei Li, Yubo Zhou, Qin WuBMC Medical Informatics Decis. Mak 2020 · DOI

TL;DR

This paper introduces the Ensemble Time Series Model (ETSM), a novel approach for predicting Acute Kidney Injury (AKI) using electronic health records. By creatively incorporating drug combinations as implicit indicators alongside traditional vital signs and lab results, ETSM achieves competitive prediction accuracy while offering improved interpretability, crucial for timely clinical intervention.

What Problem Does This Paper Address?

Acute Kidney Injury (AKI) is a serious complication in Intensive Care Units (ICUs) that significantly increases morbidity and mortality. Early prediction of AKI is critical for effective patient management and timely medical intervention. However, predicting AKI using electronic health records (EHRs) presents several challenges:

  • Data Sparsity and High Dimensionality: EHRs are often incomplete, with missing values, and contain a vast number of variables, making it difficult for models to learn effectively.
  • Imbalanced Datasets: AKI cases are typically a minority within the patient population, which can bias machine learning models towards the majority class.
  • Temporal Correlations: AKI development is influenced by the dynamic interplay of physiological indicators and, importantly, the cumulative effects of drug combinations over time. Existing models often fail to adequately capture these temporal dependencies and the impact of medication regimens.

This paper aims to address these challenges by developing a model that can effectively leverage temporal data, incorporate drug combination information, and handle imbalanced datasets for accurate and interpretable AKI prediction.

Methods

The core of this work is the Ensemble Time Series Model (ETSM), which combines an ensemble learning algorithm (XGBoost) with a time-series approach for feature engineering.

  • Feature Engineering: ETSM creates two main groups of features:
  • Explicit Indicator Group: This group uses standard patient data like vital signs and laboratory results, reflecting their immediate physiological state. These are captured at the ICU admission day and the predictive point.
  • Implicit Indicator Group: This is the novel contribution. It transforms medication information into a temporal sequence of "drug combinations" (sets of drugs taken on a given day). This sequence is then processed using a method inspired by TF-IDF (Term Frequency-Inverse Document Frequency), termed Indicator Frequency and Inverse Cohort Frequency (IFICF). IFICF quantifies the importance of a drug combination within a patient's sequence and across the entire cohort, effectively capturing the cumulative effect of medications.
  • Prediction Model: XGBoost is chosen as the classification model due to its scalability, tolerance for missing values, and strong performance on sparse and high-dimensional data.
  • Handling Imbalanced Data: The researchers explored and compared random undersampling, random oversampling, and cost-sensitive XGBoost. They found that random undersampling on the derivation set yielded the best results for mitigating class imbalance.
  • Time Series Aspect: The model explicitly models the temporal nature of both physiological data and medication intake, allowing it to capture evolving patient conditions and drug effects.

Datasets

The ETSM model was evaluated on two distinct ICU patient datasets:

  • ICUC Dataset: An internal dataset from West China Hospital, China.
  • MIMIC III Dataset: A publicly available, large-scale ICU dataset from the United States.

The use of both an internal and an external dataset helps validate the model's scalability and generalizability.

Key Insights and Ideas

  • Drug Combinations as Implicit Indicators: The most significant insight is that the combination of drugs, rather than individual drugs, plays a crucial role in predicting AKI. This moves beyond simply listing medications to understanding their synergistic or antagonistic effects over time.
  • IFICF for Medication Feature Extraction: The IFICF method provides a robust way to transform complex temporal medication data into meaningful features that capture drug importance and frequency.
  • Interpretability through Ensemble Trees: By using XGBoost (an ensemble of decision trees), the model offers better interpretability compared to "black-box" deep learning models. The importance of specific drug combinations can be analyzed, providing actionable insights for clinicians.
  • Optimal Data Balancing: The study found that a specific ratio of positive (AKI) to negative (non-AKI) samples (around 1:1 or 1:2) is crucial for optimal model performance, highlighting the importance of careful data preprocessing for imbalanced datasets.

Key Results

  • High Predictive Performance: ETSM achieved strong AUC scores on both datasets:
  • ICUC: 0.81 (24 hours ahead), 0.78 (48 hours ahead)
    • MIMIC III: 0.95 (24 hours ahead), 0.95 (48 hours ahead)
  • Medication Information Boosts Performance: An ablation study showed that including the implicit indicator group (drug combinations) significantly improved model performance. For instance, on the ICUC dataset, AUC increased from 0.74 to 0.81 for 24-hour predictions when medication information was included.
  • Competitive Comparison: ETSM outperformed several baseline models, including Naive Bayes, k-Nearest Neighbor, AdaBoost, and Random Forest, on various metrics like AUC, sensitivity, F1-score, and Average Precision.
  • Interpretability: The model's ability to identify key drug combinations associated with AKI risk provides valuable clinical guidance.

Limitations and Future Work

  • Generalizability: While validated on two datasets, the model's performance in non-ICU hospital departments needs further validation.
  • Detailed Drug Effects: The precise mechanisms by which drug combinations influence AKI incidence require more in-depth research.
  • Future Directions: The authors plan to further explore the correlation between drug combinations and AKI incidence. They also suggest that the time-series modeling method is scalable and can be applied to other clinical prediction tasks.

Summary updated

BibTeX
@article{wang2020utilizing,
  author = {Yuan Wang and Yake Wei and Hao Yang and Jingwei Li and Yubo Zhou and Qin Wu},
  title = {{Utilizing imbalanced electronic health records to predict acute kidney injury by ensemble learning and time series model}},
  journal = {BMC Medical Informatics Decis. Mak.},
  year = {2020},
  doi = {10.1186/s12911-020-01245-4},
}
Figure