Oversampling techniques for predicting COVID-19 patient length of stay

Zachariah Farahany, Jiawei Wu, K M Sajjadul Islam, Praveen Madiraju

Predicting how long a patient will remain in the hospital—Length of Stay (LOS)—is a holy grail for hospital administrators. During a pandemic like COVID-19, knowing which patients will occupy a bed for weeks versus days is the difference between a functional healthcare system and a collapsed one.

A recent paper by researchers at Marquette University, titled "Oversampling techniques for predicting COVID-19 patient length of stay," explores how to use deep learning to solve this prediction problem, specifically focusing on the "imbalance" challenge: the fact that most patients have short stays, making the few long-stay "severe" cases hard for AI to learn.

TL;DR

The researchers developed an Artificial Neural Network (ANN) to predict whether COVID-19 patients would stay in the hospital for more than seven days using Electronic Health Records (EHR). By comparing various oversampling techniques and using Bayesian optimization for hyperparameter tuning, they achieved an F1 score of 85.79%, finding that while oversampling is a standard fix for imbalanced data, the raw dataset often yielded the most robust results in high-dimensional clinical settings.

What Problem Does This Paper Address?

The core challenge is data imbalance. In clinical datasets for COVID-19, the vast majority of patients have a relatively short LOS (less than 30 days, and often less than 7). If a machine learning model is trained on this data without adjustments, it becomes "lazy"—it can achieve 90% accuracy simply by guessing that everyone will have a short stay. However, the 10% of patients who stay a long time are the ones who strain hospital resources the most.

The authors sought to determine which data-balancing techniques (like SMOTE or random oversampling) combined with deep learning could most accurately identify these high-severity, long-stay patients.

Methods

The team employed a multi-stage pipeline:

  1. Data Consolidation: They transformed fragmented EHR entries (multiple diagnoses or vitals per patient) into a single, high-dimensional row per patient using pivot tables.
  2. Preprocessing: They handled missing data via K-Nearest Neighbors (KNN) imputation and removed columns with more than 50% missing values.
  3. The Model: An Artificial Neural Network (ANN) was the primary architecture.
  4. Optimization: Instead of manual "grid searching" for the best settings, they used Bayesian Optimization. This treats the model's hyperparameters (like the number of layers, dropout rate, and learning rate) as a search problem, "zooming in" on configurations that yield the best F1 score.
  5. Oversampling Strategies: They tested five distinct training sets:
  6. Raw data.
  7. Raw data with weighted loss functions (penalizing errors on long-stay patients more heavily).
  8. Random Oversampling (ROS).
  9. Random Undersampling (RUS).
  10. SMOTE-NC: A variation of the Synthetic Minority Over-sampling Technique designed for datasets with both numerical and categorical variables.

Datasets

The study used de-identified EHR data from Froedtert Hospital in Wauwatosa, Wisconsin.

  • Scope: 11,235 patients recorded between June 2016 and June 2021.
  • Features: A massive 25,008 columns covering demographics, vitals, medications, and ICD-11 diagnosis codes.
  • Target: A binary classification of LOS: "Not Severe" (< 7 days) vs. "Severe" (> 7 days).

Key Insights and Ideas

  • The "Raw" Surprise: Contrary to some machine learning conventions, Model #1 (trained on raw, imbalanced data) performed the best across almost every metric, including F1 score and AUC. This suggests that in extremely high-dimensional spaces (25k+ features), synthetic oversampling might introduce noise that confuses the model rather than helping it.
  • The Precision-Recall Trade-off: Model #4 (Undersampling) achieved the highest precision (93.75%), meaning when it predicted a long stay, it was almost always right. However, its recall was abysmal. In a hospital, this is dangerous—it means the model misses many severe cases.
  • Magnesium as a Proxy: When analyzing feature importance via mutual information, magnesium levels appeared three times in the top 20 features. This suggests that electrolyte imbalances are a significant "hidden" biomarker for COVID-19 severity that the ANN was able to exploit.
  • IV Infusions as Predictors: The presence of intravenous infusions for prophylactic reasons was a top predictor. Intuitively, if a clinician starts an IV, they already suspect a more serious, long-term trajectory for the patient.

Key Results

The researchers evaluated five models based on their training data strategy. Model #1 (Raw Data) was the clear winner for general utility:

Metric Model #1 (Raw) Model #4 (Undersampled) Model #5 (SMOTE-NC)
F1 Score 85.79% 69.62% 79.15%
Accuracy 95.50% 87.18% 92.57%
Precision 86.64% 93.75% 90.05%
Recall 84.95% 55.36% 70.60%
AUC 91.23% 77.02% 84.33%

Model #3 (Random Oversampling) was deemed "essentially useless," with an F1 score of only 27.26%, likely due to massive overfitting on the duplicated minority samples.

Limitations and Future Work

  • Homogeneity: The dataset was primarily composed of older, white, English-speaking patients from a single hospital system in Wisconsin. The authors admit the model might underperform for non-white or non-English speaking populations.
  • Dimensionality: With 25,008 features and only ~7,000 training rows, the model is at high risk of the "curse of dimensionality." Future work could involve more aggressive feature selection or dimensionality reduction.
  • Generative Models: The authors suggest that Generative Adversarial Networks (GANs) could be used in the future to create more realistic synthetic patient data than SMOTE, potentially improving the performance of oversampled models.

Summary updated

Abstract

COVID-19 is a respiratory disease that caused a global pandemic in 2019. It is highly infectious and has the following symptoms: fever or chills, cough, shortness of breath, fatigue, muscle or body aches, headache, the new loss of taste or smell, sore throat, congestion or runny nose, nausea or vomiting, and diarrhea. These symptoms vary in severity; some people with many risk factors have been known to have lengthy hospital stays or die from the disease. In this paper, we analyze patients’ electronic health records (EHR) to predict the severity of their COVID-19 infection using the length of stay (LOS) as our measurement of severity. This is an imbalanced classification problem, as many people have a shorter LOS rather than a longer one. To combat this problem, we synthetically create alternate oversampled training data sets. Once we have this oversampled data, we run it through an Artificial Neural Network (ANN), which during training has its hyperparameters tuned by using bayesian optimization. We select the model with the best F1 score and then evaluate it and discuss it.
BibTeX
@inproceedings{farahany2022oversampling,
  author = {Zachariah Farahany and Jiawei Wu and K M Sajjadul Islam and Praveen Madiraju},
  title = {{Oversampling techniques for predicting COVID-19 patient length of stay}},
  booktitle = {2022},
  year = {2022},
  doi = {10.1109/bigdata55660.2022.10020253},
}
Figure