Abstract
Intensive Care Units (ICUs) generate vast amounts of temporal data, yet this data is frequently characterized by significant gaps due to sensor failures or clinical protocols. This paper presents a comprehensive benchmark of 15 imputation methods—ranging from simple statistical baselines to advanced deep learning and generative models—across three massive ICU datasets: MIMIC-IV, eICU, and HiRID. Utilizing the Yet Another ICU Benchmark (YAIB) framework, we evaluate these methods under various missingness mechanisms. Our findings indicate that while Attention-based models often achieve the lowest Mean Absolute Error (MAE), RNN-based architectures like GRU-D and generative models like CSDI provide greater robustness against large outlier errors (RMSE), especially in "Blackout" scenarios.
Introduction
The "holiness" of ICU data—missingness caused by patient movement, sensor disconnection, or infrequent sampling—poses a major hurdle for Machine Learning (ML) models predicting critical outcomes like sepsis or mortality. Common ad-hoc solutions, such as zero or mean imputation, often degrade predictive performance. Despite the proliferation of modern deep learning techniques, the field has lacked a standardized, large-scale comparison specifically tailored to the temporal nature of ICU vital signs. This study addresses this gap by benchmarking a diverse set of imputation algorithms within a unified framework.
Methodology
The researchers integrated 15 models into the Yet Another ICU Benchmark (YAIB) framework, categorized into four groups:
- Baselines: Deterministic methods (Mean, Median, Zero).
- Algorithmic: Statistical approaches (MICE, MissForest).
- Deep Learning (RNN & Attention): Sequence-aware models (GRU-D, BRITS, SAITS).
- Generative Models: Diffusion-based techniques (CSDI).
To simulate real-world data loss, four "amputation" mechanisms were employed: Missing Completely at Random (MCAR), Missing at Random (MAR), Missing Not at Random (MNAR), and Blackout (BO)—the total loss of all features for a specific duration.
Datasets and Scope
The study utilized three major open-access databases: MIMIC-IV, eICU, and HiRID. The analysis focused on six core temporal vital signs: heart rate, respiratory rate, oxygen saturation (O2Sat), and three blood pressure metrics. HiRID, with its higher sampling resolution, generally proved to be the easiest dataset to impute.
Performance Analysis
The benchmarking revealed that the optimal imputation strategy depends heavily on the evaluation metric and the nature of the missingness.
The MAE vs. RMSE Trade-off
A key insight is the divergence between average error and outlier sensitivity. Attention-based models consistently achieved the lowest MAE (e.g., ~0.14 for MCAR). However, they are prone to occasional large errors, resulting in extremely high Root Mean Square Error (RMSE) values. In contrast, GRU-D and CSDI maintained significantly lower RMSE, indicating more stable performance across all imputed points.
Resilience and Complexity
Interestingly, the "older" GRU-D architecture remained highly competitive, particularly in handling "Blackout" (BO) scenarios where all features are missing simultaneously. As missingness proportion increased from 30% to 70%, MAE roughly doubled across most models, highlighting the increasing difficulty of reconstruction as data density drops.
Conclusion
While deep learning models generally outperform simple baselines, there is no "silver bullet" for ICU data imputation. Attention models excel in average accuracy, while RNNs and Diffusion models offer better protection against extreme errors. Future research will focus on expanding the YAIB framework to include static variables and lab values, and most importantly, evaluating how these imputation methods impact downstream clinical prediction tasks.
Summary updated