Impact of novel aggregation methods for flexible, time-sensitive EHR prediction without variable selection or cleaning

Jacob Deasy, Ari Ercole, Pietro LiòarXiv 2019 · OpenReview

Abstract

Dynamic assessment of patient status in the Intensive Care Unit (ICU) is of paramount importance for early alerting, decision support, and resource allocation. Extraction and cleaning of expert-selected clinical variables discards information and protracts collaborative efforts to introduce machine learning in medicine. We present improved aggregation methods for a flexible deep learning architecture which learns a joint representation of patient chart, lab, and output events. Our models outperform recent deep learning models for patient mortality classification using ICU timeseries, by embedding and aggregating all events with no pre-processing or variable selection. Our model achieves a strong performance of AUROC 0.878 at 48 hours on the MIMIC-III dataset while using 13,233 unique un-preprocessed variables in an interpretable manner via hourly softmax aggregation. This demonstrates how our method can be easily combined with existing electronic health record systems for automated, dynamic patient risk analysis.

Introduction

Building predictive models for the Intensive Care Unit (ICU) is traditionally a labor-intensive process. Most existing models rely on "expert-selected" variables—clinicians hand-pick a few dozen markers, such as heart rate or blood pH, and discard the rest. A systematic review found that the median number of variables used in such models is only 27. This approach suffers from significant information loss, as infrequent data points may contain subtle signals of decline. Furthermore, the manual cleaning required for every new hospital system hinders the scalability of real-time decision support tools.

This research addresses the challenge of building a flexible model capable of handling the messy, high-cardinality, and irregularly sampled nature of raw Electronic Health Record (EHR) data without manual intervention.

Methodology

The proposed architecture treats every event in the EHR as a potential signal through a three-stage pipeline:

  1. Tokenization and Discretization: Every event is converted into a discrete token. Categorical variables remain unchanged, while continuous variables are split into 20 percentile-based bins to handle outliers without manual clipping.
  2. Embedding and Aggregation: Each token is mapped to a 32-dimensional embedding space. To handle irregular sampling, an aggregation function squashes all embeddings within a one-hour window into a single fixed-size vector. The researchers evaluated four aggregation types: Summation, Average, Weighted Average, and Masked Softmax.
  3. Temporal Modeling: These hourly vectors are processed by a Gated Recurrent Unit (GRU) to capture the patient's trajectory over time, producing a mortality risk score at every time step.
Figure 1: The proposed pipeline for embedding and aggregating raw EHR events into hourly representations.

Data and Experimental Setup

The study utilized the MIMIC-III dataset, which contains over 330 million chart events. While traditional benchmark models use approximately 76 curated features, this flexible model utilizes 13,233 unique variables. The evaluation focused on a subset of 21,139 patient stays, comparing the model against traditional clinical scores and recent deep learning benchmarks.

Results and Discussion

The model demonstrated that "raw is better," outperforming benchmarks that relied on expert-curated features. Notably, Summation performed the best for raw prediction. Summation allows the model to "count" events; in an ICU, the frequency of measurements is often a proxy for clinician concern, serving as a strong signal of patient instability.

Figure 4: Performance comparison (AUROC) between traditional clinical scores, deep learning benchmarks, and various aggregation methods.

As shown in the results, the Summation variant achieved an AUROC of 0.878 at the 48-hour mark, significantly outperforming traditional clinical scores like SAPS II (0.705) and OASIS (0.663). Even at 12 hours, the model maintained a strong AUROC of ~0.81, proving its utility for early warning systems.

Beyond performance, the Masked Softmax aggregation provides interpretability by assigning weights to specific variables each hour. This allows clinicians to identify the specific drivers of a risk score jump, such as non-physiological signals (e.g., visits by a priest) that standard models would typically ignore.

Conclusion

While the model is highly flexible, it remains sensitive to "counting bias" related to hospital staffing levels rather than pure physiology. Future work will focus on the generalizability of these un-preprocessed embeddings across different EHR systems and further reducing the barriers to clinical deployment. By eliminating the need for variable selection, this framework paves the way for automated, dynamic patient risk analysis in diverse clinical settings.

Summary updated

Abstract

Dynamic assessment of patient status (e.g. by an automated, continuously updated assessment of outcome) in the Intensive Care Unit (ICU) is of paramount importance for early alerting, decision support and resource allocation. Extraction and cleaning of expert-selected clinical variables discards information and protracts collaborative efforts to introduce machine learning in medicine. We present improved aggregation methods for a flexible deep learning architecture which learns a joint representation of patient chart, lab and output events. Our models outperform recent deep learning models for patient mortality classification using ICU timeseries, by embedding and aggregating all events with no pre-processing or variable selection. Our model achieves a strong performance of AUROC 0.87 at 48 hours on the MIMIC-III dataset while using 13,233 unique un-preprocessed variables in an interpretable manner via hourly softmax aggregation. This demonstrates how our method can be easily combined with existing electronic health record systems for automated, dynamic patient risk analysis.
BibTeX
@misc{deasy2019impact,
  author = {Jacob Deasy and Ari Ercole and Pietro Liò},
  title = {{Impact of novel aggregation methods for flexible, time-sensitive EHR prediction without variable selection or cleaning}},
  year = {2019},
  eprint = {1909.08981},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.08981},
}
Figure