Abstract
Multivariate time series data, particularly in clinical settings, often suffer from significant missing values. Traditional approaches either discard incomplete records or use simple imputation, ignoring the potentially predictive information contained in the patterns of missingness itself. This paper presents GRU-D, a novel recurrent neural network architecture that incorporates trainable decay mechanisms to model the influence of missing data over time. By utilizing masking and time-interval representations, GRU-D effectively captures "informative missingness," leading to state-of-the-art performance in mortality and diagnosis prediction tasks on real-world datasets like MIMIC-III.
Introduction
In healthcare, multivariate time series data—such as vital signs and laboratory results—are characterized by irregular sampling and significant missingness. For instance, while heart rate might be recorded continuously, specialized blood tests are performed sporadically. Standard deep learning models typically handle this through mean or forward-fill imputation. However, these methods fail to account for the fact that the absence of a measurement often reflects a clinical decision, which is itself a proxy for the patient's state. This phenomenon, known as informative missingness, suggests that the timing and frequency of observations are critical features for predictive modeling.
The GRU-D Architecture
The proposed GRU-D model extends the standard Gated Recurrent Unit (GRU) by introducing a decay mechanism that accounts for the time elapsed since the last observation. The model incorporates two primary auxiliary inputs: a masking vector ($m_t$) indicating the presence of observations at time $t$, and a time interval vector ($\delta_t$) representing the duration since the last valid measurement for each variable.
As shown in the architecture above, GRU-D employs a trainable decay factor, $\gamma$, which is applied to both the input values and the hidden state. Input decay allows the model to transition missing values toward the empirical mean over time, while hidden state decay captures the diminishing influence of historical information in the absence of new data. Unlike fixed imputation, these decay rates are learned per-variable, allowing the network to adapt to the different temporal dynamics of various physiological signals.
Informative Missingness in Clinical Data
A key insight of this work is that missingness is not merely a data quality issue but a valuable signal. The authors demonstrate a clear correlation between missing rates and clinical outcomes. For example, variables like the Glasgow Coma Scale (GCS) or Albumin levels exhibit specific missingness patterns that correlate with mortality.
By explicitly modeling these patterns through the decay and masking mechanisms, GRU-D can leverage the "missingness" signal to improve predictive accuracy. This allows the model to recognize that some signals (like heart rate) change much faster than others (like lab results).
Experimental Results
The model was rigorously evaluated on the MIMIC-III (99 variables from 19,000 admissions) and PhysioNet Challenge 2012 (33 variables from 4,000 records) datasets.
- Mortality Prediction (AUC): On the MIMIC-III dataset, GRU-D achieved an AUC of 0.8527, significantly outperforming standard GRU models using mean imputation (0.8192) and even strong non-RNN baselines like Random Forest (0.8294).
- Multi-task Performance: GRU-D consistently outperformed other RNN variants in predicting 20 different ICD-9 diagnosis categories.
- Early Prediction: The model is particularly effective for "online" prediction, achieving high accuracy much earlier in a patient's stay compared to models relying on simple imputation.
Conclusion
GRU-D represents a significant advancement in processing incomplete multivariate time series. By treating missingness as a first-class feature and introducing trainable decay, the model effectively bridges the gap between raw, irregular clinical data and deep learning architectures. Future work may involve extending these decay mechanisms to other sequential models like Transformers and exploring the theoretical properties of the learned decay rates across different missing data distributions.
Summary updated