SADI: Similarity-Aware Diffusion Model-Based Imputation for Incomplete Temporal EHR Data

Zongyu Dai, Emily J. Getzen, Qi Long

Introduction

Electronic Health Records (EHR) are essential for modern medical research, yet they are frequently characterized by significant missingness. This "holey" nature of EHR data, often resulting from irregular clinical visits or varying hospital protocols, can lead to biased models and healthcare inequities if not properly addressed.

Existing state-of-the-art (SOTA) imputation methods, such as Recurrent Neural Networks (RNNs) or the diffusion-based CSDI, typically focus on two dimensions: time (intra-patient temporal correlations) and features (inter-feature correlations). While effective in high-frequency data environments like Intensive Care Units (ICU), these methods struggle in "sparse" settings, such as outpatient or specialty care, where data points may be months apart and temporal correlations are weak. SADI (Similarity-Aware Diffusion model-based Imputation) addresses this gap by introducing a third dimension: patient similarity.

Methodology: The SADI Framework

SADI builds upon the foundation of Conditional Diffusion Models, which involve a forward process of adding noise to data and a reverse denoising process. The core innovation of SADI is making this process "similarity-aware" through a multi-step pipeline and a novel architectural component.

The four-step SADI pipeline: initial imputation, patient clustering, similarity-aware training, and reverse diffusion sampling.

The SADI framework follows a four-step pipeline:

  1. Initial Imputation: A standard method (e.g., CSDI) provides a rough, complete version of the dataset.
  2. Clustering: Patients are grouped into $K$ clusters using a time-series clustering algorithm (K-Spectral Centroid) to identify similar disease trajectories.
  3. Similarity-Aware Training: The denoising function ($\epsilon_\theta$) is trained on batches of patients from the same cluster.
  4. 3D Attention Mechanism: This is the central architectural innovation. Unlike previous 2D models, SADI employs a 3D attention block that captures dependencies across Time, Features, and Patients simultaneously.
Architecture of the 3D attention mechanism, featuring Temporal, Feature, and Patient Transformer layers.

By training on clusters of similar patients and utilizing 3D attention, the model learns to impute missing values for a specific patient by "borrowing" information from clinically similar profiles—treating the batch of similar patients as a collective knowledge base.

Experimental Evaluation

The researchers evaluated SADI on two real-world datasets representing different clinical environments:

  • CPAD (Critical Path For Alzheimer’s Disease): A sparse, long-term outpatient dataset with 37 features over 12 monthly points and 79% missingness.
  • PhysioNet Challenge 2012: A high-frequency ICU dataset with 48 hourly points and 80% missingness.

Performance was tested under three missing data mechanisms: Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR).

Key Results and Insights

SADI demonstrated significant improvements over existing SOTA models, particularly in sparse data settings.

  • Sparse Data Performance (CPAD): SADI significantly outperformed CSDI, reducing Mean Absolute Error (MAE) by approximately 8%, Root Mean Square Error (RMSE) by 6%, and Mean Relative Error (MRE) by 7% in MCAR settings, with similar gains across MAR and MNAR.
  • High-Frequency Data Performance (PhysioNet): In the ICU setting, SADI still achieved the best results, though the margin over CSDI was smaller (under 2%), as temporal correlations are already strong in such data.
  • Ablation Study: Removing the "Patient" transformer layer led to a significant performance drop, confirming that similarity-awareness is the primary driver of SADI's success.

The "aha moment" of this research is that when a patient's own history is sparse, the clinical trajectories of similar patients provide a powerful predictive signal. This context-awareness is most critical when temporal data is limited.

Limitations and Future Directions

The main limitation of SADI is its computational efficiency. The additional clustering step and 3D attention mechanism increase complexity, which may pose challenges for massive datasets. Future work aims to explore more efficient patient grouping methods, such as embedding-based clustering (e.g., Signal2vec), to enhance scalability for large-scale EHR systems without sacrificing the advantages of similarity-awareness.

Summary updated

Abstract

Missing values are prevalent in temporal electronic health records (EHR) data and are known to complicate data analysis and lead to biased results. The current state-of-the-art (SOTA) models for imputing missing values in EHR primarily leverage correlations across time points and across features, which perform well when data have strong correlation across time points, such as in ICU data where high-frequency time series data are collected. However, this is often insufficient for temporal EHR data from non-ICU settings (e.g., outpatient visits for primary care or specialty care), where data are collected at substantially sparser time points, resulting in much weaker correlation across time points. To address this methodological gap, we propose the Similarity-Aware Diffusion Model-Based Imputation (SADI), a novel imputation method that leverages the diffusion model and utilizes information across dependent variables. We apply SADI to impute incomplete temporal EHR data and propose a similarity-aware denoising function, which includes a self-attention mechanism to model the correlations between time points, features, and similar patients. To the best of our knowledge, this is the first time that the information of similar patients is directly used to construct imputation for incomplete temporal EHR data. Our extensive experiments on two datasets, the Critical Path For Alzheimer’s Disease (CPAD) data and the PhysioNet Challenge 2012 data, show that SADI outperforms the current SOTA under various missing data mechanisms, including missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR).
BibTeX
@inproceedings{dai2024sadi,
  author = {Zongyu Dai and Emily J. Getzen and Qi Long},
  title = {{SADI: Similarity-Aware Diffusion Model-Based Imputation for Incomplete Temporal EHR Data}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2024},
}
Figure