Introduction
Electronic Health Records (EHR) are essential for modern medical research, yet they are frequently characterized by significant missingness. This "holey" nature of EHR data, often resulting from irregular clinical visits or varying hospital protocols, can lead to biased models and healthcare inequities if not properly addressed.
Existing state-of-the-art (SOTA) imputation methods, such as Recurrent Neural Networks (RNNs) or the diffusion-based CSDI, typically focus on two dimensions: time (intra-patient temporal correlations) and features (inter-feature correlations). While effective in high-frequency data environments like Intensive Care Units (ICU), these methods struggle in "sparse" settings, such as outpatient or specialty care, where data points may be months apart and temporal correlations are weak. SADI (Similarity-Aware Diffusion model-based Imputation) addresses this gap by introducing a third dimension: patient similarity.
Methodology: The SADI Framework
SADI builds upon the foundation of Conditional Diffusion Models, which involve a forward process of adding noise to data and a reverse denoising process. The core innovation of SADI is making this process "similarity-aware" through a multi-step pipeline and a novel architectural component.
The SADI framework follows a four-step pipeline:
- Initial Imputation: A standard method (e.g., CSDI) provides a rough, complete version of the dataset.
- Clustering: Patients are grouped into $K$ clusters using a time-series clustering algorithm (K-Spectral Centroid) to identify similar disease trajectories.
- Similarity-Aware Training: The denoising function ($\epsilon_\theta$) is trained on batches of patients from the same cluster.
- 3D Attention Mechanism: This is the central architectural innovation. Unlike previous 2D models, SADI employs a 3D attention block that captures dependencies across Time, Features, and Patients simultaneously.
By training on clusters of similar patients and utilizing 3D attention, the model learns to impute missing values for a specific patient by "borrowing" information from clinically similar profiles—treating the batch of similar patients as a collective knowledge base.
Experimental Evaluation
The researchers evaluated SADI on two real-world datasets representing different clinical environments:
- CPAD (Critical Path For Alzheimer’s Disease): A sparse, long-term outpatient dataset with 37 features over 12 monthly points and 79% missingness.
- PhysioNet Challenge 2012: A high-frequency ICU dataset with 48 hourly points and 80% missingness.
Performance was tested under three missing data mechanisms: Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR).
Key Results and Insights
SADI demonstrated significant improvements over existing SOTA models, particularly in sparse data settings.
- Sparse Data Performance (CPAD): SADI significantly outperformed CSDI, reducing Mean Absolute Error (MAE) by approximately 8%, Root Mean Square Error (RMSE) by 6%, and Mean Relative Error (MRE) by 7% in MCAR settings, with similar gains across MAR and MNAR.
- High-Frequency Data Performance (PhysioNet): In the ICU setting, SADI still achieved the best results, though the margin over CSDI was smaller (under 2%), as temporal correlations are already strong in such data.
- Ablation Study: Removing the "Patient" transformer layer led to a significant performance drop, confirming that similarity-awareness is the primary driver of SADI's success.
The "aha moment" of this research is that when a patient's own history is sparse, the clinical trajectories of similar patients provide a powerful predictive signal. This context-awareness is most critical when temporal data is limited.
Limitations and Future Directions
The main limitation of SADI is its computational efficiency. The additional clustering step and 3D attention mechanism increase complexity, which may pose challenges for massive datasets. Future work aims to explore more efficient patient grouping methods, such as embedding-based clustering (e.g., Signal2vec), to enhance scalability for large-scale EHR systems without sacrificing the advantages of similarity-awareness.
Summary updated