Reliable generation of privacy-preserving synthetic electronic health record time series via diffusion models

Muhang Tian, Bernie Chen, Allan Guo, Shiyi Jiang, Anru ZhangJAMIA(Journal of the American Medical Informatics Association) 2024 · DOI · OpenReview

TL;DR

This paper introduces TIMEDIFF, a novel diffusion model for generating realistic and privacy-preserving synthetic Electronic Health Record (EHR) time series data. By combining multinomial and Gaussian diffusion, TIMEDIFF effectively handles mixed data types and outperforms existing methods in data fidelity and privacy, paving the way for more accessible and robust medical data analysis.

What Problem Does This Paper Address?

Electronic Health Records (EHRs) are a treasure trove of patient data, crucial for advancing medical research, disease diagnosis, and treatment prediction. However, strict privacy regulations and the inherent risks of de-identification methods severely limit access to this valuable data. Existing de-identification techniques often compromise data utility or fail to prevent re-identification, while publicly available EHR datasets are scarce. This creates a significant bottleneck for developing and validating advanced machine learning models in healthcare. The paper aims to bridge this gap by generating synthetic EHR time series data that is both realistic and privacy-preserving, thereby enabling broader access for research and development.

Methods

The core of this work is TIMEDIFF, a novel diffusion probabilistic model (DDPM) specifically designed for EHR time series generation. Unlike previous approaches that often rely on Generative Adversarial Networks (GANs) which can suffer from training instability, TIMEDIFF leverages the power of diffusion models, known for their high-fidelity generation capabilities.

Key methodological contributions include:

  • Mixed Sequence Diffusion: TIMEDIFF employs a novel "mixed sequence diffusion" approach. It combines multinomial diffusion for discrete EHR variables (like ICD codes) and Gaussian diffusion for continuous variables (like vital signs). This allows for the direct and simultaneous generation of both data types, a significant improvement over methods that treat discrete data as continuous with post-processing.
  • Bidirectional Recurrent Neural Network (BRNN) Backbone: The model uses a time-conditional BRNN as its neural network backbone. This architecture is chosen for its efficiency and ability to capture temporal dynamics in sequential data, crucial for EHR time series.
  • Handling Missing Data: The method explicitly incorporates a missing indicator mask to represent and generate missing values, a common challenge in real-world EHR data. Missing values are imputed using the sample mean in the initial step of the diffusion process.

Datasets

The authors evaluated TIMEDIFF on four publicly available EHR datasets:

  • MIMIC-III
  • MIMIC-IV
  • eICU Collaborative Research Database

Additionally, to benchmark against general time series generation methods, they also used two non-EHR datasets:

  • Stocks
  • Energy

Key Insights and Ideas

  • Diffusion Models Excel at EHR Time Series: The study demonstrates that diffusion models, particularly when adapted for mixed data types, can achieve superior performance in generating realistic EHR time series compared to established GAN-based methods.
  • Multinomial Diffusion is Key for Discrete EHR Data: Directly modeling discrete EHR variables with multinomial diffusion, rather than relying on approximations or post-processing of continuous outputs, leads to more accurate and privacy-preserving synthetic data.
  • TIMEDIFF Balances Fidelity and Privacy: The proposed method not only generates data that closely resembles real EHR time series (high fidelity) but also exhibits a lower discriminative accuracy, indicating a reduced risk of re-identification (enhanced privacy).
  • Efficiency Gains: TIMEDIFF requires significantly less training time compared to GAN-based approaches, making it more practical for researchers with limited computational resources.

Key Results

  • Superior Data Fidelity: TIMEDIFF consistently outperformed nine baseline methods across all evaluated datasets in terms of discriminative and predictive scores, indicating more realistic synthetic data generation. For instance, it achieved a 95.4% lower mean discriminative score compared to DSPD-GP on EHR datasets.
  • Enhanced Privacy: The generated data from TIMEDIFF showed lower Nearest Neighbor Adversarial Accuracy Risk (NNAA) and Membership Inference Risk (MIR) scores compared to most baselines, suggesting better privacy preservation.
  • High Data Utility: Models trained on TIMEDIFF-generated data for in-hospital mortality prediction achieved AUC scores comparable to models trained on real data (TSTR scores close to TRTR scores). Furthermore, using synthetic data for augmentation (TSRTR) consistently improved model performance as the proportion of synthetic data increased.
  • Reduced Training Time: TIMEDIFF required substantially less training time than GAN-based methods, with training times ranging from 2.7 to 8.7 hours compared to 10.8 to 110 hours for baselines.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Privacy Guarantees: While TIMEDIFF offers improved privacy, it is not a replacement for official audits, and further investigation into its compatibility with formal privacy frameworks like differential privacy is warranted.
  • Interpretability: Developing methods to enhance the interpretability and explainability of TIMEDIFF, especially for subgroup analysis, is an area for future work.
  • Missing Value Imputation: While using sample mean imputation is computationally efficient, exploring more advanced imputation techniques could further improve the model's behavior.
  • Sparse and Irregular Data: Extending TIMEDIFF to model highly sparse and irregular temporal data (e.g., lab tests, medication events) is a promising avenue for future research.
  • Static and Infrequent Measurements: The paper notes that the current study primarily focuses on frequent measurements and suggests further experiments on static and infrequent measurements.

Summary updated

Abstract

Objective Electronic health records (EHRs) are rich sources of patient-level data, offering valuable resources for medical data analysis. However, privacy concerns often restrict access to EHRs, hindering downstream analysis. Current EHR deidentification methods are flawed and can lead to potential privacy leakage. Additionally, existing publicly available EHR databases are limited, preventing the advancement of medical research using EHR. This study aims to overcome these challenges by generating realistic and privacy-preserving synthetic EHRs time series efficiently. Materials and Methods We introduce a new method for generating diverse and realistic synthetic EHR time series data using denoizing diffusion probabilistic models. We conducted experiments on 6 databases: Medical Information Mart for Intensive Care III and IV, the eICU Collaborative Research Database (eICU), and non-EHR datasets on Stocks and Energy. We compared our proposed method with 8 existing methods. Results Our results demonstrate that our approach significantly outperforms all existing methods in terms of data fidelity while requiring less training effort. Additionally, data generated by our method yield a lower discriminative accuracy compared to other baseline methods, indicating the proposed method can generate data with less privacy risk. Discussion The proposed model utilizes a mixed diffusion process to generate realistic synthetic EHR samples that protect patient privacy. This method could be useful in tackling data availability issues in the field of healthcare by reducing barrier to EHR access and supporting research in machine learning for health. Conclusion The proposed diffusion model-based method can reliably and efficiently generate synthetic EHR time series, which facilitates the downstream medical data analysis. Our numerical results show the superiority of the proposed method over all other existing methods.
BibTeX
@article{tian2024reliable,
  author = {Muhang Tian and Bernie Chen and Allan Guo and Shiyi Jiang and Anru Zhang},
  title = {{Reliable generation of privacy-preserving synthetic electronic health record time series via diffusion models}},
  journal = {JAMIA(Journal of the American Medical Informatics Association)},
  year = {2024},
  doi = {10.1093/jamia/ocae229},
}
Figure