TL;DR
This paper introduces a novel, end-to-end pipeline for generating "digital twin" synthetic datasets from complex electronic health records (EHR) and wearable data. By offering tunable obfuscation levels, this approach aims to maximize the utility of biomedical data for research while significantly minimizing the risk of re-identifying individuals.
What Problem Does This Paper Address?
Sharing sensitive biomedical data, such as electronic health records (EHR) and wearable device data, is crucial for advancing medical research, improving clinical predictivity, and optimizing resource utilization. However, the inherent sensitivity of this data, coupled with privacy regulations, makes it difficult to access and use. Traditional methods for anonymizing data often lead to a significant loss of data utility, hindering their effectiveness for research. This paper tackles the challenge of creating realistic synthetic data (digital twins) that can serve as a substitute for real data, thereby enabling broader data sharing and collaboration without compromising individual privacy or the integrity of the data for analysis.
Methods
The core of this work lies in the development and evaluation of a titratable pipeline for synthetic data generation. The authors employed two primary methods:
- DataSifter: This statistical obfuscator offers fine-grained control over de-identification levels, allowing for "small," "medium," and "large" obfuscation. It's highlighted for its efficiency, especially with longitudinal data, and its adaptability in balancing privacy and utility.
- Synthetic Data Vault (SDV): This package was used to generate synthetic data using two specific models: CTGAN (Conditional Tabular Generative Adversarial Network) and Gaussian Copula.
The pipeline integrates these methods to generate five distinct digital twin datasets. These synthetic datasets were then rigorously evaluated using a comprehensive suite of metrics for:
- Utility: Statistical fidelity (e.g., correlation structure, univariate summaries) and machine learning performance.
- Privacy: Re-identification risk and detection likelihood.
The study also introduces a framework for evaluating synthetic data, emphasizing the importance of both privacy and utility metrics.
Datasets
The study utilized a dataset comprising:
- Electronic Health Records (EHR): Including laboratory data, ICD codes, and patient demographics.
- Wearable Device Data: Specifically, Apple Watch data from 3029 participants, capturing metrics like active energy burned (AEB) and physical activity.
This dataset represents a complex, longitudinal, and heterogeneous collection of sensitive health information. The authors generated five synthetic digital twin datasets derived from this original data, each with varying degrees of obfuscation.
Key Insights and Ideas
- Tunable Obfuscation is Key: The DataSifter's ability to offer multiple, distinct levels of obfuscation (small, medium, large) is a significant advantage, allowing data governors to tailor synthetic data to specific privacy needs without a complete loss of analytical value.
- Longitudinal Data is Tricky but Achievable: Generating realistic synthetic longitudinal data is a notable challenge. DataSifter demonstrated superior performance in handling this compared to SDV's limited sequence generation method.
- Privacy-Utility Trade-off is Real, But Manageable: Higher levels of obfuscation generally lead to stronger privacy protection but can impact data utility. The study shows that it's possible to achieve a good balance, particularly with DataSifter.
- Homogeneous Obfuscation is Preferable: The study suggests that a more uniform obfuscation across all variables is generally better than methods that disproportionately affect specific features, unless a particular outcome is targeted for obfuscation.
- DataSifter Outperforms SDV: In head-to-head comparisons, DataSifter was found to be faster and more effective at preserving statistical properties and relationships, especially for longitudinal data, compared to SDV.
Key Results
- The highest-obfuscation DataSifter twin (DS_Large) provided the strongest privacy protection (0.83 on a privacy score) while retaining significant statistical and predictive signals (83.1% confidence interval overlap in regression models).
- DataSifter was significantly faster than SDV, generating longitudinal data in under a second, while SDV was 10-20 times slower.
- While higher obfuscation levels in DataSifter led to a decrease in machine learning task performance, overall utility was generally preserved.
- The study found that the CTGAN method within SDV dramatically lacked utility when predicting a key outcome (AVG_AEB_day), with estimates not significantly different from zero.
- Standardized differences between synthetic and original data showed that DataSifter_Large had the smallest median difference, indicating better fidelity.
Limitations and Future Work
- Single Cohort/Institution: The study was conducted on data from a single institution, and future work should validate these methods on multi-institutional datasets and with unstructured or multi-modal data.
- Computational Inefficiencies in SDV: The paper acknowledges potential computational inefficiencies in SDV's longitudinal synthesis, suggesting algorithmic refinements might be available.
- Privacy Law Nuances: The authors note that while their methods aim to minimize re-identification risk, synthetic data can still fall under privacy laws like GDPR and HIPAA, requiring further work to understand safe sharing practices.
- ICD Code Recasting: The recasting of ICD codes as integers for DataSifter processing may introduce bias and reduce data granularity.
- Future Directions: The authors suggest exploring Bayesian optimization for hyperparameter tuning to further improve ML performance and utility metrics. They also propose investigating the balance between privacy and utility on unstructured or multi-modal datasets and exploring unsupervised ML techniques.
Summary updated