Introduction
Electronic Health Records (EHRs) are a cornerstone of modern biomedical research, providing deep insights into patient outcomes and clinical pathways. However, the utility of this data is often restricted by stringent privacy regulations and the ethical necessity of patient consent. Synthetic EHR data—computationally generated records that mimic the statistical properties of real patient data without containing information from actual individuals—has emerged as a promising solution to these accessibility challenges.
Despite the proliferation of generative models, the field remains fragmented. Researchers often face a "Wild West" of methodologies with little standardization in evaluation metrics or cross-dataset validation. This paper addresses these gaps by providing a comprehensive scoping review of existing methods and introducing SynthEHRella, an open-source Python toolkit designed for the rigorous benchmarking of synthetic EHR generation.
Methodological Landscape
The authors conducted a scoping review of 48 studies, categorizing the current state of the art into five primary methodological families:
- Rule-based Models: These utilize predefined clinical algorithms and census data to simulate patient lifecycles (e.g., Synthea).
- Generative Adversarial Networks (GANs): These models employ a competitive framework where a generator attempts to create realistic data while a discriminator attempts to distinguish it from real records (e.g., MedGAN, CorGAN).
- Variational Autoencoders (VAEs): These models compress EHR data into a lower-dimensional latent space before reconstructing it to generate new samples.
- Transformer-based Models: Leveraging attention mechanisms similar to Large Language Models (LLMs), these are increasingly used to capture complex dependencies in clinical sequences (e.g., PromptEHR).
- Diffusion-based Models: A recent innovation in generative AI, these models create data by iteratively reversing a noise process (e.g., EHRDiff).
The SynthEHRella Benchmarking Framework
To evaluate these methods, the authors developed SynthEHRella, a unified evaluation framework. The benchmarking process utilized the gold-standard open-source EHR datasets, MIMIC-III and MIMIC-IV, which contain data from hundreds of thousands of ICU admissions. To ensure clinical relevance, raw ICD diagnostic codes were mapped to the PhecodeX system, allowing for more meaningful phenotype analysis.
A critical component of this study was the evaluation of transportability. By training models on MIMIC-III (2001–2012) and evaluating them on MIMIC-IV (2008–2019), the researchers tested how well synthetic data handles the distributional shifts that naturally occur in healthcare settings over time.
Key Insights and Performance Metrics
The benchmarking process evaluated models across three primary dimensions: Fidelity, Utility, and Privacy.
Fidelity and Association
While simple baselines like Prevalence-based Random (PBR) models can accurately match the frequency of individual diseases, they fail to capture the complex correlations between them. In contrast, GAN-based methods, particularly CorGAN, demonstrated superior fidelity by preserving the intricate associations between variables (e.g., the co-occurrence of obesity and type 2 diabetes).
Downstream Utility
In "Train on Synthetic, Test on Real" (TSTR) scenarios—where a predictive model is trained on synthetic data and validated on real patient records—MedGAN emerged as the leader. It showed the smallest performance drop (only a 0.063 decrease in AUC), suggesting that its synthetic output is highly effective for training machine learning models intended for clinical application.
Efficiency and Privacy
The study highlighted a significant trade-off between privacy and utility. Rule-based methods like Synthea offer high privacy because they do not "memorize" real patient patterns, but they often struggle to replicate the messy correlations found in real-world clinical data. Furthermore, the computational efficiency varied wildly: GANs were remarkably fast, with MedGAN generating samples in fractions of a second, while some rule-based simulators took several hours to produce comparable cohorts.
Conclusion and Future Directions
The research concludes that while GANs currently offer the most balanced performance for cross-sectional phenotype data, the field is rapidly evolving. Newer architectures like Diffusion and Transformers show significant promise for longitudinal data—tracking patient trajectories over time—which remains a complex challenge for current generators.
Future work must address the limitations of current benchmarks, including the handling of missing data, the integration of multimodal sources (such as clinical notes and medical imaging), and the development of conditional generation methods to ensure fairness and representation across diverse demographic groups. By providing SynthEHRella to the community, the authors have established a foundation for more transparent and reproducible synthetic EHR research.
Summary updated