Abstract
With progressive digitalization of healthcare systems worldwide, large-scale collection of electronic health records (EHRs) has become commonplace. However, an extensible framework for comprehensive exploratory analysis that accounts for data heterogeneity is missing. Here, we introduce ehrapy, a modular open-source Python framework designed for exploratory end-to-end analysis of heterogeneous epidemiology and electronic health record data. Ehrapy incorporates a series of analytical steps, from data extraction and quality control to the generation of low-dimensional representations. Complemented by rich statistical modules, ehrapy facilitates associating patients with disease states, differential comparison between patient clusters, survival analysis, trajectory inference, causal inference, and more. Leveraging ontologies, ehrapy further enables data sharing and training EHR deep learning models paving the way for foundational models in biomedical research. We demonstrated ehrapy’s features in five distinct examples: stratifying pneumonia phenotypes, revealing survival biomarkers, quantifying medication effects, analyzing cardiovascular risks, and reconstructing SARS-CoV-2 disease trajectories. Ehrapy provides a framework to standardize analysis pipelines on EHR data and serve as a cornerstone for the community.
Introduction
While healthcare digitalization has led to massive collections of Electronic Health Record (EHR) data, analyzing this information remains a significant challenge. EHR datasets are notoriously difficult to work with because they are highly heterogeneous—combining tabular data (vitals, labs), text (clinical notes), and images (X-rays)—and often non-standardized across different hospital systems. Currently, researchers spend roughly 80% of their time on preprocessing and data cleaning. Most existing tools are either vendor-specific, closed-source, or lack the ability to perform the kind of exploratory, visual, and multimodal analysis that has become standard in fields like genomics.
The ehrapy Framework
The core innovation of ehrapy is its adoption of the AnnData (Annotated Data) structure, originally developed for single-cell genomics. This format keeps the primary data matrix (patients vs. measurements) perfectly aligned with metadata (patient demographics) and unstructured annotations (embeddings, graphs).
Key technical features include:
- Preprocessing Pipeline: Built-in modules for automated quality control, winsorization (handling outliers), and multiple imputation strategies (KNN, MICE, MissForest) to handle missing data.
- Dimensionality Reduction: Uses UMAP and t-SNE to visualize "patient landscapes," allowing researchers to see how patients cluster based on clinical similarity.
- Knowledge Inference: Integrated modules for causal inference, survival analysis, and trajectory mapping.
Knowledge Inference and Analytical Capabilities
By treating a hospital visit like a single-cell observation, ehrapy allows for powerful "fate mapping" of disease progression. The framework integrates several sophisticated statistical libraries to provide a comprehensive analytical suite.
- Causal Inference: Leveraging the
dowhylibrary to estimate treatment effects while accounting for confounders. - Survival Analysis: Utilizing
lifelinesfor Kaplan-Meier and Cox Hazard modeling to identify risk factors. - Trajectory Inference: Adapting "pseudotime" algorithms from genomics to model how a patient's disease state progresses over time.
Key Results and Case Studies
The authors validated ehrapy across several large-scale, diverse datasets, demonstrating its versatility and predictive power:
- Pneumonia Stratification: In the PIC (Paediatric Intensive Care) database, ehrapy successfully broke down the generic diagnosis of "unspecified pneumonia" into four distinct clinical phenotypes (sepsis-like, severe with co-infection, viral, and mild). The "sepsis-like" group showed significantly higher mortality (p < 1.09e-18).
- Multimodal Risk Modeling: Using the UK Biobank, the authors showed that combining clinical predictors with imaging-derived features and genetics significantly boosted the predictive power (C-index of 0.81) for Myocardial Infarction compared to genetic-only models (0.63).
- Disease Trajectories: In COVID-19 patients, the framework successfully reconstructed disease severity pathways using only raw X-ray embeddings, matching expert radiologist annotations.
Limitations and Future Directions
Despite its strengths, the authors acknowledge hurdles such as informative missingness—where the absence of a test is itself a clinical indicator—and potential data loss during the translation of clinical codes between languages. Future development will focus on deeper integration with the scverse ecosystem to better support high-frequency time-series data and natural language processing (NLP) for medical notes.
Ehrapy is open-source and available on GitHub, aiming to become a cornerstone for standardized, reproducible EHR analysis.
Summary updated