Exploratory electronic health record analysis with ehrapy

Lukas Heumos, Philipp Ehmele, Tim Treis, Julius Upmeier zu Belzen, Altana Namsaraeva, Nastassya Horlava, Vladimir A. Shitov, Xinyue Zhang, Luke Zappia, Rainer Knoll, Niklas J. Lang, Leon Hetzel, Isaac Virshup, Lisa Sikkema, Eljas Roellin, Fabiola Curion, Roland Eils, Herbert B. Schiller, Anne Hilgendorff, Fabian J. TheismedRxiv 2023 · DOI

Abstract

With progressive digitalization of healthcare systems worldwide, large-scale collection of electronic health records (EHRs) has become commonplace. However, an extensible framework for comprehensive exploratory analysis that accounts for data heterogeneity is missing. Here, we introduce ehrapy, a modular open-source Python framework designed for exploratory end-to-end analysis of heterogeneous epidemiology and electronic health record data. Ehrapy incorporates a series of analytical steps, from data extraction and quality control to the generation of low-dimensional representations. Complemented by rich statistical modules, ehrapy facilitates associating patients with disease states, differential comparison between patient clusters, survival analysis, trajectory inference, causal inference, and more. Leveraging ontologies, ehrapy further enables data sharing and training EHR deep learning models paving the way for foundational models in biomedical research. We demonstrated ehrapy’s features in five distinct examples: stratifying pneumonia phenotypes, revealing survival biomarkers, quantifying medication effects, analyzing cardiovascular risks, and reconstructing SARS-CoV-2 disease trajectories. Ehrapy provides a framework to standardize analysis pipelines on EHR data and serve as a cornerstone for the community.

Introduction

While healthcare digitalization has led to massive collections of Electronic Health Record (EHR) data, analyzing this information remains a significant challenge. EHR datasets are notoriously difficult to work with because they are highly heterogeneous—combining tabular data (vitals, labs), text (clinical notes), and images (X-rays)—and often non-standardized across different hospital systems. Currently, researchers spend roughly 80% of their time on preprocessing and data cleaning. Most existing tools are either vendor-specific, closed-source, or lack the ability to perform the kind of exploratory, visual, and multimodal analysis that has become standard in fields like genomics.

The ehrapy Framework

The core innovation of ehrapy is its adoption of the AnnData (Annotated Data) structure, originally developed for single-cell genomics. This format keeps the primary data matrix (patients vs. measurements) perfectly aligned with metadata (patient demographics) and unstructured annotations (embeddings, graphs).

The ehrapy data preparation pipeline converts heterogeneous health records into a standardized AnnData structure.

Key technical features include:

  • Preprocessing Pipeline: Built-in modules for automated quality control, winsorization (handling outliers), and multiple imputation strategies (KNN, MICE, MissForest) to handle missing data.
  • Dimensionality Reduction: Uses UMAP and t-SNE to visualize "patient landscapes," allowing researchers to see how patients cluster based on clinical similarity.
  • Knowledge Inference: Integrated modules for causal inference, survival analysis, and trajectory mapping.

Knowledge Inference and Analytical Capabilities

By treating a hospital visit like a single-cell observation, ehrapy allows for powerful "fate mapping" of disease progression. The framework integrates several sophisticated statistical libraries to provide a comprehensive analytical suite.

Ehrapy provides modular tools for causal inference, trajectory mapping, survival analysis, and patient stratification.
  • Causal Inference: Leveraging the dowhy library to estimate treatment effects while accounting for confounders.
  • Survival Analysis: Utilizing lifelines for Kaplan-Meier and Cox Hazard modeling to identify risk factors.
  • Trajectory Inference: Adapting "pseudotime" algorithms from genomics to model how a patient's disease state progresses over time.

Key Results and Case Studies

The authors validated ehrapy across several large-scale, diverse datasets, demonstrating its versatility and predictive power:

  1. Pneumonia Stratification: In the PIC (Paediatric Intensive Care) database, ehrapy successfully broke down the generic diagnosis of "unspecified pneumonia" into four distinct clinical phenotypes (sepsis-like, severe with co-infection, viral, and mild). The "sepsis-like" group showed significantly higher mortality (p < 1.09e-18).
  2. Multimodal Risk Modeling: Using the UK Biobank, the authors showed that combining clinical predictors with imaging-derived features and genetics significantly boosted the predictive power (C-index of 0.81) for Myocardial Infarction compared to genetic-only models (0.63).
  3. Disease Trajectories: In COVID-19 patients, the framework successfully reconstructed disease severity pathways using only raw X-ray embeddings, matching expert radiologist annotations.

Limitations and Future Directions

Despite its strengths, the authors acknowledge hurdles such as informative missingness—where the absence of a test is itself a clinical indicator—and potential data loss during the translation of clinical codes between languages. Future development will focus on deeper integration with the scverse ecosystem to better support high-frequency time-series data and natural language processing (NLP) for medical notes.

Ehrapy is open-source and available on GitHub, aiming to become a cornerstone for standardized, reproducible EHR analysis.

Summary updated

Abstract

Abstract With progressive digitalization of healthcare systems worldwide, large-scale collection of electronic health records (EHRs) has become commonplace. However, an extensible framework for comprehensive exploratory analysis that accounts for data heterogeneity is missing. Here, we introduce ehrapy, a modular open-source Python framework designed for exploratory end-to-end analysis of heterogeneous epidemiology and electronic health record data. Ehrapy incorporates a series of analytical steps, from data extraction and quality control to the generation of low-dimensional representations. Complemented by rich statistical modules, ehrapy facilitates associating patients with disease states, differential comparison between patient clusters, survival analysis, trajectory inference, causal inference, and more. Leveraging ontologies, ehrapy further enables data sharing and training EHR deep learning models paving the way for foundational models in biomedical research. We demonstrated ehrapy’s features in five distinct examples: We first applied ehrapy to stratify patients affected by unspecified pneumonia into finer-grained phenotypes. Furthermore, we revealed biomarkers for significant differences in survival among these groups. Additionally, we quantify medication-class effects of pneumonia medications on length of stay. We further leveraged ehrapy to analyze cardiovascular risks across different data modalities. Finally, we reconstructed disease state trajectories in SARS-CoV-2 patients based on imaging data. Ehrapy thus provides a framework that we envision will standardize analysis pipelines on EHR data and serve as a cornerstone for the community.
BibTeX
@misc{heumos2023exploratory,
  author = {Lukas Heumos and Philipp Ehmele and Tim Treis and Julius Upmeier zu Belzen and Altana Namsaraeva and Nastassya Horlava and Vladimir A. Shitov and Xinyue Zhang and Luke Zappia and Rainer Knoll and Niklas J. Lang and Leon Hetzel and Isaac Virshup and Lisa Sikkema and Eljas Roellin and Fabiola Curion and Roland Eils and Herbert B. Schiller and Anne Hilgendorff and Fabian J. Theis},
  title = {{Exploratory electronic health record analysis with ehrapy}},
  year = {2023},
  doi = {10.1101/2023.12.11.23299816},
  howpublished = {medRxiv},
}
Figure