Clinical knowledge extraction via sparse embedding regression (KESER) with multi-center large scale electronic health record data

Chuan Hong, Everett Neil Rush, Molei Liu, Doudou Zhou, Jiehuan Sun, Aaron Sonabend W., Victor M. Castro, Petra Schubert, Vidul Ayakulangara Panickan, Tianrun Cai, Lauren Costa, Zeling He, Nicholas B. Link, Ronald G. Hauser, J. Michael Gaziano, Shawn N. Murphy, George Ostrouchov, Yuk-Lam Ho, Edmon Begoli, Junwei Lu, Kelly Cho, Katherine P. Liao, Tianxi CaiNpj Digit. Medicine 2021 · DOI · OpenReview

Abstract

The increasing availability of Electronic Health Record (EHR) systems offers immense potential for translational research. However, the sheer volume and complexity of clinical codes—spanning diagnoses, medications, procedures, and laboratory tests—make it difficult for researchers to identify all relevant features for specific phenotypes. Furthermore, privacy regulations often prevent the sharing of patient-level data across institutions, hindering the creation of integrated knowledge networks. This paper introduces Knowledge Extraction via Sparse Embedding Regression (KESER), a framework that utilizes large-scale code embeddings to efficiently identify clinical features and infer relationships between medical concepts. By performing sparse regression on these embeddings rather than raw patient records, KESER enables multi-center collaboration while preserving privacy. Our results demonstrate that KESER-selected features are as effective as those curated by domain experts, and the integrative analysis improves the accuracy of disease-disease and disease-drug relationship detection.

Introduction

Electronic Health Records (EHR) are a goldmine for clinical research, yet they are notoriously difficult to mine. A single patient’s history is often fragmented across thousands of disparate codes, such as ICD (diagnoses), RxNorm (medications), CPT (procedures), and LOINC (labs). Traditionally, researchers have faced two primary bottlenecks: the Manual Labor Gap, where clinical experts must spend months curating feature lists for specific diseases, and the Privacy Wall, where legal and administrative hurdles prevent the sharing of individual-level data between hospital systems.

KESER addresses these challenges by treating medical codes like words in a language. By applying representation learning to EHR data, KESER transforms clinical concepts into high-dimensional vectors (embeddings) and uses sparse regression to automatically identify the most relevant features for any given condition.

Methodology

The KESER pipeline is structured into four distinct stages designed to move from raw data to actionable clinical knowledge:

1. Representation Learning

The process begins by constructing a co-occurrence matrix that tracks how often specific codes appear together within a 30-day window. The researchers then apply SVD-SPPMI (Singular Value Decomposition on Shifted Positive Pointwise Mutual Information) to this matrix. This mathematical transformation converts each medical code into a high-dimensional embedding vector, capturing the latent relationships between concepts.

2. Sparse Embedding Regression (KESER)

To identify features related to a specific "target" disease, the framework performs a sparse regression. In this model, the target's embedding vector serves as the dependent variable, while the vectors of all other codes serve as predictors. This allows the model to identify which codes most strongly "explain" the target concept in vector space.

3. Integrative Multi-Center Analysis

To combine data from multiple hospitals without sharing patient-level records, KESER employs a Group Lasso penalty. This statistical technique allows the model to "borrow strength" from different sites, identifying features that are consistently important across various healthcare systems while accounting for local variations in coding practices.

4. Automated Feature Selection

The non-zero coefficients resulting from the regression provide an automated "shopping list" of the most relevant clinical features for a given condition, effectively replacing the need for manual expert curation.

Data and Validation

The study utilized two massive, distinct EHR datasets to test the framework:

  • VA Corporate Data Warehouse (CDW): Data from over 12.6 million veterans across 150 facilities.
  • Mass General Brigham (MGB): Data from 2.5 million patients in the Boston area.

The resulting embeddings were validated against "ground truth" relationships curated from external knowledge sources, including Wikipedia, the PheCode hierarchy, DrugBank, and SNOMED-CT.

Key Insights

Privacy-Preserving Collaboration

Because KESER regression occurs entirely in "vector space" using summary statistics (embeddings), no patient-level data ever needs to leave an institution's firewall. This effectively uncouples clinical knowledge from individual identities, providing a significant advance for multi-center studies.

The Dimensionality Trade-off

The research found that the optimal size for a vector depends on the specific task. Lower dimensions (e.g., 300) are highly effective for finding synonyms or direct translations between different hospital systems. However, higher dimensions (retaining 95% of variance) are necessary for capturing the complex clinical relatedness required for accurate feature selection.

Overcoming Institutional Heterogeneity

Different hospitals often code the same clinical event differently. For instance, one site might use a procedure code for a drug injection while another uses a medication code. KESER’s integrative approach successfully maps these disparate codes to the same underlying clinical concept, creating a more robust, universal knowledge map.

Results

The performance of KESER was evaluated across eight diseases, yielding several key findings:

  • Relationship Detection: The embeddings achieved an Area Under the Curve (AUC) of 0.84 to 0.89 in detecting known clinical relationships, such as which drugs are indicated for specific diseases.
  • Feature Selection Sensitivity: In a case study for Rheumatoid Arthritis (RA), the integrative KESER model identified 100% of the 16 medications commonly used for the condition, significantly outperforming models trained on single-institution data.
  • Phenotyping Accuracy: Phenotype algorithms built using KESER’s automated features performed as well as, and sometimes better than, those built by human experts. On average, KESER-based models saw an F-score improvement of 0.173 over models using standard "full" feature sets.
  • Rapid Response (COVID-19): In an exploratory analysis of COVID-19 data, KESER successfully identified key clinical markers like D-dimer and Ferritin, suggesting its utility for rapidly building knowledge maps for emerging diseases.

Limitations and Future Work

Despite its success, KESER faces challenges regarding Lab Code Standardization, as laboratory codes remain highly site-specific and difficult to map perfectly across institutions. Additionally, while top-5 mapping accuracy between institutions was strong (~70%), the top-1 "direct hit" accuracy (~40%) indicates that cross-site translation remains an area for refinement.

Conclusion

KESER demonstrates that code embeddings can effectively reveal clinical knowledge and automate the grouping of clinical features. By bypassing the need for patient-level data in individual analyses, KESER provides a scalable, privacy-preserving framework for multi-center EHR research, facilitating the development of more accurate and comprehensive clinical models.

Summary updated

Abstract

The increasing availability of electronic health record (EHR) systems has created enormous potential for translational research. However, it is difficult to know all the relevant codes related to a phenotype due to the large number of codes available. Traditional data mining approaches often require the use of patient-level data, which hinders the ability to share data across institutions. In this project, we demonstrate that multi-center large-scale code embeddings can be used to efficiently identify relevant features related to a disease of interest. We constructed large-scale code embeddings for a wide range of codified concepts from EHRs from two large medical centers. We developed knowledge extraction via sparse embedding regression (KESER) for feature selection and integrative network analysis. We evaluated the quality of the code embeddings and assessed the performance of KESER in feature selection for eight diseases. Besides, we developed an integrated clinical knowledge map combining embedding data from both institutions. The features selected by KESER were comprehensive compared to lists of codified data generated by domain experts. Features identified via KESER resulted in comparable performance to those built upon features selected manually or with patient-level data. The knowledge map created using an integrative analysis identified disease-disease and disease-drug pairs more accurately compared to those identified using single institution data. Analysis of code embeddings via KESER can effectively reveal clinical knowledge and infer relatedness among codified concepts. KESER bypasses the need for patient-level data in individual analyses providing a significant advance in enabling multi-center studies using EHR data.
BibTeX
@article{hong2021clinical,
  author = {Chuan Hong and Everett Neil Rush and Molei Liu and Doudou Zhou and Jiehuan Sun and Aaron Sonabend W. and Victor M. Castro and Petra Schubert and Vidul Ayakulangara Panickan and Tianrun Cai and Lauren Costa and Zeling He and Nicholas B. Link and Ronald G. Hauser and J. Michael Gaziano and Shawn N. Murphy and George Ostrouchov and Yuk-Lam Ho and Edmon Begoli and Junwei Lu and Kelly Cho and Katherine P. Liao and Tianxi Cai},
  title = {{Clinical knowledge extraction via sparse embedding regression (KESER) with multi-center large scale electronic health record data}},
  journal = {Npj Digit. Medicine},
  year = {2021},
  doi = {10.1038/s41746-021-00519-z},
}
Figure