Abstract
The increasing availability of Electronic Health Record (EHR) systems offers immense potential for translational research. However, the sheer volume and complexity of clinical codes—spanning diagnoses, medications, procedures, and laboratory tests—make it difficult for researchers to identify all relevant features for specific phenotypes. Furthermore, privacy regulations often prevent the sharing of patient-level data across institutions, hindering the creation of integrated knowledge networks. This paper introduces Knowledge Extraction via Sparse Embedding Regression (KESER), a framework that utilizes large-scale code embeddings to efficiently identify clinical features and infer relationships between medical concepts. By performing sparse regression on these embeddings rather than raw patient records, KESER enables multi-center collaboration while preserving privacy. Our results demonstrate that KESER-selected features are as effective as those curated by domain experts, and the integrative analysis improves the accuracy of disease-disease and disease-drug relationship detection.
Introduction
Electronic Health Records (EHR) are a goldmine for clinical research, yet they are notoriously difficult to mine. A single patient’s history is often fragmented across thousands of disparate codes, such as ICD (diagnoses), RxNorm (medications), CPT (procedures), and LOINC (labs). Traditionally, researchers have faced two primary bottlenecks: the Manual Labor Gap, where clinical experts must spend months curating feature lists for specific diseases, and the Privacy Wall, where legal and administrative hurdles prevent the sharing of individual-level data between hospital systems.
KESER addresses these challenges by treating medical codes like words in a language. By applying representation learning to EHR data, KESER transforms clinical concepts into high-dimensional vectors (embeddings) and uses sparse regression to automatically identify the most relevant features for any given condition.
Methodology
The KESER pipeline is structured into four distinct stages designed to move from raw data to actionable clinical knowledge:
1. Representation Learning
The process begins by constructing a co-occurrence matrix that tracks how often specific codes appear together within a 30-day window. The researchers then apply SVD-SPPMI (Singular Value Decomposition on Shifted Positive Pointwise Mutual Information) to this matrix. This mathematical transformation converts each medical code into a high-dimensional embedding vector, capturing the latent relationships between concepts.
2. Sparse Embedding Regression (KESER)
To identify features related to a specific "target" disease, the framework performs a sparse regression. In this model, the target's embedding vector serves as the dependent variable, while the vectors of all other codes serve as predictors. This allows the model to identify which codes most strongly "explain" the target concept in vector space.
3. Integrative Multi-Center Analysis
To combine data from multiple hospitals without sharing patient-level records, KESER employs a Group Lasso penalty. This statistical technique allows the model to "borrow strength" from different sites, identifying features that are consistently important across various healthcare systems while accounting for local variations in coding practices.
4. Automated Feature Selection
The non-zero coefficients resulting from the regression provide an automated "shopping list" of the most relevant clinical features for a given condition, effectively replacing the need for manual expert curation.
Data and Validation
The study utilized two massive, distinct EHR datasets to test the framework:
- VA Corporate Data Warehouse (CDW): Data from over 12.6 million veterans across 150 facilities.
- Mass General Brigham (MGB): Data from 2.5 million patients in the Boston area.
The resulting embeddings were validated against "ground truth" relationships curated from external knowledge sources, including Wikipedia, the PheCode hierarchy, DrugBank, and SNOMED-CT.
Key Insights
Privacy-Preserving Collaboration
Because KESER regression occurs entirely in "vector space" using summary statistics (embeddings), no patient-level data ever needs to leave an institution's firewall. This effectively uncouples clinical knowledge from individual identities, providing a significant advance for multi-center studies.
The Dimensionality Trade-off
The research found that the optimal size for a vector depends on the specific task. Lower dimensions (e.g., 300) are highly effective for finding synonyms or direct translations between different hospital systems. However, higher dimensions (retaining 95% of variance) are necessary for capturing the complex clinical relatedness required for accurate feature selection.
Overcoming Institutional Heterogeneity
Different hospitals often code the same clinical event differently. For instance, one site might use a procedure code for a drug injection while another uses a medication code. KESER’s integrative approach successfully maps these disparate codes to the same underlying clinical concept, creating a more robust, universal knowledge map.
Results
The performance of KESER was evaluated across eight diseases, yielding several key findings:
- Relationship Detection: The embeddings achieved an Area Under the Curve (AUC) of 0.84 to 0.89 in detecting known clinical relationships, such as which drugs are indicated for specific diseases.
- Feature Selection Sensitivity: In a case study for Rheumatoid Arthritis (RA), the integrative KESER model identified 100% of the 16 medications commonly used for the condition, significantly outperforming models trained on single-institution data.
- Phenotyping Accuracy: Phenotype algorithms built using KESER’s automated features performed as well as, and sometimes better than, those built by human experts. On average, KESER-based models saw an F-score improvement of 0.173 over models using standard "full" feature sets.
- Rapid Response (COVID-19): In an exploratory analysis of COVID-19 data, KESER successfully identified key clinical markers like D-dimer and Ferritin, suggesting its utility for rapidly building knowledge maps for emerging diseases.
Limitations and Future Work
Despite its success, KESER faces challenges regarding Lab Code Standardization, as laboratory codes remain highly site-specific and difficult to map perfectly across institutions. Additionally, while top-5 mapping accuracy between institutions was strong (~70%), the top-1 "direct hit" accuracy (~40%) indicates that cross-site translation remains an area for refinement.
Conclusion
KESER demonstrates that code embeddings can effectively reveal clinical knowledge and automate the grouping of clinical features. By bypassing the need for patient-level data in individual analyses, KESER provides a scalable, privacy-preserving framework for multi-center EHR research, facilitating the development of more accurate and comprehensive clinical models.
Summary updated