Dense Embeddings for Sparse Electronic Health Records

This direction is split into other directions: high-dimensional data sparsity (high-dimensional clinical data sparsity). Its last version stays readable and citable here.

Version 1 of 191 papers98 conceptsfrozen 2026-10-04

Electronic Health Records (EHR) are often described as a "goldmine" for medical research, but for machine learning models, they are more like a desert. While a hospital system may use tens of thousands of unique ICD diagnosis codes, procedure codes, and medications, any single patient record contains only a tiny fraction of them.

This creates a fundamental problem: extreme high-dimensional sparsity. When data is represented as a simple binary vector (a 1 if a patient has a code, a 0 if they don’t), the resulting matrices are mostly empty. As Aron Henriksson and Jing Zhao note in Ensembles of randomized trees using diverse distributed representations of clinical events, this sparsity triggers the "curse of dimensionality," making it nearly impossible for traditional algorithms to distinguish meaningful clinical signals from random noise.

The research direction across these 81 papers focuses on a shared solution: Representation Learning. Instead of treating medical codes as independent, isolated entities, researchers are developing ways to "distill" sparse, high-dimensional data into dense, low-dimensional embeddings that capture the underlying relationships between diseases, treatments, and outcomes.

The Intuition: From Orthogonal to Meaningful

In a raw EHR dataset, "Type 2 Diabetes" and "Insulin" are treated as two completely different dimensions. Mathematically, they are orthogonal. A model seeing these for the first time doesn't know they are related.

To solve this, researchers like Zihao Zhu and Buyue Qian borrow techniques from Natural Language Processing. In Measuring Patient Similarities via a Deep Architecture with Medical Concept Embedding, they treat a patient's history like a sentence and medical codes like words. By using "Word2Vec-style" embeddings, the model learns that codes appearing in similar clinical contexts should be close together in a mathematical space.

This transformation turns a sparse vector of 50,000 dimensions into a dense vector of perhaps 100 dimensions. In this new space, "Diabetes" and "Insulin" are no longer orthogonal; they are neighbors. This allows models to generalize: if a model learns that a certain pattern of "neighboring" codes predicts heart failure, it can apply that knowledge even to patients who have slightly different, but related, codes.

Strategies for Densification

The field has converged on several distinct mechanisms to bridge the gap between sparse inputs and dense insights:

1. Autoencoders and Self-Supervision

When labeled data is scarce—a common problem in medicine—researchers use the data to "supervise itself." Sébastien Dubois and Nigam Shah demonstrated in The Effectiveness of Transfer Learning in Electronic Health Records Data that a "split-brain" autoencoder can learn dense representations by trying to predict one part of a patient's record (like medications) from another (like diagnoses). Similarly, Dongxiao Zhu found that Stacked Sparse Autoencoders (SSAE) are particularly effective at extracting features when the number of clinical variables far exceeds the number of patients (A Predictive Approach Using Deep Feature Learning for Electronic Medical Records).

2. Tensor Factorization

Clinical data is multi-way: it involves patients, medications, and diagnoses simultaneously. Yejin Kim and Jimeng Sun utilize Tensor Factorization to decompose these massive, sparse three-dimensional blocks into smaller, dense factors (Federated Tensor Factorization for Computational Phenotyping). This "compresses" the data while preserving the interactions between different types of clinical events.

3. Knowledge Graph Enrichment

Sometimes, the EHR itself doesn't contain enough information to overcome sparsity. Edward W Huang and Sheng Wang 0012 address this in VisAGE by injecting external biological knowledge. By linking sparse patient codes to dense protein and genomic knowledge graphs, they "fill in the blanks," allowing the model to see connections that aren't explicitly recorded in the hospital's billing logs.

Why It Works: Stability Through Abstraction

The reason these methods outperform raw data is stability. Raw clinical codes are noisy and subject to administrative changes. As Matthew B.A. McDermott points out in MIMIC-Extract, using raw identifiers makes models vulnerable to software updates or varying hospital practices.

By moving to a dense embedding space, the model stops looking for exact matches of rare codes and starts looking for clinical phenotypes. Whether a patient is labeled with a specific ICD-9 code or a slightly different ICD-10 code, their "location" in the embedding space remains similar. This abstraction allows for more robust predictions of complex outcomes, such as the four distinct subphenotypes of Long COVID identified by Hao Zhang 0050 and Chengxi Zang (Data-driven identification of post-acute SARS-CoV-2 infection subphenotypes).

Ultimately, the shift from sparse, high-dimensional vectors to dense, learned representations is what enables AI to move from simple administrative counting to true clinical "understanding."

Papers in this version

Showing 60 of 91 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dense_embeddings_for_sparse_electronic_h_37f2bcd1,
  title = {Dense Embeddings for Sparse Electronic Health Records},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dense-embeddings-for-sparse-electronic-health-records/rdr_37f2bcd11b024c7485a8b9ec7ca35560},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_dense_embeddings_for_sparse_electronic_h_37f2bcd1_v1,
  title = {Dense Embeddings for Sparse Electronic Health Records},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dense-embeddings-for-sparse-electronic-health-records/rdr_37f2bcd11b024c7485a8b9ec7ca35560/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}