Patient Similarity Analysis with Longitudinal Health Data

Ahmed Allam, Matthias Dittberner, Anna Sintsova, Dominique Brodbeck, Michael KrauthammerarXiv 2020 · arXiv

TL;DR

This paper provides a comprehensive review of Patient Similarity Analysis (PSA), a computational framework that treats Electronic Health Record (EHR) data as chronological "patient journeys." By identifying clusters of similar disease trajectories, researchers can move beyond one-size-fits-all medicine to enable personalized outcome predictions and targeted treatment strategies.

What Problem Does This Paper Address?

Modern medicine is sitting on a goldmine of data: hundreds of exabytes of Electronic Health Records (EHRs) containing every medical visit, lab test, and procedure a patient undergoes. However, this data is notoriously "messy"—it is high-dimensional, multi-modal (mixing text, numbers, and images), and irregularly sampled.

Traditional Randomized Controlled Trials (RCTs) remain the gold standard for evidence, but they often only look at a few specific time points and highly curated populations. The challenge addressed here is how to computationally compare complex, noisy, longitudinal "patient journeys" to find "patients like me." If we can define what makes two patient histories similar, we can build a "learning health care system" that uses historical data to predict future outcomes for new patients.

Methods

The authors outline a prototypical three-stage workflow for PSA, as illustrated in the processing schema below:

Figure 3. Basic processing schema (top, grey boxes) and application domains (bottom) for patient similarity analysis with longitudinal health data.
  1. Data Abstraction: Raw, irregular timelines are transformed into lower-dimensional, non-sparse representations.
  2. Standard techniques: Symbolic Aggregate Approximation (SAX) and Piecewise Aggregate Approximation (PAA) to simplify numerical time series.
  3. Modern approaches: Using Neural Networks (Autoencoders, LSTMs, and GRUs) to automatically learn vector representations (embeddings) of patient journeys.
  4. Similarity Metrics & Algorithms: Once data is abstracted, the system must calculate "distance" between patients.
  5. Heuristic/Distance-based: Euclidean distance or Cosine similarity.
  6. Temporal Matching: Dynamic Time Warping (DTW) and Smith-Waterman algorithms, which allow for comparing sequences that may be shifted or stretched in time.
  7. Supervised Learning: Using expert physician feedback to "teach" the algorithm which features actually matter for similarity.
  8. Applications: The final stage applies these similarities to three main domains: Cohort Building (finding homogeneous groups), Pathway Discovery (uncovering common disease trajectories), and Personalized Prediction (training models on similar patients rather than the whole population).

Datasets

The review identifies several cornerstone datasets used in the field:

  • MIMIC-II and III: De-identified clinical data from intensive care unit (ICU) patients.
  • eICU-CRD: A large multi-center critical care database.
  • ADNI: The Alzheimer’s Disease Neuroimaging Initiative.
  • PPMI: Parkinson's Progression Markers Initiative.
  • Real-world EHR/Claims Data: Large-scale datasets from providers like the New York-Presbyterian/Columbia University Medical Center and population-wide registries (e.g., 6 million Danish patients).

Key Insights and Ideas

  • The "Patient Journey" Paradigm: Instead of viewing a patient as a static set of symptoms, PSA views them as a dynamic trajectory. This allows for the discovery of "endotypes"—subtypes of diseases that look similar at one point but follow very different paths.
  • Local vs. Global Models: A key insight is that outcome prediction models trained only on similar patients (local models) often outperform "global" models trained on the entire population. This minimizes bias by ensuring the training data is relevant to the specific "index patient."
  • The Rise of Deep Learning: Research has shifted significantly toward using neural networks to learn similarity metrics automatically, rather than relying on hand-crafted heuristic rules.
  • Handling Irregularity: PSA is uniquely suited to handle the "gaps" in medical data. Techniques like DTW allow the system to recognize that two patients are following the same path even if their doctor visits happened months apart.

Key Results

The review analyzed 125 papers published between 1995 and 2019. As shown in Figure 2, there has been an exponential growth in PSA research, particularly in the realm of Personalized Outcome Prediction.

Quantitatively, the reviewed studies showed that:

  • Personalized cohorts led to higher accuracy (AUROC) in predicting events like ICU mortality, sepsis, and heart failure compared to traditional non-temporal models.
  • Neural network-derived representations consistently outperformed baseline symbolic representations in both classification and patient retrieval tasks.
  • Pathway discovery algorithms successfully identified prototypical trajectories for complex conditions like septicemia and Type 2 Diabetes, revealing previously unrecognized sub-groups of patients.

Limitations and Future Work

Despite the progress, several hurdles remain:

  • Lack of Consensus: There is currently no "gold standard" methodology for PSA. Different abstraction methods and similarity metrics can yield wildly different results on the same data.
  • Data Noise and Missingness: While some algorithms (like HEOM or specific RNN architectures) handle missing data well, the inherent "messiness" of EHRs remains a significant barrier to clinical deployment.
  • Interpretability: As the field moves toward deep learning, the "black box" nature of these models makes it difficult for clinicians to understand why two patients are considered similar.
  • Scalability: Comparing every patient journey to every other journey in a database of millions is computationally expensive, requiring more research into scalable retrieval methods like locality-sensitive hashing (LSH).

Summary updated

Abstract

Healthcare professionals have long envisioned using the enormous processing powers of computers to discover new facts and medical knowledge locked inside electronic health records. These vast medical archives contain time-resolved information about medical visits, tests and procedures, as well as outcomes, which together form individual patient journeys. By assessing the similarities among these journeys, it is possible to uncover clusters of common disease trajectories with shared health outcomes. The assignment of patient journeys to specific clusters may in turn serve as the basis for personalized outcome prediction and treatment selection. This procedure is a non-trivial computational problem, as it requires the comparison of patient data with multi-dimensional and multi-modal features that are captured at different times and resolutions. In this review, we provide a comprehensive overview of the tools and methods that are used in patient similarity analysis with longitudinal data and discuss its potential for improving clinical decision making.
BibTeX
@misc{allam2020patient,
  author = {Ahmed Allam and Matthias Dittberner and Anna Sintsova and Dominique Brodbeck and Michael Krauthammer},
  title = {{Patient Similarity Analysis with Longitudinal Health Data}},
  year = {2020},
  eprint = {2005.06630},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2005.06630},
}

Loading related content...

Figure