Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms

David J. Albers, Noémie Elhadad, Jan Claassen, Rimma Perotte, Ayelet Goldstein, George HripcsakJournal of Biomedical Informatics 2018 · DOI

TL;DR

This paper introduces PopKLD, an automated algorithm that summarizes complex electronic health record (EHR) laboratory data into a more informative and robust representation. This new summary better captures underlying physiological signals, even in the presence of healthcare process biases, and proves superior to traditional methods like mean and standard deviation for tasks like disease phenotyping.

What Problem Does This Paper Address?

Electronic health records (EHRs) contain a wealth of laboratory data, but this data is often noisy, irregularly sampled, and influenced by the "health care process" (e.g., when and why tests are ordered). Traditional methods for summarizing this data, like calculating the mean and standard deviation, often fail to capture important physiological information and can be misled by biases. This makes it challenging to use EHR laboratory data effectively for research, particularly for high-throughput phenotyping (identifying patient subgroups based on their characteristics). The authors aim to develop a method that can automatically extract a more meaningful, robust, and interpretable summary from this raw data, accounting for biases and preserving physiological signals.

Methods

The core of the paper is the PopKLD (Population Kullback-Leibler Divergence) algorithm. It works by:

  1. Selecting a set of parameterized probability distributions (e.g., Gaussian, log-normal, GEV) that could potentially model the laboratory data.
  2. Estimating the parameters for each of these distributions using the entire population's laboratory data.
  3. Estimating the true, non-parametric distribution of the population's laboratory data using a Kernel Density Estimate (KDE).
  4. Calculating the Kullback-Leibler (KL) divergence between each parameterized distribution and the non-parametric KDE. This measures how much information is lost when approximating the true distribution with a simpler parameterized one.
  5. Selecting the parameterized distribution that minimizes the KL divergence. This distribution is considered the "best fit" for the population's data.
  6. Estimating the parameters of this selected distribution for each individual patient. These individual parameter estimates serve as the summary statistics.

Additionally, they introduce PopKLD-CAT, which transforms these continuous summary parameters into categorical variables, making them suitable for methods like topic modeling.

Datasets

The study utilized two primary datasets:

  1. NewYork-Presbyterian Hospital (NYPH) clinical data warehouse: This dataset comprised longitudinal records for approximately 14,000 patients visiting the Ambulatory Internal Medicine clinic over 20 years, including 64 frequently ordered laboratory tests.
  2. Neurological Intensive Care Unit (ICU) data: This dataset included 814 patients who were comatose and tube-fed, focusing specifically on glucose measurements collected during their ICU stay. This dataset was used to isolate physiological signals with minimal healthcare process bias.

Key Insights and Ideas

  • Healthcare process biases are significant: The way data is collected and influenced by clinical decisions (the "health care process") profoundly impacts laboratory data distributions, often obscuring underlying physiology.
  • Traditional summaries are often insufficient: Mean and standard deviation are non-robust and can be easily misled by outliers and biases, failing to capture crucial information.
  • Model selection is key: Using information-theoretic measures like KL divergence allows for a principled way to select the best parametric model to represent complex, noisy data.
  • Different contexts demand different models: The optimal statistical model for summarizing a laboratory test can vary significantly depending on the clinical context (e.g., ICU vs. general outpatient).
  • Physiological relationships are preserved: The PopKLD algorithm successfully identifies and preserves known physiological relationships (e.g., between mean and standard deviation of glucose) that are lost with simpler summary statistics.
  • Maximum entropy as a guide: The principle of maximum entropy can predict which distributions are likely to be most informative for certain data types, serving as a useful benchmark for evaluating PopKLD's model choices.

Key Results

  • Improved Phenotyping: In a phenotyping task for diabetes, CKD, and pancreatitis, PopKLD-derived summaries (both continuous and categorical) significantly outperformed traditional mean and standard deviation summaries in identifying disease states, achieving higher cluster purity against a gold standard. For example, in identifying diabetes using glucose, PopKLD achieved a purity of 0.93 (15/15 patients) with a GEV model, while mean/standard deviation achieved only 0.53 (8/15 patients).
  • Preservation of Physiology: For glucose data, PopKLD identified models (log-normal and gamma) that revealed the known linear relationship between mean and standard deviation, a relationship obscured by raw mean/standard deviation calculations.
  • Model Diversity: The study found no single "best" distribution for all laboratory tests; different tests and contexts necessitate different models, highlighting the value of PopKLD's model selection approach.
  • Robustness to Outliers: PopKLD models, particularly GEV and log-normal, demonstrated a better ability to handle outliers compared to simple mean and standard deviation.

Limitations and Future Work

  • Assumptions: The algorithm assumes that a single parameterized distribution can reasonably represent the population's data, even with variations. It also assumes that the chosen models are sufficient to capture the underlying generating process.
  • Interpretability vs. Accuracy: While PopKLD strikes a balance, further modifications could increase accuracy (e.g., using more complex models like KDEs or neural networks) at the potential cost of interpretability.
  • Uncertainty Quantification: The current PopKLD algorithm does not explicitly quantify uncertainty in its estimates. The authors suggest that incorporating methods like bootstrapping could address this.
  • Model Averaging: For applications where multiple models perform similarly well, using model averaging or ensemble learning could further improve robustness.
  • Health Care Process Understanding: While PopKLD accounts for biases, a deeper understanding of the specific health care processes generating the data could lead to even more refined summarization.

Summary updated

Abstract

We study the question of how to represent or summarize raw laboratory data taken from an electronic health record (EHR) using parametric model selection to reduce or cope with biases induced through clinical care. It has been previously demonstrated that the health care process (Hripcsak and Albers, 2012, 2013), as defined by measurement context (Hripcsak and Albers, 2013; Albers et al., 2012) and measurement patterns (Albers and Hripcsak, 2010, 2012), can influence how EHR data are distributed statistically (Kohane and Weber, 2013; Pivovarov et al., 2014). We construct an algorithm, PopKLD, which is based on information criterion model selection (Burnham and Anderson, 2002; Claeskens and Hjort, 2008), is intended to reduce and cope with health care process biases and to produce an intuitively understandable continuous summary. The PopKLD algorithm can be automated and is designed to be applicable in high-throughput settings; for example, the output of the PopKLD algorithm can be used as input for phenotyping algorithms. Moreover, we develop the PopKLD-CAT algorithm that transforms the continuous PopKLD summary into a categorical summary useful for applications that require categorical data such as topic modeling. We evaluate our methodology in two ways. First, we apply the method to laboratory data collected in two different health care contexts, primary versus intensive care. We show that the PopKLD preserves known physiologic features in the data that are lost when summarizing the data using more common laboratory data summaries such as mean and standard deviation. Second, for three disease-laboratory measurement pairs, we perform a phenotyping task: we use the PopKLD and PopKLD-CAT algorithms to define high and low values of the laboratory variable that are used for defining a disease state. We then compare the relationship between the PopKLD-CAT summary disease predictions and the same predictions using empirically estimated mean and standard deviation to a gold standard generated by clinical review of patient records. We find that the PopKLD laboratory data summary is substantially better at predicting disease state. The PopKLD or PopKLD-CAT algorithms are not meant to be used as phenotyping algorithms, but we use the phenotyping task to show what information can be gained when using a more informative laboratory data summary. In the process of evaluation our method we show that the different clinical contexts and laboratory measurements necessitate different statistical summaries. Similarly, leveraging the principle of maximum entropy we argue that while some laboratory data only have sufficient information to estimate a mean and standard deviation, other laboratory data captured in an EHR contain substantially more information than can be captured in higher-parameter models.
BibTeX
@article{albers2018estimating,
  author = {David J. Albers and Noémie Elhadad and Jan Claassen and Rimma Perotte and Ayelet Goldstein and George Hripcsak},
  title = {{Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms}},
  journal = {Journal of Biomedical Informatics},
  year = {2018},
  doi = {10.1016/j.jbi.2018.01.004},
}
Figure