A Predictive Approach Using Deep Feature Learning for Electronic Medical Records: A Comparative Study

Milad Zafar Nezhad, Dongxiao Zhu, Najibesadat Sadati, Kai YangarXiv 2018 · arXiv

Abstract

Electronic Health Records (EHRs) offer a wealth of information for personalized medicine but present significant challenges due to high dimensionality, sparsity, and heterogeneous data types. This paper presents a comparative study of deep autoencoder architectures for extracting robust feature representations from EHR data. We evaluate four architectures—Stacked Sparse Autoencoders (SSAE), Deep Belief Networks (DBN), Variational Autoencoders (VAE), and Adversarial Autoencoders (AAE)—across datasets of varying scales. Our findings reveal that while deep feature learning consistently improves predictive accuracy over raw data, the optimal architecture depends on dataset size: SSAE is most effective for small datasets, whereas VAE dominates in large-scale applications, significantly outperforming traditional clinical scoring systems like APACHE.

Introduction: The Challenge of EHR Data

Electronic Health Records (EHRs) are a cornerstone of modern healthcare, yet their utility for predictive modeling is often hampered by inherent "noise." EHR data is characterized by extreme sparsity, high dimensionality, and a complex mixture of structured and unstructured information, including demographics, medical history, laboratory results, and imaging data.

Diverse data types within Electronic Health Records (EHR).

A major bottleneck in utilizing this data is the scarcity of labeled samples. While raw data is abundant, expert clinical labeling is resource-intensive. This study addresses the challenge of representation learning: automatically transforming raw, complex EHR data into compact, informative features that enhance the performance of downstream clinical prediction tasks.

Methodology: The Deep Integrated Predictive (DIP) Approach

The authors propose a three-stage framework termed "Deep Integrated Predictive" (DIP) to handle the complexities of EHR data and improve clinical outcomes prediction.

The three-stage Deep Integrated Predictive (DIP) framework for feature representation and risk prediction.
  1. Preprocessing & Embedding: Outliers and missing values are addressed first. Categorical and textual variables are then transformed into dense numerical vectors using the GloVe (Global Vectors) algorithm, ensuring a uniform input format for deep models.
  2. Feature Learning: The core of the DIP approach involves compressing the high-dimensional input into a lower-dimensional "latent space." Four architectures were compared:
  3. Stacked Sparse Autoencoders (SSAE): Employs sparsity constraints to capture only the most salient features.
  4. Deep Belief Networks (DBN): A generative model composed of stacked Restricted Boltzmann Machines (RBMs).
  5. Variational Autoencoders (VAE): A probabilistic model that learns the underlying probability distribution of the data.
  6. Adversarial Autoencoders (AAE): Combines autoencoders with an adversarial training objective to regularize the latent space.
  7. Supervised Learning: The learned latent features are fed into standard classifiers (e.g., Random Forest, SVM, Lasso regression) to predict specific clinical outcomes.

Experimental Design and Datasets

The study evaluates the DIP framework across two distinct scales:

  • DMC Dataset (Small Scale): Comprising 91 African-American patients from the Detroit Medical Center, with 172 attributes used to predict heart damage risk (Left Ventricular Mass Index).
  • eICU Database (Large Scale): A multi-center database from which two subsets were extracted: Cardiac-eICU (~7,000 records) and Neuro-eICU (~8,000 records). Over 150 features were used to predict patient Length of Stay (LOS).

Results and Discussion

The experimental results demonstrate that deep feature representation consistently yields lower error rates (RMSE) compared to using raw features.

The Impact of Dataset Scale

A key insight from this study is that the "best" architecture is highly dependent on the amount of available data. For the small DMC dataset, SSAE achieved the best performance (RMSE of 6.89 vs. 11.08 for raw features), likely because its sparsity regularization prevents overfitting on limited data. Conversely, for the larger eICU datasets, VAE was the superior performer (e.g., RMSE of 0.14 in Neuro-ICU vs. 1.38 for raw features), as its generative capacity effectively models complex distributions when sufficient data is present.

Outperforming Clinical Baselines

In the eICU experiments, the deep learning models significantly outperformed the APACHE score, a long-standing clinical standard for ICU outcome prediction. While APACHE scores resulted in RMSEs exceeding 8.0, the proposed deep learning models reduced this error to near-negligible levels, suggesting that automated feature learning captures critical nuances missed by manual scoring systems.

Conclusion

This comparative study highlights the power of deep representation learning in transforming messy EHR data into actionable clinical insights. While generative models like VAE and AAE excel at scale, regularized models like SSAE remain vital for smaller clinical cohorts. Future work will focus on improving the interpretability of these "black box" models and optimizing the computationally intensive hyperparameter tuning process required for these deep architectures.

Summary updated

Abstract

Increasing volume of Electronic Health Records (EHR) in recent years provides great opportunities for data scientists to collaborate on different aspects of healthcare research by applying advanced analytics to these EHR clinical data. A key requirement however is obtaining meaningful insights from high dimensional, sparse and complex clinical data. Data science approaches typically address this challenge by performing feature learning in order to build more reliable and informative feature representations from clinical data followed by supervised learning. In this paper, we propose a predictive modeling approach based on deep learning based feature representations and word embedding techniques. Our method uses different deep architectures (stacked sparse autoencoders, deep belief network, adversarial autoencoders and variational autoencoders) for feature representation in higher-level abstraction to obtain effective and robust features from EHRs, and then build prediction models on top of them. Our approach is particularly useful when the unlabeled data is abundant whereas labeled data is scarce. We investigate the performance of representation learning through a supervised learning approach. Our focus is to present a comparative study to evaluate the performance of different deep architectures through supervised learning and provide insights in the choice of deep feature representation techniques. Our experiments demonstrate that for small data sets, stacked sparse autoencoder demonstrates a superior generality performance in prediction due to sparsity regularization whereas variational autoencoders outperform the competing approaches for large data sets due to its capability of learning the representation distribution.
BibTeX
@misc{nezhad2018predictive,
  author = {Milad Zafar Nezhad and Dongxiao Zhu and Najibesadat Sadati and Kai Yang},
  title = {{A Predictive Approach Using Deep Feature Learning for Electronic Medical Records: A Comparative Study}},
  year = {2018},
  eprint = {1801.02961},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1801.02961},
}
Figure