Improving Diagnostics with Deep Forest Applied to Electronic Health Records

Atieh Khodadadi, Nima Ghanbari Bousejin, Soheila Molaei, Vinod Kumar Chauhan, Tingting Zhu, David A. Clifton

TL;DR

This paper introduces "Patient Forest," a novel deep learning approach that learns patient representations from Electronic Health Records (EHRs) to predict readmission and mortality. Patient Forest outperforms existing methods, particularly when training data is limited, by leveraging a cascade of random forests and statistical features for robust diagnostic predictions.

What Problem Does This Paper Address?

Electronic Health Records (EHRs) are a treasure trove of patient information, holding demographics, medical history, lab results, and more. This data has immense potential for improving patient care, predicting outcomes like readmission or mortality, and optimizing treatment. However, EHR data is notoriously complex and challenging to work with. Key issues include:

  • High dimensionality and sparsity: EHRs contain a vast amount of information, much of which might be missing or irrelevant for a specific prediction task.
  • Data limitations: Finding stable models that can effectively relate diverse medical concepts within EHRs is difficult.
  • Imbalanced classes: Certain conditions or outcomes are much rarer than others, making it hard for models to learn effectively.
  • Need for unified representations: Integrating various data types (vitals, diagnoses, medications) into a single, meaningful representation is crucial for comprehensive analysis.

Existing machine learning and deep learning methods often struggle with these challenges, requiring large datasets and significant computational power, or failing to capture the intricate relationships within EHRs.

Methods

The core of this paper is the Patient Forest model, which is built upon the gcForest framework. Here's a breakdown of the approach:

  • Patient Forest: This is an end-to-end model designed to learn patient representations from EHR data. It frames the prediction task as a classification problem.
  • gcForest Framework: Patient Forest leverages the gcForest architecture, which consists of two main modules:
  • Multi-grained Scanning: This module acts like a set of sliding windows (akin to convolutional filters) that process the input EHR data at different granularities. It generates a diverse set of features, capturing various aspects of the patient's record.
  • Cascade Forest: This module is an ensemble of random forests organized in multiple levels. Each level takes the refined features from the previous level and further processes them, progressively improving the representation. This cascade structure allows for iterative refinement of the learned features.
  • Statistical Features: A key innovation is the incorporation of statistical features alongside the raw EHR data. This helps in creating a more accurate and reliable classifier.
  • Training Objective: The model is trained using a standard binary cross-entropy loss function, aiming to minimize the difference between predicted and actual outcomes.

The approach automates data integration and aims to capture complex patterns, offering advantages over traditional methods, especially with limited data.

Datasets

The study utilizes two large-scale, publicly available Electronic Health Record (EHR) datasets:

  • eICU Collaborative Research Database: A multi-center ICU database with high-resolution data from over 200,000 admissions across 335 units at 208 US hospitals. It includes vital signs, care plans, severity measurements, diagnostics, and treatments.
  • MIMIC-III (Medical Information Mart for Intensive Care III): A large, single-center database from Beth Israel Deaconess Medical Center, containing data from 2001 to 2012. It includes vital signs, medications, lab results, observations, diagnostic codes, and more.

These datasets are commonly used for EHR research and provide a robust benchmark for evaluating the proposed model. The primary outcomes of interest for this study were readmission rates and death rates.

Key Insights and Ideas

  • Deep Forest for EHRs: The paper demonstrates that a cascade of random forests (Deep Forest) can be effectively adapted to learn meaningful representations from complex EHR data, outperforming traditional methods.
  • Statistical Features Enhance Prediction: Incorporating statistical features alongside raw EHR data significantly improves the accuracy and reliability of diagnostic predictions, particularly for readmission and mortality.
  • Robustness with Limited Data: Patient Forest shows superior performance compared to baselines when training data is scarce, a common challenge in healthcare settings. This suggests its potential for scenarios where large, annotated datasets are unavailable.
  • Visualizing Learned Representations: The t-SNE visualization of the learned patient embeddings reveals clear clustering of patients based on their outcomes (e.g., death vs. no death). This qualitative assessment provides strong evidence that the model is learning discriminative and interpretable representations.

Key Results

The Patient Forest model consistently outperformed several baseline machine learning models (including Random Forest, MLP, LR, SVM, NB, CART, and K-NN) on both AUPRC and AUROC metrics across different data splits and datasets.

  • Predictive Performance:
    • Using a 75%:25% train-test split, Patient Forest achieved the highest AUPRC and AUROC scores for MIMIC Mortality, eICU Mortality, and eICU Readmission. For instance, it achieved 0.619 AUPRC and 0.8010 AUROC for MIMIC Mortality.
  • Even with reduced training data (e.g., 30% train-70% test split), Patient Forest maintained its superior performance, demonstrating robustness and generalizability.
  • Qualitative Evaluation: t-SNE plots of the learned patient embeddings for the MIMIC mortality prediction task showed clear separation between patient classes (death vs. no death), indicating that the model learns discriminative representations.

Limitations and Future Work

The authors acknowledge several limitations and propose future directions:

  • Temporal Properties: The current model does not explicitly incorporate the temporal dynamics of EHRs. Future work aims to extend the approach to handle time-series EHR data, which could be crucial for tasks like predicting patient deterioration.
  • Limited Data Integration: The study primarily used three main tables from MIMIC and eICU (diagnosis, lab results, and treatment). Future research plans to incorporate more tables, such as demographics and procedure data, for a more comprehensive representation.
  • Hyperparameter Optimization: Further research is needed to determine the optimal hyperparameters for Patient Forest to maximize its accuracy and performance.
  • Clinical Integration: The ultimate utility of this model in a clinical setting needs to be evaluated through real-world deployment and assessment by healthcare professionals.
  • Exploration in Other Areas: The authors suggest exploring Patient Forest's potential in other healthcare applications like disease stratification, diagnostics, and prognosis.

Summary updated

Abstract

An electronic health record (EHR) is a vital high-dimensional part of medical concepts. Discovering implicit correlations in the information of this data set and the research and informative aspects can improve the treatment and management process. The challenge of concern is the data sources’ limitations in finding a stable model to relate medical concepts and use these existing connections. This paper presents Patient Forest, a novel end-to-end approach for learning patient representations from tree-structured data for readmission and mortality prediction tasks. By leveraging statistical features, the proposed model is able to provide an accurate and reliable classifier for predicting readmission and mortality. Experiments on MIMIC-III and eICU datasets demonstrate Patient Forest outperforms existing machine learning models, especially when the training data are limited. Additionally, a qualitative evaluation of Patient Forest is conducted by visualising the learnt representations in 2D space using the t-SNE, which further confirms the effectiveness of the proposed model in learning EHR representations.
BibTeX
@inproceedings{khodadadi2023improving,
  author = {Atieh Khodadadi and Nima Ghanbari Bousejin and Soheila Molaei and Vinod Kumar Chauhan and Tingting Zhu and David A. Clifton},
  title = {{Improving Diagnostics with Deep Forest Applied to Electronic Health Records}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=yCRDEd7Kynj},
}
Figure