Assessing the Predictability of Hospital Readmission Using Machine Learning

Arian Hosseinzadeh, Masoumeh T. Izadi, Aman Verma, Doina Precup, David L. BuckeridgeIAAI 2013 · DOI

TL;DR

This paper tackles the challenge of predicting unplanned hospital readmissions, a major cost driver in healthcare. By applying machine learning to a rich administrative dataset from Quebec, the authors demonstrate that a significant portion of readmissions are inherently difficult to predict, questioning the sole use of raw readmission rates as a quality indicator.

What Problem Does This Paper Address?

Unplanned hospital readmissions are a significant financial burden and a source of distress for patients. Identifying patients at high risk of readmission is crucial for optimizing care and resource allocation. While various factors are known to influence readmissions (patient demographics, chronic conditions, post-discharge care coordination, medical errors), accurately predicting them remains a challenge. Existing methods, often relying on traditional statistical approaches like logistic regression, struggle with the sheer volume and complexity of data available from modern electronic health records. This paper aims to leverage machine learning to better understand the determinants of readmission and assess the predictability of these events, particularly questioning the appropriateness of using raw readmission rates as a sole measure of hospital quality.

Methods

The authors employ supervised machine learning algorithms to predict hospital readmissions within 30 days of discharge. Their approach involves:

  • Feature Engineering: They utilize a vast array of administrative data, including patient demographics, dispensed drugs (grouped by AHFS classification), medical/surgical procedure codes (using Canadian Classification of Diagnostic, Therapeutic, and Surgical Procedures), and diagnostic codes (ICD9, grouped by Elixhauser comorbidities).
  • Dimensionality Reduction: Given the high dimensionality (over 20,000 features), they explore various feature selection techniques to reduce the number of predictors while maintaining interpretability. These include:
  • Information-based: Gini Indexing.
  • Frequency-based: Ranking features by frequency counts.
  • Domain-knowledge-based: Utilizing medical taxonomies (AHFS, Canadian Classification, ICD9) to group features.
  • Classification Algorithms: They compare two common algorithms:
  • Naive Bayes: A generative classifier.
  • Decision Tree: A discriminative classifier.
  • Handling Class Imbalance: Down-sampling of the majority (non-readmitted) class was applied to the training set to address class imbalance.
  • Identifying "Hard-to-Predict" Cases: A key aspect of their methodology is identifying cases that are consistently missed by various classifiers and feature sets, hypothesizing these represent truly difficult-to-predict or potentially unavoidable readmissions.

Datasets

The study utilizes a large administrative dataset from the Quebec healthcare system, obtained from the Regie de l'assurance maladie du Quebec (RAMQ). This dataset includes:

  • Discharges from twenty hospitals for patients aged 65 or older diagnosed with a respiratory illness between 1996 and 2006.
  • Information on age, sex, census tract, hospital procedures, diagnostic codes, outpatient physician visits, and all prescription drugs (due to universal coverage for those 65+).
  • The dataset comprises approximately 619,274 hospital discharges, with over 20,000 features.

Key Insights and Ideas

  • Predictability Limits: A substantial portion of hospital readmissions appear to be inherently difficult to predict, even with a rich administrative dataset and advanced machine learning techniques. This challenges the notion that all readmissions are preventable or indicative of poor care.
  • Raw Readmission Rate as Quality Indicator: The findings suggest that using the raw readmission rate as a sole indicator of hospital quality might be inappropriate, as it doesn't account for the inherent unpredictability of certain cases.
  • Value of Medical Taxonomies: Domain-knowledge-based feature grouping (using medical classification systems like AHFS) is an effective method for dimensionality reduction, offering a way to manage large numbers of infrequent features without significant loss in prediction performance.
  • "Hard-to-Predict" Cases: Identifying these outlier cases is important. They may represent patients with complex underlying conditions or situations where current data is insufficient to predict readmission. Further analysis of these cases could reveal new insights.
  • Dataset Impact: The choice of dataset (original vs. outlier-cleared) had a more significant impact on prediction performance than the specific feature selection method used.

Key Results

  • Performance Metrics: The study reports Area Under the ROC Curve (AUC) values for prediction performance.
  • With the original dataset, AUC values ranged from 0.63 to 0.67 for Naive Bayes and Decision Tree classifiers across different feature reduction methods.
  • After removing the "hard-to-predict" outliers, AUC values improved to 0.82 to 0.84.
  • Feature Selection Comparison:
    • On the original dataset, frequency-based feature selection generally outperformed Gini Indexing.
  • After removing outliers, Gini Indexing showed a slight advantage over frequency-based selection.
  • Domain-knowledge-based grouping performed comparably to statistical methods when the same number of features were selected.
  • Outlier Identification: Approximately 6% of the entire dataset was identified as "hard-to-predict" cases.
  • Descriptive Statistics: Table 3 highlights some differences between readmitted patients, non-readmitted patients, and the identified outliers, showing variations in age, emergency admission rates, length of stay, and previous readmissions.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that not all readmission records can be treated equally, and the identified "outliers" may represent unavoidable readmissions.
  • The dataset does not include hospital names or locations, preventing differentiation between Montreal and other Quebec hospitals.
  • The definition of readmission time frame (30 days) is a subject of ongoing debate.
  • Future Work:
    • Extend the analysis to include different feature reduction algorithms and further experimental studies.
  • Investigate shorter and longer readmission time frames.
  • Identify and assess different types of patients from the perspective of readmission and explore the causality of readmission.
  • Develop software that interfaces with electronic health records for real-time prediction.
  • Further analyze the characteristics of "hard-to-predict" cases to gain deeper insights.

Summary updated

BibTeX
@inproceedings{hosseinzadeh2013assessing,
  author = {Arian Hosseinzadeh and Masoumeh T. Izadi and Aman Verma and Doina Precup and David L. Buckeridge},
  title = {{Assessing the Predictability of Hospital Readmission Using Machine Learning}},
  booktitle = {IAAI},
  year = {2013},
  doi = {10.1609/aaai.v27i2.18995},
}
Figure