TL;DR
This paper introduces a novel evolutionary computation framework to automatically optimize how multilevel patient data (like hierarchical medical codes) is used for predictive modeling. By intelligently selecting the right level of detail from complex patient records, their method significantly improves the prediction of critical outcomes for emergency department patients compared to traditional approaches.
What Problem Does This Paper Address?
The increasing availability of Electronic Health Records (EHRs) has opened doors for powerful predictive analytics in healthcare. However, EHR data is often "multilevel," meaning information exists at various granularities (e.g., broad disease categories vs. specific diagnoses). This hierarchical nature presents a significant challenge: how do we best leverage this data for accurate predictions? Simply using the most general or most specific information might miss crucial predictive signals. Traditional methods often rely on manual, hypothesis-driven choices for data specificity, which may not be optimal and can be difficult to adapt across different patient populations or outcomes. This paper aims to solve this by developing an automated, data-driven approach to find the optimal way to represent and utilize multilevel patient data for prediction.
Methods
The core of the paper is a general evolutionary computation (EC) framework, specifically a Genetic Algorithm (GA). This GA is designed to search for the optimal combination of patient data features, particularly focusing on chief complaints in their case study.
Here's how it works:
- Candidate Solutions: Each potential solution is represented as a binary bit string. Each bit corresponds to a specific complaint, indicating whether it's selected (1) or excluded (0) for use in the predictive model.
- Two Approaches:
- Flattening: This approach collapses the hierarchical structure into a single level. If a specific complaint is selected, it's removed from its broader category, creating a mutually exclusive representation.
- Hierarchical: This approach retains the hierarchical structure. If a specific complaint is selected, its parent categories are also retained.
- Fitness Function: The "fitness" of each candidate solution is evaluated using 5-fold cross-validated Area Under the Receiver Operating Characteristic Curve (AUC). This measures how well a predictive model (logistic regression in this case) built with the selected features predicts critical outcomes.
- Evolutionary Process: The GA iteratively refines its population of solutions through:
- Selection: Tournament selection picks the fittest solutions.
- Crossover: Uniform crossover combines parts of parent solutions.
- Mutation: Bit-flip mutation randomly inverts bits, introducing new variations.
- Baseline Models: The performance of the GA-optimized models is compared against two simpler baseline models: one using only specific complaints and another using only categorized complaints.
Datasets
The study uses data from five different patient populations to predict critical care events (in-hospital mortality, direct ICU admission, or emergent surgery/catheterization). These populations include:
- A large, urban academic medical center (ACAD)
- A medium-sized community hospital (COMM)
- International hospitals in Brazil (BRAZIL) and the United Arab Emirates (UAE)
- The nationally representative National Hospital Ambulatory Medical Care Survey (NAT)
The primary multilevel data feature optimized was chief complaints, alongside other standard predictors like age, gender, arrival mode, and vital signs (which were discretized).
Key Insights and Ideas
- Automated Feature Engineering: The EC framework automates the complex and often subjective process of selecting the optimal level of specificity for multilevel data, moving beyond manual, hypothesis-driven approaches.
- Hierarchical vs. Flattened Structures: Both the flattened and hierarchical approaches to representing multilevel data yielded significant improvements over baseline models, suggesting that intelligently structuring this data is key. The hierarchical approach was notably faster to train.
- Subgroup Performance Matters: The improvements weren't just at the overall prediction level; the models showed significant gains for specific patient subgroups (e.g., those directly affected by selected complaints), indicating a more nuanced and robust prediction.
- Interpretability: The selected complaints provide valuable insights into what factors are most predictive of critical outcomes, offering practitioners actionable information beyond just a prediction score.
- Generalizability: The framework is designed to be flexible and adaptable to various predictive modeling tasks and different types of multilevel data within healthcare and beyond.
Key Results
- Both the flattened and hierarchical GA approaches demonstrated statistically significant improvements in overall 5-fold cross-validated AUC compared to baseline models (p < 0.001).
- Significant improvements were observed for inner and middle bullseye subgroups (patients most directly and indirectly affected by the GA's feature selection) relative to baseline models.
- The hierarchical approach was significantly faster to train (e.g., ~3-8 hours vs. ~15-78 hours for the flattened approach across populations).
- Despite performance similarities, the selected complaints between the flattened and hierarchical approaches showed only ~55-60% agreement, highlighting how different structural representations can lead to different feature selections.
- The study found minimal differences in overall AUC performance between the flattened and hierarchical approaches across the five populations.
Limitations and Future Work
- Stochastic Nature: The authors acknowledge the stochastic nature of EC, but note that top candidate solutions tend to be consistent across runs, suggesting reliability.
- Uniquely Selected Complaints: It's unclear if complaints uniquely selected by one approach (flattened vs. hierarchical) are truly meaningful or just artifacts of the GA's stochasticity.
- Hybrid Approaches: The authors suggest future work could explore hybrid solutions, such as selecting complaints only if they are jointly selected by both flattened and hierarchical approaches, or if they meet a minimum sample size requirement.
- Preprocessing: The framework could incorporate additional preprocessing steps before fitness computation, similar to the dynamic restructuring seen in the flattening approach.
- Adaptability: While the framework is presented as adaptable, further research could explore its application to different evolutionary algorithms, fitness functions, and prediction tasks.
Summary updated