TL;DR
This paper presents machine learning models to predict Ebola patient outcomes, developed from a large, harmonized dataset. These models, integrated into a mobile app, aim to improve clinical decision-making in resource-limited settings, demonstrating that ML can effectively capture clinical intuition and provide actionable guidance.
What Problem Does This Paper Address?
The 2014-2016 Ebola outbreak highlighted the critical need for field-deployable tools to manage the disease, especially in resource-scarce environments. Existing clinical guidelines, while valuable, are often difficult to access and use on mobile devices. Furthermore, the heterogeneity of data collected across different treatment centers during outbreaks poses a significant challenge for developing generalizable predictive models. This paper tackles these issues by developing and validating machine learning prognostic models for Ebola, harmonizing diverse datasets, and integrating them into an mHealth application to support frontline healthcare workers.
Methods
The authors employed multivariate logistic regression to build prognostic models. A key methodological challenge was data harmonization, where data from different Ebola Treatment Units (ETUs) with varying collection protocols were standardized. They used techniques like intra-site normalization for PCR data (CT values) and multiple imputation to handle missing values, particularly for variables like body temperature. Variable selection was performed using penalized logistic regression (Elastic Net) to identify the most predictive features. The models were then internally validated using bootstrap resampling and externally validated on independent datasets.
Datasets
The primary dataset used for training and internal validation comprised 470 confirmed EVD cases collected from five ETUs operated by International Medical Corps (IMC) in Sierra Leone and Liberia between September 2014 and September 2015. This is noted as the largest and most diverse clinical EVD dataset available at the time.
For external validation, two independent datasets were used:
- KGH dataset: 106 Ebola-positive patients treated at Kenema Government Hospital (KGH) in Sierra Leone during the early months of the outbreak.
- GOAL dataset: 158 Ebola patients treated at a GOAL Global ETU in Port Loko, Sierra Leone, during the later months of the epidemic.
Key Insights and Ideas
- ML can approximate clinical intuition: The study found that machine learning models, when properly designed and applied to rich data, can approximate the "gestalt" or clinical intuition that experienced physicians develop. This is crucial for situations where expert experience is limited.
- Data harmonization is critical for generalizability: The success in building models that performed well across different sites and time points underscores the importance of robust data harmonization techniques in outbreak settings.
- mHealth as a powerful delivery mechanism: Integrating prognostic models into a mobile app makes complex clinical knowledge actionable and accessible to frontline healthcare workers, especially in remote areas.
- Predictors of mortality: Viral load and patient age emerged as the strongest predictors of death, but a parsimonious model including clinical signs and symptoms also showed good performance.
- Wellness scale as a valuable predictor: Incorporating an observational "wellness scale" (WS) into models proved to be at least as effective, and sometimes superior, to models relying solely on detailed clinical features.
Key Results
- The developed prognostic models achieved an Area Under the Curve (AUC) ranging from 0.7 to 0.8, with overall accuracy between 64% and 74%.
- The parsimonious model, incorporating viral load, age, body temperature, bleeding, jaundice, dyspnea, dysphagia, and referral time, performed comparably to observational assessments by experienced health workers.
- External validation on the GOAL dataset showed AUCs of 0.82, 0.84, 0.73, and 0.82 for the parsimonious, parsimonious without temperature, clinical-only, and minimal models, respectively.
- The "Ebola Care Guidelines" app was developed, integrating these models with evidence-based care recommendations, designed for offline use after initial installation.
Limitations and Future Work
- Data limitations: Despite being the largest study of its kind, the amount and quality of available clinical data remain limited. Variations in clinical assessments, data collection errors, and differences in lab protocols can still affect data accuracy.
- Generalizability: While efforts were made to harmonize data and validate models externally, achieving truly generalizable models across all contexts remains a challenge. Future work should focus on larger and better datasets with consensus mechanisms for data consistency.
- Model updates: The authors envision a future where apps can retrieve updated models from a remote server, eliminating the need for app updates and creating a positive feedback loop where improved data generation leads to more precise models.
- Broader application: The approach is generalizable and can be applied to develop similar mHealth tools for other tropical diseases in rural and low-resource areas.
Summary updated