Who Gets the Flu? Individualized Validation of Influenza-like Illness in Urban Spaces

Shiran Zhong, Fenglong Ma, Jing Gao, Ling BianInternational Journal of Environmental Research and Public Health 2023 · DOI

TL;DR

This paper tackles the challenge of validating influenza-like illness (ILI) models at the individual level, a crucial but often overlooked step. By developing and applying an ensemble machine learning approach to smartphone survey data, they demonstrate that individual-scale validation is achievable and reveals the distinct impacts of various urban environments (home-work, service, ambient) and demographics on ILI risk.

What Problem Does This Paper Address?

Predicting the spread of communicable diseases like influenza in urban environments is complex. While many models exist, they often struggle with validation. Specifically, most models are validated at a coarse, population level, even if they predict individual outcomes. This "scale mismatch" means we can't be sure if the factors used in the models (like individual behaviors or environmental conditions) truly influence health at the individual level as intended. This gap hinders our ability to accurately assess individual vulnerability and develop targeted interventions. The paper aims to bridge this gap by performing individualized validation of ILI symptoms.

Methods

The core of the study is an ensemble approach designed to handle imbalanced data (many more instances of no ILI symptoms than ILI symptoms). This approach integrates three key components:

  1. Random Forest as the Frame: This provides a robust framework for combining multiple predictive models.
  2. Support Vector Machines (SVMs) as Building Blocks: SVMs are used as the individual classifiers within the ensemble, chosen for their ability to handle dichotomous classifications (presence/absence of ILI).
  3. "Train-Small-Evaluate-Big" Strategy: This novel training strategy is crucial for dealing with data imbalance. It involves:
  4. Iterative Partitioning: The training data is repeatedly split into smaller subsets.
  5. Balanced Subsets: For each subset, positive (ILI) instances are bootstrapped, and negative (no ILI) instances are randomly sampled to create balanced mini-datasets.
  6. Preliminary Classifiers: An SVM is trained on each balanced subset.
  7. Candidate Selection: The best-performing SVM from each subset (based on evaluation on the full training data) is selected as a candidate.
  8. Ensemble Creation: The collection of best candidate classifiers forms the ensemble.
  9. Weighted Majority Voting: The final prediction is made by combining the predictions of the ensemble members, with stronger classifiers receiving higher weights.

This ensemble method is compared against six benchmark machine learning models (logistic regression, standalone SVM, decision tree, classic random forest, ANN, and CNN) to demonstrate its effectiveness.

Datasets

The study utilizes data collected through a smartphone-based survey conducted weekly during the 2016-2017 influenza season in a metropolitan area in the northeastern U.S.

  • Individualized Health Outcome Data: Weekly reports on the presence or absence of ILI symptoms (fever, cough, sore throat, etc.). This serves as the ground truth for validation.
  • Individualized Attribute and Behavior Data: Information on demographics, exposure environments, mobility, interactions, and protective actions.
  • Ambient Environmental Data: Meteorological data (temperature) and air quality index (PM2.5, O3, NO2, CO) from local monitoring stations.

The dataset comprises 1485 individuals with a total of 25,221 individual-week records. The data was collected over an influenza season (more than seven months).

Key Insights and Ideas

  • Individual-Scale Validation is Feasible and Necessary: The study provides strong evidence that validating ILI models at the individual level is achievable and yields valuable insights that coarse-scale validation misses. This opens the door for validating many existing and future individual-level models.
  • Urban Spaces Matter Differently: The research highlights that different urban spaces (home-work, service, ambient environment) have distinct impacts on ILI risk. The home-work space emerges as the most significant factor, followed by service spaces, underscoring the importance of daily routines and social interactions.
  • Compound Effects of Urban Spaces: The combination of home-work and service spaces shows a synergistic effect, meaning their combined impact is greater than the sum of their individual impacts. This suggests that the interconnectedness of these spaces amplifies transmission risks.
  • Ambient Environment Amplifies Risks: The ambient environment (temperature, air quality) doesn't just have a direct effect; it significantly amplifies the risks associated with the home-work and service spaces, particularly during colder months when indoor activities increase.
  • Demographics Play a Role, But Differently: While demographics are considered, their impact is less pronounced than spatial factors. Notably, gender showed no significant difference, but household size was significant, suggesting larger households may facilitate transmission.

Key Results

  • High Individual-Scale Validation Accuracy: The ensemble approach achieved an overall accuracy of 84.1% for predicting ILI symptoms at the individual level.
  • Robustness to Imbalanced Data: The F1 score for minority presence (ILI symptoms) was 73.2%, which is considered satisfactorily high given the ~1:9 ratio of presence to absence records. This significantly outperforms benchmark models, which ranged from 55.0% to 68.4% for this metric.
  • Factor Contributions:
    • Separate Contributions: Home-work space (8.2%), Service space (5.6%), Ambient environment (4.7%), Demographics (4.0%).
  • Compound Contributions: The combination of home-work and service spaces showed a compound contribution of 20.5%, with an "extra" contribution of 6.7% beyond their separate impacts. The combination of all three spaces (home-work, service, ambient) yielded the highest compound contribution at 40.5%.
  • Model Performance: The ensemble approach consistently outperformed benchmark models, especially in predicting the minority class (ILI presence). The CNN model did not converge, highlighting the challenges of complex models with limited data.

Limitations and Future Work

  • Data Imbalance Handling: While the ensemble approach is effective, the authors acknowledge that advanced deep learning methods (like CNN) could potentially offer even better performance but require more data to converge.
  • Smartphone Survey Trade-offs: The weekly data collection, while practical, compromises the ability to capture real-time intervention strategies that daily data might provide. Future advancements in smartphone technology could address this.
  • Factor Exploration: The study focused on four sets of factors. The authors suggest ample room for further exploration of transmission-driving factors beyond those considered, particularly those at the individual level.
  • Temporal Resolution of Demographics: Demographic factors are temporally invariant, which might limit their explicit contribution to dynamic predictions.
  • Future Directions: The study lays the groundwork for validating individual-level models and understanding the complex interplay between urban spaces and health. Future work could involve exploring more advanced modeling techniques, incorporating a wider range of factors, and leveraging increasingly available fine-scaled health data to inform public health policies and improve urban livability.

Summary updated

Abstract

Urban dwellers are exposed to communicable diseases, such as influenza, in various urban spaces. Current disease models are able to predict health outcomes at the individual scale but are mostly validated at coarse scales due to the lack of fine-scaled ground truth data. Further, a large number of transmission-driving factors have been considered in these models. Because of the lack of individual-scaled validations, the effectiveness of factors at their intended scale is not substantiated. These gaps significantly undermine the efficacy of the models in assessing the vulnerability of individuals, communities, and urban society. The objectives of this study are twofold. First, we aim to model and, most importantly, validate influenza-like illness (ILI) symptoms at the individual scale based on four sets of transmission-driving factors pertinent to home-work space, service space, ambient environment, and demographics. The effort is supported by an ensemble approach. For the second objective, we investigate the effectiveness of the factor sets through an impact analysis. The validation accuracy reaches 73.2-95.1%. The validation substantiates the effectiveness of factors pertinent to urban spaces and unveils the underlying mechanism that connects urban spaces and population health. With more fine-scaled health data becoming available, the findings of this study may see increasing value in informing policies that improve population health and urban livability.
BibTeX
@article{zhong2023who,
  author = {Shiran Zhong and Fenglong Ma and Jing Gao and Ling Bian},
  title = {{Who Gets the Flu? Individualized Validation of Influenza-like Illness in Urban Spaces}},
  journal = {International Journal of Environmental Research and Public Health},
  year = {2023},
  doi = {10.3390/ijerph20105865},
}
Figure