Machine Learning–Based Prediction of Growth in Confirmed COVID-19 Infection Cases in 114 Countries Using Metrics of Nonpharmaceutical Interventions and Cultural Dimensions: Model Development and Validation

Arnold Ys Yeung, François Roewer-Després, Laura C. Rosella, Frank RudziczJournal of Medical Internet Research 2021 · DOI

Abstract

This work provides new considerations in using machine learning techniques with nonpharmaceutical interventions and cultural dimensions as metrics to predict the national growth of confirmed COVID-19 infections. By integrating diverse datasets across 114 countries, the study demonstrates that sociocultural norms are powerful predictors of transmission dynamics, often offering more predictive insight than specific government policies alone.

Introduction

Early in the COVID-19 pandemic, forecasting relied heavily on traditional epidemiological models, such as the Susceptible-Infectious-Recovered (SIR) framework. While effective, these models require specific assumptions about transmission rates ($R_0$) that can be difficult to estimate in real-time. Machine learning (ML) emerged as a flexible alternative, yet early efforts were often limited to single-country analyses or relied on narrow features like local weather or internet search trends.

A critical gap identified by researchers was the varying effectiveness of Nonpharmaceutical Interventions (NPIs)—such as school closures, workplace restrictions, and travel bans—across different nations. The authors hypothesized that this variance is driven by "cultural dimensions," the deep-seated sociocultural norms that dictate how a population responds to authority, collective rules, and social distancing. This study explores a novel approach by combining NPI data with national cultural metrics to improve global pandemic forecasting.

Methodology and Data Integration

The researchers developed a robust data pipeline to predict Confirmed Infection Growth (CIG), defined as the percentage change in cumulative cases over a 14-day window. The study integrated three primary data sources to create a comprehensive feature set:

  1. OxCGRT (Oxford COVID-19 Government Response Tracker): Daily metrics for 17 NPI categories, including workplace closures, testing policies, and stay-at-home requirements.
  2. CSSE (Johns Hopkins University): Daily reported infection and mortality statistics used to calculate the target labels.
  3. Hofstede Cultural Dimensions: A framework consisting of six metrics—Power Distance, Individualism, Masculinity, Uncertainty Avoidance, Long-term Orientation, and Indulgence—that describe a nation's core values.
The data processing pipeline involves merging NPI, infection, and cultural datasets followed by rigorous feature engineering and filtering.

The authors evaluated five non-time series regression models: Ridge Regression, Decision Tree, Random Forest, Adaptive Boosting (AdaBoost), and Support Vector Regression (SVR). Non-time series models were chosen to avoid the need for long, accurate historical trajectories, which were often unreliable during the early stages of the outbreak. This allowed the models to learn from granular, point-in-time feature sets.

Validation Strategy

To ensure the models were robust and generalizable, the researchers employed a rigorous three-tier validation strategy:

  • In-distribution (Interpolation): Randomly splitting data across all dates and countries to test basic predictive capability.
  • Out-of-distribution (Extrapolation): Training on early data and testing on later dates to simulate real-world forecasting of future trends.
  • Country-based Cross-validation: Training on one set of countries and testing on entirely different ones to determine if the model could generalize across borders and different cultural contexts.

Key Insights: The Role of Culture

Using Mutual Information (MI) analysis, the study revealed that cultural dimensions often had higher predictive power than individual NPIs. Specifically, dimensions like "Long-term orientation" and "Indulgence" were found to be stronger signals for infection growth than policies like "Income support" or "Debt relief." This suggests that the success of a government intervention is heavily mediated by the underlying cultural fabric of the society it is applied to.

Results and Performance

Random Forest and AdaBoost emerged as the top-performing models across the validation scenarios. The results highlighted a significant "use case" performance gap:

  • In-distribution: The models achieved a high $R^2$ of 0.959, capturing nearly all variance when interpolating between known data points.
  • Out-of-distribution: Performance dropped to an $R^2$ of 0.513, reflecting the inherent difficulty of predicting future trends as pandemic dynamics evolve over time.
  • Country-based: The models achieved an $R^2$ of 0.574, confirming that cultural dimensions significantly help the model generalize to countries not included in the training set.

While the models were well-calibrated (with slopes near 1.0), they showed a slight tendency to overestimate growth in scenarios with lower infection ranges.

Limitations and Future Directions

The study acknowledges several challenges, including the imprecision of NPI data (which tracks the existence of a policy rather than the rigor of its enforcement) and potential reporting biases in confirmed case counts. Furthermore, cultural dimensions are measured at the national level, which may overlook significant sub-national or regional variations.

Future research could refine these models by incorporating real-time mobility data and more granular, sub-national cultural metrics. Despite these hurdles, this work demonstrates that sociocultural factors are indispensable for understanding and predicting the global trajectory of infectious diseases.

Summary updated

Abstract

This work provides new considerations in using machine learning techniques with nonpharmaceutical interventions and cultural dimensions as metrics to predict the national growth of confirmed COVID-19 infections.
BibTeX
@article{yeung2021machine,
  author = {Arnold YS Yeung and François Roewer-Després and Laura C. Rosella and Frank Rudzicz},
  title = {{Machine Learning–Based Prediction of Growth in Confirmed COVID-19 Infection Cases in 114 Countries Using Metrics of Nonpharmaceutical Interventions and Cultural Dimensions: Model Development and Validation}},
  journal = {Journal of Medical Internet Research},
  year = {2021},
  doi = {10.2196/26628},
}
Figure