TL;DR
This paper presents a comprehensive two-year study on forecasting COVID-19 infections in Cyprus, integrating diverse data sources like epidemiological, vaccination, policy, and weather data. The research highlights the significant impact of combining these heterogeneous data streams, particularly epidemiological and policy data, for improving the accuracy of forecasting models like XGBoost.
What Problem Does This Paper Address?
The COVID-19 pandemic underscored the critical need for accurate and timely forecasting of infectious disease outbreaks. Traditional epidemiological models, while foundational, often struggle to capture the complex interplay of factors influencing disease spread in real-world scenarios. The rapid global transmission, coupled with the implementation of various interventions (lockdowns, travel restrictions) and external influences (vaccination rates, weather), created a dynamic environment that challenged existing forecasting methods. This paper addresses the challenge of building robust forecasting models by investigating how to effectively leverage heterogeneous data sources to improve prediction accuracy for COVID-19 infections, aiming to enhance preparedness and response strategies for future pandemics.
Methods
The study employs two primary forecasting models: XGBoost, an advanced gradient boosting algorithm known for its scalability and predictive accuracy, and ARIMAX, a classical statistical model that extends ARIMA by incorporating external variables.
Key methodological aspects include:
- Heterogeneous Data Integration: The core of the approach is the integration of diverse data types:
- Epidemiological Data: Daily case counts, hospitalizations, ICU admissions, and deaths.
- Vaccination Records: Weekly vaccination numbers per dose category.
- Policy Data: Government interventions captured by the Oxford COVID-19 Government Response Tracker (OxCGRT), focusing on the general stringency index.
- Weather Data: Daily average temperature, humidity, wind speed, and precipitation.
- Feature Engineering and Selection: Extensive feature engineering was performed, including:
- Lagged time series of positive cases.
- Trend indicators (upward/downward, case ratio).
- 7-day moving averages.
- Days of the week.
- Categorical features (e.g., age groups for infections).
- Log transformation for infection data to handle outliers and exponential growth.
- One-hot encoding for categorical variables.
- A rigorous feature selection process using Recursive Feature Elimination (RFE) and XGBoost's native Top Gain Ranking, along with manual selection to remove highly correlated features and identify the most impactful ones.
- Time Series Cross-Validation: To prevent overfitting and ensure temporal order, a rolling split strategy was used for cross-validation.
- Evaluation Metrics: The performance of the models was evaluated using:
- Mean Absolute Percentage Error (MAPE) for relative error.
- Mean Absolute Error (MAE) for absolute error.
- Aggregated metrics over the forecasting horizon.
Datasets
The primary dataset used in this study is a two-year collection of daily SARS-CoV-2 positive cases in Cyprus, spanning from October 1, 2020, to December 31, 2022. This dataset was augmented with:
- Epidemiological Data: Publicly available data on hospitalizations, deaths, ICU admissions, and intubations collected by the Ministry of Health of the Republic of Cyprus.
- Vaccination Records: Data from the National Open Data Portal of Cyprus.
- Policy Data: The Oxford COVID-19 Government Response Tracker (OxCGRT).
- Weather Data: Data obtained from Visual Crossing.
The study does not introduce a new benchmark dataset but rather utilizes publicly available data for a specific geographical region (Cyprus) over an extended period.
Key Insights and Ideas
- Heterogeneity is Key: The study strongly suggests that integrating diverse data sources significantly improves forecasting accuracy compared to using only raw infection data.
- Epidemiological and Policy Data Dominate: Epidemiological features (beyond raw cases) and the general stringency index of government policies were found to be highly influential in improving XGBoost's predictive performance.
- Vaccination Data's Limited Short-Term Impact: Aggregated vaccination data showed little predictive power for short-term infection forecasts. This is attributed to the time lag for vaccine efficacy, the focus on reducing severe illness rather than infection, and the aggregated nature of the data.
- Weather's Subtle but Important Role: Excluding weather data led to a notable decline in predictive performance, indicating its necessity in the final feature set.
- Model Performance Varies by Period: XGBoost tended to perform better during "wave" periods (high outbreaks), while ARIMAX showed an advantage during "non-wave" periods. This is likely due to ARIMAX's stationarity assumption being violated during the rapid, dynamic changes of outbreaks.
- Feature Importance Matters: A well-curated set of "important" features (epidemiological, policy, weather) significantly boosted XGBoost's performance compared to using only raw infection data.
Key Results
- XGBoost Outperforms ARIMAX in Many Scenarios: While both models were evaluated, XGBoost generally demonstrated superior performance, especially when incorporating a rich set of features.
- Significant Improvement with Feature Engineering: Removing infection-related features (beyond raw cases) led to a substantial deterioration in XGBoost's performance, highlighting the value of engineered features.
- Policy Stringency is Predictive: The general stringency index was identified as a valuable feature for improving XGBoost's predictive performance.
- Weather Data's Contribution: Excluding weather data resulted in a decline in predictive performance, confirming its importance.
- Specific Performance Metrics: While detailed tables are provided, a general observation is that MAPE values for XGBoost with important features were often in the range of 15-25% for various forecasting horizons, indicating a reasonable level of accuracy for short-to-medium term predictions. For instance, in Table VIII, XGBoost with "Important Feat. (excl. Vacc.)" achieved MAPE values as low as 13.6% for 7-day aggregated forecasts in the non-wave period.
Limitations and Future Work
Limitations:
- Vaccination Data Aggregation: The aggregated nature of vaccination data limited its predictive power for short-term forecasts.
- Policy Data Stability: In some periods, policy indices showed minimal variability, making them less informative for forecasting.
- ARIMAX Stationarity Assumption: ARIMAX's reliance on stationarity can limit its effectiveness during highly dynamic outbreak periods.
- Geographical Specificity: While the methodology is generalizable, the findings are tailored to the Cypriot context.
Future Work:
- Predicting Hospitalizations and ICU Admissions: Expanding the scope beyond infections to include these critical metrics would enhance preparedness for future outbreaks.
- Long-Term Vaccination Effects: Investigating the long-term impact of vaccination and immunity development.
- Hybrid Models: Exploring hybrid models that combine statistical and machine learning techniques to further enhance predictive accuracy.
- Investigating Other Data Sources: Incorporating additional data streams could further refine forecasting capabilities.
Summary updated