TL;DR
This paper introduces ARGO2, a novel two-step method that significantly improves the accuracy of regional influenza tracking by combining national and regional Google search data with traditional CDC surveillance data. ARGO2 achieves a 30% error reduction compared to the best existing methods, offering a more reliable and timely tool for public health decision-making.
What Problem Does This Paper Address?
Accurate and timely tracking of influenza epidemics at the regional level is crucial for public health agencies to make informed decisions, especially during outbreaks. While internet search data, like Google searches, holds promise for real-time monitoring, existing methods struggle with the complexity and lower quality of regional-level internet data. Traditional surveillance data from the CDC, while reliable, often has a lag of 1-2 weeks, making it less than ideal for immediate action. This gap leaves a need for methods that can effectively leverage internet data at a granular level to provide timely and accurate regional influenza insights.
Methods
ARGO2 employs a two-step approach:
- Step 1: Preliminary Regional Estimation: This step focuses on extracting relevant information from Google search data. It uses a $L_1$ regularized linear estimator to build a preliminary regional influenza estimate. A key innovation here is the use of a two-year sliding window for model training, which helps capture evolving search patterns and epidemic dynamics. The $L_1$ penalty automatically filters out noisy or irrelevant search terms, addressing the issue of low-quality regional data. Additionally, a national-level estimate is generated using the ARGO method.
- Step 2: Cross-Regional Boosting: This step refines the preliminary estimates by incorporating multi-resolution and time-series information. It uses a best linear predictor framework that combines:
- The time-series increments of influenza-like illness (ILI) from the previous week.
- The preliminary regional ILI estimates from Step 1.
- The accurate national ILI estimate from Step 1. This step also explicitly models cross-regional dependencies and temporal trends, further enhancing accuracy. The method assumes a structured covariance matrix for efficient estimation, leveraging empirical evidence and statistical assumptions.
Datasets
- Google Trends Data: Publicly available Google search query data at both national and regional (state-level, aggregated to regional) resolutions was used.
- Google Flu Trends (GFT) Data: Discontinued GFT data was used for benchmarking purposes.
- CDC's Influenza-like Illness (ILI) Surveillance Network (ILINet) Data: This is the traditional surveillance data, providing weekly %ILI at national and US Health and Human Services (HHS) regional levels. This data served as the ground truth for training and evaluation.
The study uses data for the period of March 29, 2009, to March 17, 2018, covering multiple flu seasons, including the H1N1 outbreak.
Key Insights and Ideas
- Regional Data Quality is a Major Hurdle: The paper highlights that while internet search data is abundant, its quality significantly degrades at the regional level compared to national data, posing a substantial challenge for accurate tracking.
- $L_1$ Regularization as a Data Cleaner: The use of $L_1$ regularization in the first step is a clever way to automatically select relevant search terms and discard noisy ones, effectively improving the quality of the input data for regional modeling.
- Leveraging National Data as a Baseline: Using the national-level estimate as a baseline in the second step provides a robust anchor, especially when regional data is sparse or less reliable.
- Cross-Regional Dependencies are Key: Explicitly modeling the correlations between different regions is crucial for accurate regional forecasting, as influenza spread is inherently a spatial phenomenon.
- Robustness Across Flu Seasons: ARGO2 demonstrates consistent performance across different flu seasons, including irregular ones like H1N1, suggesting its reliability beyond typical seasonal patterns.
Key Results
- Significant Error Reduction: ARGO2 achieved over 30% error reduction in Mean Squared Error (MSE) compared to the best alternative method (VAR) over the entire study period (2009-2018).
- Outperforms Naive Methods: ARGO2 consistently outperformed the "naive" persistence forecast method (using the previous week's data as the current estimate), which is a surprisingly difficult benchmark to beat for regional tracking.
- Competitive Performance: Across various metrics (MSE, MAE, MAPE, correlation), ARGO2 generally performed best or competitively with other methods, particularly in MSE and MAE.
- Reliable Confidence Intervals: The method provides 95% confidence intervals for its estimates, with an average coverage of 94.96% across the ten US HHS regions, indicating good reliability.
- Consistent Regional Performance: Detailed analysis showed ARGO2 provided robust estimation across all ten US HHS regions, with significant error reduction in most cases.
Limitations and Future Work
- CDC %ILI as a Proxy: The authors acknowledge that CDC's %ILI is a proxy for actual flu incidence and can be susceptible to sampling bias, especially at the regional level due to smaller sample sizes.
- Tracking Rare Events at the Regional Level: The high sparsity of regional Google search data might limit ARGO2's effectiveness in tracking rare regional disease events.
- Future Work Suggestions:
- Extend ARGO2 by incorporating alternative data sources like electronic health records or Twitter data to improve regional rare disease tracking.
- Adapt the ARGO2 framework to track or forecast other diseases and public health/social events that leave footprints on internet user activity.
- The framework's flexibility allows for the integration of other models or data sources into the first step, potentially leading to further improvements.
Summary updated