Estimating the COVID-19 Prevalence in Spain with Indirect Reporting via Open Surveys

Augusto García-Agúndez, Oluwasegun Ojo, Harold Hernández, Carlos Baquero, Davide Frey, Chryssis Georgiou, Mathieu Goessens, Rosa E. Lillo, Raquel Menezes, Nicolas Nicolaou, Antonio Ortega, Efstathios Stavrakis, Antonio Fernández Anta

TL;DR

This paper introduces "CoronaSurveys," an innovative approach using open, anonymous online surveys with indirect reporting to estimate COVID-19 prevalence. By comparing their results to a gold-standard serology study in Spain, they demonstrate high accuracy (R²=0.89), suggesting this method can effectively track pandemic progress.

What Problem Does This Paper Address?

Accurately tracking the spread of infectious diseases, especially in the early stages of a pandemic like COVID-19, is incredibly challenging. Official case counts often underestimate the true prevalence due to limited testing capacity, asymptomatic cases, and reporting delays. Traditional methods like massive serology testing are expensive and time-consuming, making them impractical for real-time monitoring. The paper addresses this by proposing and validating an alternative: using crowdsourced data from open online surveys, but with a clever twist – indirect reporting. This method aims to overcome the limitations of direct reporting (where individuals report on their own health) by asking participants about their social network, which can provide a more robust and less biased estimate of community-level prevalence.

Methods

The core of the CoronaSurveys method lies in its two-question survey:

  1. Reach (ri): "How many people do you know in your area for which you know their health condition?"
  2. Cumulative Number of Cases (ci): "How many of those were diagnosed with or have symptoms of COVID-19?"

This "indirect reporting" approach leverages the network scale-up method. Instead of asking individuals if they are sick, it asks them to report on their acquaintances. This is theorized to reduce social desirability bias and capture a broader spectrum of cases, including asymptomatic ones. The data is collected via a web or mobile app, hosted on a private LimeSurvey instance for anonymity, and aggregated daily. Outlier responses are removed based on specific criteria (e.g., unusually high reach or case ratios). The final prevalence estimate for a region is calculated as the sum of reported cases divided by the sum of reported reach across all participants in that region (∑ci / ∑ri).

Datasets

The primary dataset used for evaluation is the ENE-COVID study, a large-scale, population-based seroepidemiological study conducted by the Spanish government between April 27 and May 11, 2020. This study serves as the "gold standard" for comparison. The CoronaSurveys project itself collected data through its open online surveys, promoted via social networks and paid advertising.

Key Insights and Ideas

  • Indirect Reporting as a Powerful Tool: The study strongly suggests that indirect reporting via social networks can be a surprisingly accurate method for estimating disease prevalence, even in the early, chaotic phases of a pandemic.
  • Underestimation by Official Counts: The paper highlights a significant discrepancy between officially reported COVID-19 cases and deaths in Spain and estimates derived from both their survey and serology studies, underscoring the problem of underreporting.
  • Asymptomatic Cases Matter: The consistent underestimation by CoronaSurveys (by about 46%) is attributed, in part, to asymptomatic cases, a finding corroborated by the ENE-COVID study itself. This means even with high testing, a significant portion of infections might be missed.
  • Simplicity Drives Engagement: The survey's brevity (only two questions) was crucial for maximizing user interest and ensuring consistent data collection, yielding hundreds of responses per week.
  • Regional Data is Key: Aggregating data by region proved effective, and the analysis showed that even a moderate number of responses per region (around 50) could provide a reasonable estimation of cases.

Key Results

The CoronaSurveys estimates showed a high correlation (R² = 0.8994) with the ENE-COVID serology study data. While CoronaSurveys consistently underestimated the prevalence by approximately 46%, this underestimation was found to be homogeneous across regions. This suggests that the method, while not perfectly capturing every case, provides a reliable and consistent signal of pandemic progression that can be adjusted. The study also noted that the number of replies per region significantly impacts the accuracy, with around 50 replies yielding a reasonable R² value.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Responses in Some Regions: For certain regions in Spain (e.g., La Rioja, Navarra, Cantabria), the number of responses was too low to provide reliable data.
  • Heuristic Outlier Criteria: The criteria used to eliminate outlier responses are heuristic and may need refinement as more data becomes available.
  • Potential for Bias: While indirect reporting aims to reduce bias, it's not entirely immune. Factors like social network structure and participation bias in online surveys could still influence results.

Future work includes evaluating the robustness of their model against additional serology studies and further refining the model to provide more accurate cumulative case counts for each region. The paper concludes by strongly advocating for the use of open surveys with indirect reporting as a valuable tool for broad pandemic sensing.

Summary updated

Abstract

ABSTRACT During the initial phases of the COVID-19 pandemic, accurate tracking has proven unfeasible. Initial estimation methods pointed towards case numbers that were much higher than officially reported. In the CoronaSurveys project, we have been addressing this issue using open online surveys with indirect reporting. We compare our estimates with the results of a serology study for Spain, obtaining high correlations (R squared 0.89). In our view, these results strongly support the idea of using open surveys with indirect reporting as a method to broadly sense the progress of a pandemic.
BibTeX
@inproceedings{garciaagundez2021estimating,
  author = {Augusto García-Agúndez and Oluwasegun Ojo and Harold Hernández and Carlos Baquero and Davide Frey and Chryssis Georgiou and Mathieu Goessens and Rosa E. Lillo and Raquel Menezes and Nicolas Nicolaou and Antonio Ortega and Efstathios Stavrakis and Antonio Fernández Anta},
  title = {{Estimating the COVID-19 Prevalence in Spain with Indirect Reporting via Open Surveys}},
  booktitle = {Frontiers in Public Health},
  year = {2021},
  doi = {10.3389/fpubh.2021.658544},
}
Figure