Improving the representativeness of UK’s national COVID-19 Infection Survey through spatio-temporal regression and post-stratification

Koen B. Pouwels, David W. Eyre, Thomas House, Ben Aspey, Philippa C. Matthews, Nicole Stoesser, John Newton, Ian Diamond, Ruth Studley, Nick G. H.Taylor, John I. Bell, Jeremy Farrar, Jaison Kolenchery, Brian D. Marsden, Sarah Hoosdally, E. Yvonne Jones, David I. Stuart, Derrick W. Crook, Tim Peto, A. Sarah Walker, the Covid-19 Infection Survey TeammedRxiv 2023 · DOI

Abstract

Population-representative estimates of SARS-CoV-2 infection prevalence and antibody levels are essential for optimizing public health policy. However, even large-scale surveys like the UK’s national COVID-19 Infection Survey (CIS) are susceptible to selection bias, as participation rates often vary across demographic groups. This study applies spatio-temporal multilevel regression and post-stratification (MRP) models to over 6.4 million PCR tests and 1.9 million antibody tests to produce more accurate estimates. The findings reveal that failing to account for vaccination status in post-stratification leads to a significant overestimation of population immunity—by as much as 21% in certain areas—highlighting the importance of accounting for participation drivers in infectious disease surveillance.

Introduction

The UK’s national COVID-19 Infection Survey (CIS) was widely regarded as a gold standard for tracking the pandemic. However, participation in the survey was voluntary, leading to a classic statistical challenge: selection bias. If the individuals who choose to participate in a health survey are systematically different from the general population—for instance, by being more health-conscious or more likely to be vaccinated—the resulting data will not accurately reflect the community at large.

This paper investigates how to correct these biases to produce truly representative estimates of PCR positivity (active infections) and antibody prevalence (immunity). By integrating vaccination status into the statistical weighting process, the researchers demonstrate how traditional survey methods can inadvertently mask significant public health vulnerabilities.

Methodology: Spatio-Temporal MRP

To address selection bias, the researchers utilized a framework known as Spatio-Temporal Multilevel Regression and Post-stratification (MRP). This approach consists of two primary stages:

  1. Multilevel Regression: The probability of a positive test result was modeled based on individual characteristics (age, sex, and ethnicity) and geographic location across 116 sub-regions in England. To stabilize estimates in areas with fewer participants, the model used a BYM2 spatial prior, which allows the model to "borrow strength" from neighboring geographic areas. A temporal component was also integrated using autoregressive processes to capture weekly changes in infection and antibody levels.
  2. Post-stratification: This "re-weighting" step involves taking the model’s predictions for every possible combination of demographic and geographic variables and multiplying them by the actual population counts derived from administrative data.

The critical innovation in this study was the explicit inclusion of vaccination status as a post-stratification variable. This allowed the researchers to correct for the fact that survey participants were significantly more likely to be vaccinated than the average citizen.

Data Sources

The analysis drew upon a massive repository of real-world data spanning December 2020 to May 2022:

  • UK COVID-19 Infection Survey (CIS): 6,496,052 PCR test results and 1,941,333 antibody blood samples.
  • National Immunisation Management Service (NIMS): Real-time administrative data on vaccine uptake across England.
  • Census Data (2011 and 2021): Population denominators for age, sex, and location.
  • ETHPOP Database: Estimates for the distribution of ethnicity across regions and age groups.

Key Insights and Results

The "Healthy Volunteer" Effect

The data confirmed a strong "healthy volunteer" effect. By May 2022, 93% of survey participants were vaccinated, compared to only 75% in the general population. This discrepancy suggests that individuals who engage with health research are often those already taking the most significant precautions against the virus.

Overestimating Immunity

While ignoring vaccination status resulted in only a minor underestimation of active infections (PCR positivity)—roughly 0.38% nationally—the impact on antibody prevalence was profound. On average, antibody levels were overestimated by 4.8% across England. However, in specific urban areas with lower vaccine uptake, the discrepancy was much larger. In the London borough of Newham, for example, failing to account for vaccination status made the population appear much more protected than it was, showing a 77% antibody prevalence compared to a corrected estimate of 54%.

Geography and Socio-economics

The model highlighted that areas with higher deprivation and higher urban density consistently faced greater infection risks. A simple linear model using only deprivation and rurality explained 56% of the variance in PCR positivity rankings across the 116 sub-regions, underscoring the socio-economic nature of the pandemic’s spread.

Limitations and Future Directions

Despite the robustness of the MRP framework, the authors noted several limitations:

  • Binary Vaccination Data: To maintain model convergence, vaccination was treated as a simple "yes/no" variable, rather than accounting for the number of doses or the time elapsed since the last dose.
  • Unobserved Variables: The model assumes that a vaccinated survey participant is representative of a vaccinated person in the general population. If volunteers differ in unobserved ways (e.g., social distancing behavior), some residual bias may remain.
  • Data Latency: The study relied on 2011 Census data for certain denominators, as full 2021 Census results were not yet available during the analysis.

Conclusion

This research demonstrates that vaccination status is not merely a clinical variable but a primary driver of survey participation. Future infectious disease surveillance must account for such factors in statistical weighting to avoid a "false sense of security" in public health data. Accurate, representative estimates are vital for ensuring that public health interventions and resources are directed toward the communities that need them most.

Summary updated

Abstract

Abstract Population-representative estimates of SARS-CoV-2 infection prevalence and antibody levels in specific geographic areas at different time points are needed to optimise policy responses. However, even population-wide surveys are potentially impacted by biases arising from differences in participation rates across key groups. Here, we use spatio-temporal regression and post-stratification models to UK’s national COVID-19 Infection Survey (CIS) to obtain representative estimates of PCR positivity (6,496,052 tests) and antibody prevalence (1,941,333 tests) for different regions, ages and ethnicities (7-December-2020 to 4-May-2022). Not accounting for vaccination status through post-stratification led to small underestimation of PCR positivity, but more substantial overestimations of antibody levels in the population (up to 21%), particularly in groups with low vaccine uptake in the general population. There was marked variation in the relative contribution of different areas and age-groups to each wave. Future analyses of infectious disease surveys should take into account major drivers of outcomes of interest that may also influence participation, with vaccination being an important factor to consider.
BibTeX
@misc{pouwels2023improving,
  author = {Koen B. Pouwels and David W. Eyre and Thomas House and Ben Aspey and Philippa C. Matthews and Nicole Stoesser and John Newton and Ian Diamond and Ruth Studley and Nick G. H.Taylor and John I. Bell and Jeremy Farrar and Jaison Kolenchery and Brian D. Marsden and Sarah Hoosdally and E. Yvonne Jones and David I. Stuart and Derrick W. Crook and Tim Peto and A. Sarah Walker and the COVID-19 Infection Survey Team},
  title = {{Improving the representativeness of UK’s national COVID-19 Infection Survey through spatio-temporal regression and post-stratification}},
  year = {2023},
  doi = {10.1101/2023.02.26.23286474},
  howpublished = {medRxiv},
}
Figure