Improving the representativeness of UK’s national COVID-19 Infection Survey through spatio-temporal regression and post-stratification

Koen B. Pouwels, David W. Eyre, Thomas House, Ben Aspey, Philippa C. Matthews, Nicole Stoesser, John Newton, Ian Diamond, Ruth Studley, Nick Taylor, John I. Bell, Jeremy Farrar, Jaison Kolenchery, Brian D. Marsden, Sarah Hoosdally, E. Yvonne Jones, David I. Stuart, Derrick W. Crook, Tim Peto, A. Sarah Walker, Jia Wei, Emma Pritchard, Karina-Doris Vihta, George Doherty, James Kavanagh, Kevin Chau, Stephanie B. Hatch, Daniel Ebner, Lucas Martins Ferreira, Thomas Christott, Wanwisa Dejnirattisai, Juthathip Mongkolsapaya, Sarah Cameron, Phoebe Tamblin-Hopper, Magda Wolna, Brent Brown, Richard J. Cornall, Gavin Screaton, Katrina Lythgoe, David Bonsall, Tanya Golubchik, Helen Fryer, Tina Thomas, Daniel Ayoubkhani, Russell Black, Antonio Felton, Megan Crees, Joel W. Jones, Lina Lloyd, Esther Sutherland, Stuart Cox, Kevin Paddon, Tim James, Julie V. Robotham, Paul Birrell, Helena Jordan, Tim Sheppard, Graham Athey, Dan Moody, Leigh Curry, Pamela Brereton, Ian Jarvis, Anna Godsmark, George Morris, Bobby Mallick, Phil Eeles, Jodie Hay, Harper VanSteenhouse, Jessica Lee, Sean White, Tim Evans, Lisa Bloemberg, Katie Allison, Anouska Pandya, Sophie Davis, David I. Conway, Margaret MacLeod, Chris CunninghamNature Communications 2024 · DOI

Abstract

Population-representative estimates of SARS-CoV-2 infection prevalence and antibody levels in specific geographic areas at different time points are needed to optimise policy responses. However, even population-wide surveys are potentially impacted by biases arising from differences in participation rates across key groups. Here, we used spatio-temporal regression and post-stratification models to UK's national COVID-19 Infection Survey (CIS) to obtain representative estimates of PCR positivity (6,496,052 tests) and antibody prevalence (1,941,333 tests) for different regions, ages and ethnicities (7-December-2020 to 4-May-2022). Not accounting for vaccination status through post-stratification led to small underestimation of PCR positivity, but more substantial overestimations of antibody levels in the population (up to 21 percentage points), particularly in groups with low vaccine uptake in the general population. There was marked variation in the relative contribution of different areas and age-groups to each wave. Future analyses of infectious disease surveys should take into account major drivers of outcomes of interest that may also influence participation, with vaccination being an important factor to consider.

Introduction

Public health policy relies heavily on representative data. The UK’s COVID-19 Infection Survey (CIS) was established as a gold-standard surveillance tool, using random household invitations to monitor infection and immunity regardless of symptoms. However, voluntary surveys are susceptible to participation bias, where participants differ systematically from the general population. This study investigates the "healthy volunteer" effect: individuals more likely to participate in health research are often more likely to adopt health-protective behaviors, such as vaccination. If a survey over-represents vaccinated individuals, it may provide an overly optimistic view of population immunity, potentially misleading policy responses.

Methods

The researchers addressed these biases using Spatio-temporal Multilevel Regression and Post-stratification (MRP).

Modeling Approach

  1. Multilevel Regression: The probability of testing positive (PCR or antibodies) was modeled based on age, sex, ethnicity, vaccination status, and geography. The model utilized a Besag-York-Mollié (BYM2) specification for spatial effects and a Type IV space-time interaction, allowing it to "borrow strength" across neighboring regions and time points.
  2. Post-stratification: Model results were re-weighted using "ground truth" population data from the 2011 Census, ETHPOP, and administrative vaccine records from the National Immunisation Management Service (NIMS).

Datasets

The analysis integrated several large-scale datasets:

  • UK CIS: 6,496,052 PCR tests and 1,941,333 antibody tests (Dec 2020 – May 2022).
  • NIMS: Administrative data for actual vaccination uptake in England.
  • Census & ETHPOP: Population denominators for demographic weighting.
  • English Index of Multiple Deprivation (IMD): Used to analyze socio-economic drivers of infection.

Results

The Participation Gap

A significant discrepancy was found between survey participants and the general population. By May 2022, 93% of survey participants were vaccinated, while administrative data showed only 75% uptake in the general population. This confirms a strong correlation between survey participation and vaccination status.

Comparison of vaccine uptake between survey participants and administrative data highlights significant participation bias.

Impact on Prevalence Estimates

Failing to account for vaccination status led to systematic errors in prevalence estimates:

  • Antibody Levels: The most substantial impact was seen here, with an average overestimation of 5.2 percentage points across England. In specific areas with low vaccine uptake, such as Newham, the overestimation reached 21 percentage points (76% vs. 55%).
  • PCR Positivity: Conversely, not adjusting for vaccination led to a small but significant underestimation of PCR positivity (maximum difference of 0.35 percentage points), as the survey over-represented the more-protected vaccinated group.
Regional PCR positivity estimates showing the differences between raw data and models adjusted for vaccination status.

Geographic and Socio-economic Drivers

The spatio-temporal model revealed marked variation across regions. London, for instance, saw significantly higher peaks during the Omicron BA.1 wave compared to the South West. Furthermore, deprivation and urbanicity explained 57% of the variance in infection rankings, with more deprived urban areas consistently experiencing higher viral loads across different waves.

Discussion

This study demonstrates that even large-scale, well-designed surveys can suffer from significant participation bias. The "protection illusion" created by over-representing vaccinated individuals can lead to a false sense of security regarding herd immunity.

Limitations

The study's reliance on 2011 Census data is a limitation, as demographic shifts over the decade may introduce errors. Additionally, while MRP adjusts for observed factors like vaccination, it cannot account for unobserved behaviors (e.g., mask-wearing) that might also differ between participants and non-participants.

Future Directions

The authors recommend that future infectious disease surveys treat vaccination status as a primary driver of participation requiring rigorous adjustment. Integrating real-time socio-economic data will also be crucial for understanding the underlying causes of geographic infection clusters.

Summary updated

Abstract

Population-representative estimates of SARS-CoV-2 infection prevalence and antibody levels in specific geographic areas at different time points are needed to optimise policy responses. However, even population-wide surveys are potentially impacted by biases arising from differences in participation rates across key groups. Here, we used spatio-temporal regression and post-stratification models to UK's national COVID-19 Infection Survey (CIS) to obtain representative estimates of PCR positivity (6,496,052 tests) and antibody prevalence (1,941,333 tests) for different regions, ages and ethnicities (7-December-2020 to 4-May-2022). Not accounting for vaccination status through post-stratification led to small underestimation of PCR positivity, but more substantial overestimations of antibody levels in the population (up to 21 percentage points), particularly in groups with low vaccine uptake in the general population. There was marked variation in the relative contribution of different areas and age-groups to each wave. Future analyses of infectious disease surveys should take into account major drivers of outcomes of interest that may also influence participation, with vaccination being an important factor to consider.
BibTeX
@inproceedings{pouwels2024improving,
  author = {Koen B. Pouwels and David W. Eyre and Thomas House and Ben Aspey and Philippa C. Matthews and Nicole Stoesser and John Newton and Ian Diamond and Ruth Studley and Nick Taylor and John I. Bell and Jeremy Farrar and Jaison Kolenchery and Brian D. Marsden and Sarah Hoosdally and E. Yvonne Jones and David I. Stuart and Derrick W. Crook and Tim Peto and A. Sarah Walker and Jia Wei and Emma Pritchard and Karina-Doris Vihta and George Doherty and James Kavanagh and Kevin Chau and Stephanie B. Hatch and Daniel Ebner and Lucas Martins Ferreira and Thomas Christott and Wanwisa Dejnirattisai and Juthathip Mongkolsapaya and Sarah Cameron and Phoebe Tamblin-Hopper and Magda Wolna and Brent Brown and Richard J. Cornall and Gavin Screaton and Katrina Lythgoe and David Bonsall and Tanya Golubchik and Helen Fryer and Tina Thomas and Daniel Ayoubkhani and Russell Black and Antonio Felton and Megan Crees and Joel W. Jones and Lina Lloyd and Esther Sutherland and Stuart Cox and Kevin Paddon and Tim James and Julie V. Robotham and Paul Birrell and Helena Jordan and Tim Sheppard and Graham Athey and Dan Moody and Leigh Curry and Pamela Brereton and Ian Jarvis and Anna Godsmark and George Morris and Bobby Mallick and Phil Eeles and Jodie Hay and Harper VanSteenhouse and Jessica Lee and Sean White and Tim Evans and Lisa Bloemberg and Katie Allison and Anouska Pandya and Sophie Davis and David I. Conway and Margaret MacLeod and Chris Cunningham and Chris Cunningham},
  title = {{Improving the representativeness of UK’s national COVID-19 Infection Survey through spatio-temporal regression and post-stratification}},
  booktitle = {Nature Communications},
  year = {2024},
  doi = {10.1038/s41467-024-49201-4},
}
Figure