Estimating COVID-19 vaccine protection rates via dynamic epidemiological models—a study of 10 countries

Yuru Zhu, Jia Gu, Yumou Qiu, Song Xi Chen

TL;DR

Researchers have developed a new stochastic epidemiological model, vSVIADR, that estimates real-world vaccine efficacy (VE) using only publicly available infection and vaccination counts. By applying this model to ten countries, the study confirms that while the Delta variant significantly eroded protection, full vaccination remained substantially more effective than partial doses and consistently stayed above the WHO’s 50% efficacy threshold prior to the Omicron wave.

What Problem Does This Paper Address?

Evaluating how well vaccines work in the "wild" is notoriously difficult. Clinical trials are conducted in controlled environments that exclude certain demographics, while retrospective real-world studies often rely on "breakthrough case" data—specific records of vaccinated individuals getting sick.

The challenge is that most countries do not systematically collect or report breakthrough statistics. Furthermore, any epidemiological model must account for "latent" variables: the asymptomatic and pre-symptomatic individuals who spread the virus but never show up in official case counts. This paper addresses the need for a robust statistical framework that can estimate vaccine performance using only the aggregate data (total cases, deaths, and vaccinations) that most nations already report.

Methods

The authors introduce the varying coefficient Susceptible-Vaccinated-Infected-Diagnosed-Removed (vSVIADR) model. This is a 10-compartment stochastic model that extends the classic SEIR framework to include partially and fully vaccinated groups, as well as separate tracks for asymptomatic and diagnosed infections.

Key technical innovations include:

  • Multi-step Decentralized Estimation: Rather than solving for all variables at once, the authors use different segments of the pandemic timeline. They use pre-vaccine data to "calibrate" the diagnosis rate and post-vaccine data to isolate the vaccine's impact.
  • B-spline Approximation: To handle the highly volatile and unobserved infection rates, they use B-splines to smooth the underlying transmission dynamics.
  • Simulation-Based Objective Functions: Because many states (like the number of asymptomatic spreaders) are hidden, the model uses simulations to minimize the "contrast" between imputed data and theoretical trajectories.
  • Kernel Regressions: Time-varying rates for recovery and death are estimated using locally weighted kernel smoothing to account for changes in healthcare quality and reporting over time.

Datasets

The study analyzed publicly available data from February 23, 2020, to November 20, 2021, covering 10 countries: Brazil, Canada, Germany, India, Italy, Peru, Portugal, Turkey, the UK, and the US. * Epidemiological Data: Sourced from the Johns Hopkins University "2019 Novel Coronavirus Visual Dashboard." * Vaccination Data: Sourced from official national statistics, covering eight different vaccine brands (including mRNA, viral vector, and inactivated virus types).

Key Insights and Ideas

  • The "Hidden" Pandemic: The model estimates that the pre-symptomatic phase is significantly more contagious (modeled as 5x more) than the diagnosed phase, emphasizing why diagnosis rates are a critical lever in controlling spread.
  • Decoupling Efficacy from Trials: The study finds that real-world VE is often lower than clinical trial results due to "messy" factors like varying social behavior and the emergence of variants, yet it remains high enough to be the primary driver of pandemic suppression.
  • Scenario Analysis (The "What If"): The framework allows for counterfactual modeling. The authors could simulate what would have happened if a country had only used partial vaccination or no vaccination at all, providing a quantitative measure of the "lives saved" by specific policy choices.

Key Results

  • Full vs. Partial Protection: Across all ten countries, full vaccination provided a protection rate at least 22% higher than partial (one-dose) vaccination.
  • The Delta Hit: The Delta variant caused a noticeable drop in efficacy. Full vaccination VE ranged from 68%–95% in the pre-Delta era but fell to 45%–74% during the Delta-dominated period.
  • WHO Thresholds: Despite the decline caused by Delta, all eight vaccine brands studied remained above the 50% efficacy level recognized by the WHO before the arrival of Omicron.
  • Public Health Impact: In a "no-vaccine" simulation, the authors estimate there would have been 259 million additional cases and 2.6 million additional deaths across these 10 countries by October 2021.

Limitations and Future Work

The model relies on certain parameters—such as the average duration of immunity ($\mu$) and the ratio of contagiousness between asymptomatic and symptomatic cases ($r$)—that must be taken from existing clinical literature rather than estimated directly from the data.

Additionally, the current model assumes a "well-mixed" population, which ignores local clusters or "superspreader" events. The authors suggest that future iterations could incorporate negative binomial distributions to better account for "overdispersion" (the statistical tendency for cases to cluster more than a standard Poisson model predicts). Finally, while the paper touches on the early Omicron period, more work is needed to model the rapid immune escape seen in 2022 and beyond.

Summary updated

Abstract

The real-world performance of vaccines against COVID-19 infections is critically important to counter the pandemics. We propose a varying coefficient stochastic epidemic model to estimate the vaccine protection rates based on the publicly available epidemiological and vaccination data. To tackle the challenges posed by the unobserved state variables, we develop a multistep decentralized estimation procedure that uses different data segments to estimate different parameters. A B-spline structure is used to approximate the underlying infection rates and to facilitate model simulation in obtaining an objective function between the imputed and the simulation-based estimates of the latent state variables, leading to simulation-based estimation of the diagnosis rate using data in the prevaccine period and the vaccine effect parameters using data in the postvaccine periods. The time-varying infection, recovery and death rates are estimated by kernel regressions. We apply the proposed method to analyze the data in ten countries which collectively used eight vaccines. The analysis reveals that the average protection rate of the full vaccination was at least 22% higher than that of the partial vaccination and was largely above the WHO recognized level of 50% before November 20, 2021, including the Delta variant dominated period. The protection rates for the booster vaccine in the Omicron period were also provided.
BibTeX
@article{zhu2023estimating,
  author = {Yuru Zhu and Jia Gu and Yumou Qiu and Song Xi Chen},
  title = {{Estimating COVID-19 vaccine protection rates via dynamic epidemiological models—a study of 10 countries}},
  journal = {The Annals of Applied Statistics},
  year = {2023},
  doi = {10.1214/23-aoas1764},
}

Loading related content...

Figure