Quantifying the risks versus benefits of the Pfizer COVID-19 vaccine in Australia: a Bayesian network analysis

Jane E. Sinclair, Helen J. Mayfield, Kirsty R. Short, S. J. Brown, Rajesh Puranik, Kerrie Mengersen, John Litt, Colleen L. LaumedRxiv 2022 · DOI

TL;DR

This paper utilizes a Bayesian network model to quantify the risk-benefit trade-offs of the Pfizer COVID-19 vaccine in Australia, specifically weighing the rare risk of vaccine-associated myocarditis against the dangers of COVID-19 infection. The study finds that the risk of developing or dying from myocarditis due to a COVID-19 infection is hundreds to thousands of times higher than the risk from the vaccine, confirming a substantial net benefit for vaccination across all age groups and transmission scenarios.

What Problem Does This Paper Address?

Public health policy during the pandemic was often reactive to emerging safety signals. One of the most significant concerns regarding mRNA vaccines (like Pfizer’s BNT162b2) was the increased incidence of myocarditis (heart muscle inflammation), particularly in young males.

However, risk-benefit analysis is a "moving target." Factors like waning vaccine effectiveness, the emergence of more transmissible variants (like Delta), and varying community transmission levels make it difficult for clinicians and policymakers to provide clear, data-driven advice. The authors sought to create a dynamic, integrated model that could synthesize disparate international and local data to provide a transparent, scenario-based assessment of vaccination risks versus benefits.

Methods

The researchers built a Bayesian Network (BN) model, an extension of their previous work on the COVID-19 Risk Calculator (CoRiCal).

  • Structure: The BN consists of 15 nodes representing inputs (age, sex, dose number, transmission intensity), intermediate variables (vaccine effectiveness, relative risk of infection), and outcomes (probability of myocarditis, symptomatic COVID-19, and death).
  • Logic: Unlike simple statistical models, BNs use Conditional Probability Tables (CPTs) to define the relationships between variables. This allows the model to handle uncertainty and integrate different types of data, from government surveillance reports to peer-reviewed literature and expert elicitation.
  • Scenarios: The model was specifically designed to simulate the Delta variant landscape, incorporating "time since second dose" to account for waning immunity and the impact of booster (third) doses.

Datasets

The model was "fed" with a combination of Australian and international data:

  • Australian National Interoperable Notifiable Diseases Surveillance System (NINDSS): For age-sex specific case fatality rates and infection distributions in Australia.
  • Therapeutic Goods Administration (TGA): For local rates of vaccine-associated myocarditis.
  • International Cohorts: Data from Israel, the USA (CDC/VAERS), the UK (Public Health England), and Finland were used to fill gaps, particularly regarding background myocarditis rates and vaccine effectiveness against the Delta variant.
  • Clinical Studies: Specific parameters for infection-induced myocarditis were sourced from international cohort studies (e.g., Buckley et al.).

Key Insights and Ideas

  • The "Myocarditis Paradox": While the vaccine does increase the risk of myocarditis, the infection itself is a far more potent trigger for the same condition. The model estimated that the probability of developing myocarditis from symptomatic COVID-19 was 239 to 5,847 times higher than from the vaccine.
  • Mortality Disparity: The risk of dying from infection-related myocarditis was even more extreme—up to 384,684 times higher than dying from vaccine-associated myocarditis.
  • Sensitivity to Waning Immunity: The model is highly sensitive to vaccine effectiveness. A hypothetical 10% drop in effectiveness against Delta results in a ~35% increase in symptomatic cases and a ~55% increase in deaths, highlighting the critical importance of booster doses.
  • Age-Specific Benefits: Younger age groups (12–29) see the greatest benefit in terms of preventing symptomatic cases, while older groups (70+) see the most significant benefit in terms of preventing death.

Key Results

In a simulated population of one million people (with 90% having at least two doses) under a high-transmission scenario (10% chance of infection over two months):

  • Prevention: Vaccination would prevent 68,813 symptomatic cases and 981 deaths.
  • Adverse Events: In that same million-person cohort, only 42 cases of vaccine-associated myocarditis would be expected in males and 16 in females.
  • Risk Ratio: For 12–19-year-old males (the highest-risk group for vaccine side effects), the model showed that even under low transmission (1%), vaccination prevents 11,029 cases and 1 death for every 146 cases of vaccine-associated myocarditis.

Limitations and Future Work

  • Variant Lag: The model was calibrated for the Delta variant. While the framework is updatable, the specific results may differ for Omicron or future variants with different severity profiles.
  • Granularity: The model does not currently account for individual comorbidities (e.g., diabetes, obesity) or personal behaviors that influence infection risk.
  • Data Gaps: At the time of writing, Australian data for third-dose myocarditis rates were limited, requiring the authors to use conservative "worst-case" assumptions (assuming third-dose risk equals second-dose risk).
  • Future Scope: The authors suggest expanding the model to include Long COVID outcomes, other vaccines (like Moderna), and younger pediatric age groups (<12 years).

Summary updated

Abstract

ABSTRACT The Pfizer COVID-19 vaccine is associated with increased myocarditis incidence. Constantly evolving evidence regarding incidence and case fatality of COVID-19 and myocarditis related to infection or vaccination, creates challenge for risk-benefit analysis of vaccination programs. Challenges are complicated further by emerging evidence of waning vaccine effectiveness, and variable effectiveness against variants. Here, we build on previous work on the COVID-19 Risk Calculator (CoRiCal) by integrating Australian and international data to inform a Bayesian network that calculates probabilities of outcomes for the Delta variant under different scenarios of Pfizer COVID-19 vaccine coverage, age groups (≤12 years), sex, community transmission intensity and vaccine effectiveness. The model estimates that in a population where 5% were unvaccinated, 5% had one dose, 60% had two doses and 30% had three doses, the probabilities of developing and dying from COVID-19-related myocarditis were 239-5847 and 1430-384,684 times higher (depending on age and sex), respectively, than developing vaccine-associated myocarditis. For one million people with this vaccine coverage, where transmission intensity was equivalent to 10% chance of infection over two months, 68,813 symptomatic COVID-19 cases and 981 deaths would be prevented, with 42 and 16 expected cases of vaccine-associated myocarditis in males and females, respectively. The model may be updated to include emerging best evidence, data pertinent to different countries or vaccines, and other outcomes such as long COVID.
BibTeX
@misc{sinclair2022quantifying,
  author = {Jane E. Sinclair and Helen J. Mayfield and Kirsty R. Short and S. J. Brown and Rajesh Puranik and Kerrie Mengersen and John Litt and Colleen L. Lau},
  title = {{Quantifying the risks versus benefits of the Pfizer COVID-19 vaccine in Australia: a Bayesian network analysis}},
  year = {2022},
  doi = {10.1101/2022.02.07.22270637},
  howpublished = {medRxiv},
}
Figure