Comparability of clinical trials and spontaneous reporting data regarding COVID-19 vaccine safety

Chongliang Luo, Jingcheng Du, Adam Cuker, Ebbing Lautenbach, David A. Asch, Gregory A. Poland, Cui Tao, Yong ChenScientific 2022 · DOI

TL;DR

This study compares the safety profiles of the Pfizer, Moderna, and Janssen COVID-19 vaccines by analyzing data from both clinical trials (RCTs) and the Vaccine Adverse Event Reporting System (VAERS). The findings demonstrate a remarkable consistency in vaccine safety rankings between these two distinct data sources, particularly for common local and systemic side effects, lending credibility to VAERS for identifying rare but serious adverse events.

What Problem Does This Paper Address?

Monitoring the safety of vaccines, especially for rare but severe adverse events (AEs), presents a significant challenge. Randomized controlled trials (RCTs), while providing robust evidence, are often limited by their relatively small sample sizes and short follow-up periods, making it difficult to detect very infrequent AEs. This is particularly true for COVID-19 vaccines, where the sheer scale of global vaccination campaigns means that even rare AEs can affect a substantial number of people. The Vaccine Adverse Event Reporting System (VAERS) offers a much larger, real-world dataset but is known to have inherent biases due to its passive, self-reporting nature. The core problem this paper addresses is how to effectively leverage and compare these different data sources to gain a comprehensive understanding of COVID-19 vaccine safety, specifically by assessing the consistency of safety signals between RCTs and VAERS.

Methods

The researchers employed a comparative analysis approach. They first extracted safety data from three pivotal RCTs for the Pfizer, Moderna, and Janssen COVID-19 vaccines, focusing on local and systemic AEs reported within seven days of vaccination. Concurrently, they analyzed a large dataset of over 559,000 VAERS reports submitted between December 14, 2020, and September 17, 2021. For common AEs, they calculated event rates from RCTs and reporting rates from VAERS and then compared the relative rankings of the vaccines across these two datasets. To assess consistency, they used Spearman's rank correlation coefficient. Crucially, they also used VAERS data to investigate rare but severe AEs (e.g., shingles, thrombosis, myocarditis, GBS) that were not adequately captured in the RCTs due to their rarity or short follow-up.

Datasets

  • Randomized Controlled Trials (RCTs): Data from three pivotal RCTs for the Pfizer-BioNTech (BNT162b2), Moderna (mRNA-1273), and Janssen (Ad26.COV2.S) COVID-19 vaccines. These trials provided event rates for local and systemic AEs.
  • Vaccine Adverse Event Reporting System (VAERS): A large, publicly available database of post-marketing safety reports. The study utilized 559,717 reports related to COVID-19 vaccines from December 14, 2020, to September 17, 2021. This dataset was used to calculate reporting rates for both common and rare AEs.

Key Insights and Ideas

  • Cross-Validation of Safety Data: The most significant insight is the strong consistency in vaccine safety rankings between RCTs and VAERS for common local and systemic AEs. This suggests that VAERS, despite its limitations, can provide a reliable relative ranking of vaccine safety, especially when analyzed appropriately.
  • VAERS as a Complement to RCTs: The study highlights VAERS' crucial role in identifying and monitoring rare but severe AEs that are difficult to detect in RCTs. This "extrapolation" of consistency allows for a more complete safety picture.
  • Dose-Dependent Systemic AEs: For Pfizer and Moderna vaccines, the reporting risks of systemic AEs were consistently higher after the second dose compared to the first dose, a finding supported by both RCT and VAERS data.
  • Specific Rare AE Signals: VAERS data pointed to specific concerns: higher risks of thrombosis and GBS with the Janssen vaccine, a higher risk of shingles with Moderna (particularly the first dose), increased myocarditis risk with the second doses of Pfizer and Moderna, and a higher risk of anaphylaxis with the first dose of Pfizer.

Key Results

  • Rank Consistency: Spearman's rank correlation coefficients for local and systemic AEs between RCTs and VAERS were generally positive (ranging from 0.36 to 0.80), indicating good consistency in how the vaccines were ranked by risk.
  • Janssen Vaccine Concerns: VAERS data indicated the highest reported risks for thrombosis and GBS following the Janssen vaccine.
  • Moderna Shingles Risk: The first dose of the Moderna vaccine showed the highest reported risk of shingles, followed by its second dose.
  • Myocarditis and Anaphylaxis: Higher risks of myocarditis were observed after the second dose of Pfizer and Moderna, while anaphylaxis risk was higher after the first dose of Pfizer.
  • VAERS Reporting Rates: VAERS reports were more common per dose for the Janssen vaccine compared to Pfizer or Moderna. Females were more likely to submit VAERS reports than males across all vaccine types.

Limitations and Future Work

  • Inherent VAERS Biases: The authors acknowledge the inherent biases of the spontaneous reporting system, including potential underreporting of mild AEs and overreporting of severe AEs, which can be influenced by public attention and media coverage. This could explain some inconsistencies, particularly for the Janssen vaccine.
  • Limited RCT Data: The study only included three pivotal RCTs and did not compare VAERS data with other active vaccine safety surveillance systems (e.g., BEST, SPEAC, CISA, VSD).
  • No Absolute Incidence: The analysis focuses on relative rankings and does not provide absolute incidence rates of AEs in the general population.
  • Future Work: The authors suggest that their methodology can be applied to ongoing and future vaccine surveillance, including booster doses and pediatric vaccinations. They also maintain a dynamic online visualization tool to continuously monitor COVID-19 vaccine safety. Future work could involve integrating data from additional safety surveillance systems for a more robust comparison.

Summary updated

Abstract

Severe adverse events (AEs) after COVID-19 vaccination are not well studied in randomized controlled trials (RCTs) due to rarity and short follow-up. To monitor the safety of COVID-19 vaccines ("Pfizer" vaccine dose 1 and 2, "Moderna" vaccine dose 1 and 2, and "Janssen" vaccine single dose) in the U.S., especially regarding severe AEs, we compare the relative rankings of these vaccines using both RCT and the Vaccine Adverse Event Reporting System (VAERS) data. The risks of local and systemic AEs were assessed from the three pivotal COVID-19 vaccine trials and also calculated in the VAERS cohort consisting of 559,717 reports between December 14, 2020 and September 17, 2021. AE rankings of the five vaccine groups calculated separately by RCT and VAERS were consistent, especially for systemic AEs. For severe AEs reported in VAERS, the reported risks of thrombosis and GBS after Janssen vaccine were highest. The reported risk of shingles after the first dose of Moderna vaccine was highest, followed by the second dose of the Moderna vaccine. The reported risk of myocarditis was higher after the second dose of Pfizer and Moderna vaccines. The reported risk of anaphylaxis was higher after the first dose of Pfizer vaccine. Limitations of this study are the inherent biases of the spontaneous reporting system data, and only including three pivotal RCTs and no comparison with other active vaccine safety surveillance systems.
BibTeX
@inproceedings{luo2022comparability,
  author = {Chongliang Luo and Jingcheng Du and Adam Cuker and Ebbing Lautenbach and David A. Asch and Gregory A. Poland and Cui Tao and Yong Chen},
  title = {{Comparability of clinical trials and spontaneous reporting data regarding COVID-19 vaccine safety}},
  booktitle = {Scientific},
  year = {2022},
  doi = {10.1038/s41598-022-13809-7},
}
Figure