Corrigendum: Vaccine safety surveillance using routinely collected healthcare data—An empirical evaluation of epidemiological designs

Martijn J. Schuemie, Faaizah Arshad, Nicole Pratt, Fredrik Nyberg, Thamir M. Alshammari, George Hripcsak, Patrick Ryan, Daniel Prieto‐Alhambra, Lana Yin Hui Lai, Xintong Li, Stephen Fortin, Evan Minty, Marc A. SuchardFrontiers in Pharmacology 2022 · DOI · OpenAlex

Abstract

[This corrects the article DOI: 10.3389/fphar.2022.893484.].

Introduction

This corrigendum addresses a technical error in a large-scale evaluation of vaccine safety monitoring designs previously published in Frontiers in Pharmacology. While the primary scientific conclusions regarding the relative performance of different epidemiological designs remain intact, the corrected data reveals that detecting moderate risks is more challenging and time-consuming than initially reported.

The Nature of the Error

The original study utilized "positive controls"—simulated associations where a known signal is injected into the data—to evaluate the sensitivity of various surveillance methods. A software implementation error resulted in the intended effect sizes (Relative Risks of 1.25, 2, and 4) being applied twice. This effectively squared the values; for instance, an intended 4x risk was erroneously treated as a 16x risk.

Because larger effect sizes are significantly easier to detect, this error led to an underestimation of Type 2 errors (false negatives) and an overly optimistic "time to 50% sensitivity" (the duration of data collection required to have a 50% probability of detecting a signal).

Methods and Datasets

The researchers evaluated several standard epidemiological designs using the Optum EHR database:

  • Case-control: Comparing cases with adverse events to matched or random controls.
  • Cohort methods: Utilizing various comparators such as outpatient visits or historical data, often with Propensity Score (PS) matching or weighting.
  • Self-Controlled Designs (SCCS/SCRI): Where individuals serve as their own controls, comparing post-vaccination risk windows to other periods.

The evaluation spanned several vaccines, including H1N1pdm, HPV (Gardasil 9), Seasonal flu (Fluvirin, Fluzone), and Zoster (Shingrix).

Updated Error Rates

The corrected Figure 4 illustrates the trade-off between Type 1 (false positive) and Type 2 (false negative) errors. Empirical calibration—using negative controls to adjust p-values—effectively minimizes Type 1 errors across designs. However, the corrected data shows that Type 2 error rates are higher than previously estimated, particularly for smaller effect sizes.

Figure 4: Updated error rates showing Type 1 and Type 2 errors across different epidemiological designs and vaccines. Darker bars represent calibrated methods.

Time to Detection

The corrected Figure 5 provides a realistic timeline for safety signal detection. For a moderate effect size (e.g., a doubling of risk), many surveillance designs require 6 to 12 months of data collection in a large EHR database to achieve 50% sensitivity.

Figure 5: Updated time to 50% sensitivity (in months) for various vaccines and surveillance methods.

Conclusion

The "double-multiplication" error serves as a critical reminder of the complexities involved in large-scale computational epidemiology. While the absolute metrics for sensitivity and error rates have been adjusted, the relative rankings of the methods remain consistent. The authors reaffirm that empirical calibration is essential for reducing false positives, though it necessitates a longer surveillance period to reliably detect true safety signals.

Summary updated

Abstract

[This corrects the article DOI: 10.3389/fphar.2022.893484.].
BibTeX
@inproceedings{schuemie2022corrigendum,
  author = {Martijn J. Schuemie and Faaizah Arshad and Nicole Pratt and Fredrik Nyberg and Thamir M. Alshammari and George Hripcsak and Patrick Ryan and Daniel Prieto‐Alhambra and Lana Yin Hui Lai and Xintong Li and Stephen Fortin and Evan Minty and Marc A. Suchard},
  title = {{Corrigendum: Vaccine safety surveillance using routinely collected healthcare data—An empirical evaluation of epidemiological designs}},
  booktitle = {Frontiers in Pharmacology},
  year = {2022},
  doi = {10.3389/fphar.2022.1088973},
}
Figure