Stochastic interventional approach to assessing immune correlates of protection: Application to the COVE messenger RNA-1273 vaccine trial

Nima S. Hejazi, Xiaoying Shen, Lindsay N. Carpp, David Benkeser, Dean Follmann, Holly Janes, Lindsey R. Baden, Hana M. El Sahly, Weiping Deng, Haiyu Zhou, Brett Leav, David C. Montefiori, Peter B. GilbertInternational Journal of Infectious Diseases 2023 · DOI

TL;DR

This paper introduces a novel statistical method, Stochastic Interventional Vaccine Efficacy (SVE) analysis, to better understand how immune responses correlate with vaccine protection against COVID-19. Applying this to the mRNA-1273 vaccine trial, they found that neutralizing antibody titers are indeed a correlate of protection, but predicting efficacy against new variants remains challenging, suggesting higher antibody levels might be needed for newer strains.

What Problem Does This Paper Address?

Understanding "correlates of protection" (CoPs) is crucial for vaccine development. A CoP is a measurable immune marker that reliably predicts how well a vaccine will protect against a disease. For COVID-19 vaccines, neutralizing antibody (nAb) titers have been a key focus. However, traditional methods for assessing CoPs often rely on observational data or simplified assumptions. This paper addresses the need for a more robust statistical framework to assess CoPs, particularly in the context of evolving viral variants and the desire to predict vaccine efficacy (VE) against these variants. They aim to move beyond simply observing associations to a more causal inference-based approach that can explore hypothetical scenarios, like how VE might change if immune responses were boosted.

Methods

The core innovation here is the Stochastic Interventional Vaccine Efficacy (SVE) analysis. Unlike traditional methods that might look at associations, SVE is a causal inference technique. It allows researchers to simulate hypothetical scenarios by "intervening" on an immune marker (like nAb titer) and observing how vaccine efficacy would change.

Here's a breakdown:

  • Stochastic Intervention: Instead of a fixed, deterministic shift in an immune marker, SVE introduces a stochastic (random) shift. This means they model how VE would change if an individual's immune marker was hypothetically shifted by a certain amount (represented by 'δ'). This accounts for individual variability.
  • Counterfactual VE: The SVE approach estimates the "counterfactual" vaccine efficacy – what VE would have been under these hypothetical immune marker shifts.
  • Application to COVE Trial: They applied this method to data from the mRNA-1273 (Moderna) COVE phase III vaccine trial.
  • Variant Prediction: They further explored SVE's ability to predict VE against various SARS-CoV-2 variants by using the observed differences in nAb titers against variants compared to the ancestral D614G strain as the hypothetical shifts (δ).
  • Validation: The predicted VE against variants was then compared to empirical VE estimates from external test-negative design (TND) and randomized controlled trial (RCT) studies.

Datasets

The primary dataset used is from the COVE trial of the mRNA-1273 vaccine. This trial involved participants receiving either the mRNA-1273 vaccine or a placebo. Serum samples were collected from recipients at various time points and tested for neutralizing antibody titers against the ancestral SARS-CoV-2 strain (D614G) and numerous other variants.

For validation, they utilized data from 12 test-negative design (TND) studies and 1 RCT that reported variant-specific vaccine effectiveness (VE) or efficacy (VE) estimates for mRNA vaccines against SARS-CoV-2 infection or symptomatic COVID-19.

Key Insights and Ideas

  • nAb Titer as a Correlate of Protection: The SVE analysis strongly supports the idea that neutralizing antibody titers are a valid correlate of protection for mRNA vaccines against the ancestral SARS-CoV-2 strain. The method clearly shows how VE increases with higher antibody titers.
  • Predicting Variant VE is Tricky: While SVE can estimate how VE might change with shifts in antibody titers, directly predicting VE against specific variants proved difficult. This highlights the limitations of using a single, ancestral strain-based correlate to predict efficacy against diverse and rapidly evolving variants.
  • Higher Titers for Newer Variants? The findings suggest that protection against newer variants, particularly Omicron, might require higher neutralizing antibody titers than what was sufficient for protection against earlier strains like D614G.
  • SVE as a Promising Tool: The paper demonstrates SVE's potential as a flexible statistical tool for exploring immune marker-VE relationships and for making hypothetical predictions, even if direct variant prediction has limitations.

Key Results

  • Ancestral Strain VE: SVE analysis of the COVE trial data confirmed that post-dose 2 nAb titers against the D614G strain are a correlate of protection. Estimated VE increased from 66.9% to 99.3% with 10-fold increases in nAb titers.
  • Variant VE Prediction: SVE estimates for variant-specific VE were generally lower than the VE against the ancestral strain, ranging from 67% (mu) to 86% (lambda).
  • Two-Dose Prediction Accuracy: For two-dose vaccination, the correlation between SVE-predicted VE and empirical VE from external studies was weak (Spearman correlation of 0.143, CCC of 0.062), with predictions tending to underestimate.
  • Three-Dose Prediction Accuracy: For three-dose vaccination, the correlation was higher (Spearman correlation of 0.894), but concordance remained weak (CCC of 0.017), with predictions tending to overestimate.

Limitations and Future Work

The authors acknowledge several limitations:

  • Validation Data Heterogeneity: The validation analysis relied on external studies, many of which were TNDs, which can have inherent biases and limitations compared to RCTs. There was also variation in study designs, follow-up periods, and variant definitions.
  • Hypothetical Context: Predicting variant-specific VE involved strong assumptions about maintaining the same placebo-arm risk and the "variant-invariant CoP model," which may not hold true in reality.
  • Limited Sample Size for Omicron: The validation analysis for Omicron variants was particularly limited by sample size.
  • "Variant-Invariant CoP Model" Questioned: The weak concordance in validation suggests that a simple "variant-invariant CoP model" might be too simplistic.
  • nAb Assay Limitations: The study primarily used a single nAb assay and focused on the D614G strain as a reference. Future work could explore other assays and more direct "variant-matched correlates."

Future work could involve testing the SVE framework with more diverse datasets, exploring more sophisticated ways to define hypothetical shifts (e.g., based on specific vaccine modifications), and investigating the utility of SVE for predicting VE with updated vaccine formulations.

Summary updated

Abstract

SVE analysis of COVE supports nAb titer as a CoP for messenger RNA vaccines. Predicting variant-specific VE proved difficult due to many limitations. Greater anti-Omicron titers may be needed for high-level protection against Omicron vs anti-D614G titers needed for high-level protection against pre-Omicron COVID-19.
BibTeX
@article{hejazi2023stochastic,
  author = {Nima S. Hejazi and Xiaoying Shen and Lindsay N. Carpp and David Benkeser and Dean Follmann and Holly Janes and Lindsey R. Baden and Hana M. El Sahly and Weiping Deng and Haiyu Zhou and Brett Leav and David C. Montefiori and Peter B. Gilbert},
  title = {{Stochastic interventional approach to assessing immune correlates of protection: Application to the COVE messenger RNA-1273 vaccine trial}},
  journal = {International Journal of Infectious Diseases},
  year = {2023},
  doi = {10.1016/j.ijid.2023.09.012},
}
Figure