Modeling for COVID-19 college reopening decisions: Cornell, a case study

Peter I. Frazier, J. Massey Cashore, Ning Duan, Shane G. Henderson, Alyf Janmohamed, Brian Liu, David B. Shmoys, Jiayue Wan, Yujia ZhangProceedings of the National Academy of Sciences 2021 · DOI

TL;DR

This paper presents an epidemiological modeling framework that helped Cornell University make data-driven decisions about reopening for in-person instruction during the COVID-19 pandemic. The model effectively navigated significant parameter uncertainty, demonstrating that targeted asymptomatic screening, even for vaccinated individuals, is crucial for minimizing risk, especially against variants like Delta.

What Problem Does This Paper Address?

The COVID-19 pandemic presented a monumental challenge for universities: how to safely reopen for in-person instruction while minimizing the risk of outbreaks. This decision was complicated by several factors:

  • Unique University Populations: University students are generally younger, more socially connected, and have higher rates of asymptomatic infection compared to the general population, potentially accelerating virus spread.
  • Intervention Effectiveness Uncertainty: The efficacy of interventions like vaccination, mask mandates, and testing strategies was constantly evolving due to new viral variants and changing public health guidance.
  • Parameter Uncertainty: Epidemiological models are highly sensitive to input parameters (e.g., transmission rates, incubation periods, vaccine efficacy). When these parameters are uncertain, it becomes difficult to make accurate predictions of future case counts.
  • Balancing Safety and Costs: Interventions come with social and financial costs. Universities needed to find a balance between ensuring safety and minimizing these burdens.

The paper addresses the critical question of how to design effective COVID-19 interventions for university settings despite these significant uncertainties, using Cornell University's decision to reopen in Fall 2020 as a case study.

Methods

The researchers developed and employed a compartmental epidemiological model to simulate COVID-19 transmission within the university population. Key aspects of their methodology include:

  • Compartmental Model Structure: The model divides the population into compartments representing different stages of infection (Susceptible, Exposed, Infectious, Recovered) and intervention status (Quarantine, Isolation). It further stratifies these compartments by subpopulation (undergraduates, graduate students, employees) to capture group-specific dynamics.
  • Stochastic Simulation: The model incorporates randomness to reflect the inherent variability in disease spread and intervention outcomes.
  • Parameter Uncertainty Handling: A core innovation is their approach to managing parameter uncertainty. Instead of aiming for precise point estimates, they focused on defining ranges of plausible parameter values and using a Bayesian prior distribution to represent this uncertainty. They then explored how interventions performed across a spectrum of these plausible scenarios, from "nominal" (average) to "pessimistic" (worst-case within plausible ranges).
  • Pessimism Level Framework: To systematically explore the impact of uncertainty, they developed a "pessimism level" framework. This allows them to identify parameter configurations that lead to increasingly worse outcomes and evaluate interventions under these challenging scenarios.
  • Retrospective Calibration and Validation: The model was calibrated using de-identified data from Cornell's Fall 2020 and Spring 2021 semesters to assess its accuracy and inform future projections.
  • Intervention Modeling: The model explicitly incorporated various interventions, including symptomatic testing, asymptomatic screening (with varying frequencies and sampling methods), contact tracing, social distancing, and quarantine/isolation protocols.

Datasets

The study utilized de-identified aggregated data from Cornell University's surveillance testing program during the 2020-2021 academic year. This data was used for:

  • Model Calibration: Fitting model parameters to observed case trajectories.
  • Retrospective Analysis: Evaluating the accuracy of the model's predictions and the effectiveness of implemented interventions.

The paper does not introduce new benchmark datasets but rather uses real-world data from a specific university setting.

Key Insights and Ideas

  • Uncertainty is a Feature, Not a Bug: The paper demonstrates that even with significant uncertainty in predicting exact case numbers, epidemiological models can still be invaluable for making robust decisions about interventions. The focus shifts from precise prediction to identifying interventions that perform well across a wide range of plausible scenarios.
  • Targeted Asymptomatic Screening is Crucial: Even when vaccination rates are high, asymptomatic screening remains a vital tool for controlling outbreaks, especially against variants like Delta.
  • The "Most Social" Matter Most: A small subset of highly social students drives a disproportionate amount of transmission. Targeting interventions (like more frequent testing) to these groups offers significant risk reduction.
  • Virtual vs. Residential Instruction Trade-offs: Modeling suggested that residential instruction, when coupled with appropriate interventions, posed less risk than a purely virtual instruction model where students might still return to the local area without robust oversight.
  • Hindsight Can Improve, But Doesn't Always Revolutionize: While retrospective analysis revealed minor optimizations for testing strategies, the initial decisions made based on the model's foresight were largely effective, highlighting the value of proactive modeling.

Key Results

  • Cornell's Reopening Decision: The modeling framework was instrumental in Cornell's decision to reopen for in-person instruction in Fall 2020, supported by a comprehensive asymptomatic screening program.
  • Asymptomatic Screening Value: Once-per-week asymptomatic screening of vaccinated undergraduate students was found to provide substantial risk reduction against the Delta variant.
  • Targeted Testing Benefits: More frequent testing, specifically targeting the most social vaccinated students, provided further value in reducing transmission.
  • Vaccination Impact: Moving from 75% to 100% vaccination provided substantial additional protection.
  • Delta Variant Considerations: Even with high vaccination rates, the Delta variant necessitated continued vigilance and interventions like regular asymptomatic screening.
  • Virtual Instruction Risk: Modeling indicated that virtual instruction presented a higher risk of infections compared to residential instruction with interventions.

Limitations and Future Work

  • Model Simplifications: The model assumes homogeneity within subgroups and does not explicitly model individual contact network structures. Contact tracing assumptions are also simplified.
  • Parameter Uncertainty: While the paper addresses parameter uncertainty, the accuracy of the model's predictions remains sensitive to the ranges and distributions assigned to these parameters.
  • Generalizability: While the framework is designed to be generalizable, specific parameter values and intervention effectiveness will vary across different university settings.
  • Future Work: The authors suggest that the modeling framework can be applied to other closed communities (e.g., cruise ships, prisons, military bases) and that future work could explore more nuanced decision-making processes for interventions. They also highlight the need to account for the impact of infection-acquired immunity in addition to vaccination.

Summary updated

Abstract

We consider epidemiological modeling for the design of COVID-19 interventions in university populations, which have seen significant outbreaks during the pandemic. A central challenge is sensitivity of predictions to input parameters coupled with uncertainty about these parameters. Nearly 2 y into the pandemic, parameter uncertainty remains because of changes in vaccination efficacy, viral variants, and mask mandates, and because universities' unique characteristics hinder translation from the general population: a high fraction of young people, who have higher rates of asymptomatic infection and social contact, as well as an enhanced ability to implement behavioral and testing interventions. We describe an epidemiological model that formed the basis for Cornell University's decision to reopen for in-person instruction in fall 2020 and supported the design of an asymptomatic screening program instituted concurrently to prevent viral spread. We demonstrate how the structure of these decisions allowed risk to be minimized despite parameter uncertainty leading to an inability to make accurate point estimates and how this generalizes to other university settings. We find that once-per-week asymptomatic screening of vaccinated undergraduate students provides substantial value against the Delta variant, even if all students are vaccinated, and that more targeted testing of the most social vaccinated students provides further value.
BibTeX
@inproceedings{frazier2021modeling,
  author = {Peter I. Frazier and J. Massey Cashore and Ning Duan and Shane G. Henderson and Alyf Janmohamed and Brian Liu and David B. Shmoys and Jiayue Wan and Yujia Zhang},
  title = {{Modeling for COVID-19 college reopening decisions: Cornell, a case study}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2021},
  doi = {10.1073/pnas.2112532119},
}
Figure