Abstract
Understanding the scope, prevalence, and impact of the COVID-19 pandemic response will be a rich ground for research for many years. Key to the response to COVID-19 was the non-pharmaceutical intervention (NPI) measures, such as mask mandates or stay-in-place orders. For future pandemic preparedness, it is critical to understand the impact and scope of these interventions. Given the ongoing nature of the pandemic, existing NPI studies covering only the initial portion provide only a narrow view of the impact of NPI measures. This paper describes a dataset of NPI measures taken by counties in the U.S. state of Virginia that include measures taken over the first two years of the pandemic beginning in March 2020. This data enables analyses of NPI measures over a long time period that can produce impact analyses on both the individual NPI effectiveness in slowing the pandemic spread, and the impact of various NPI measures on the behavior and conditions of the different counties and state.
Introduction
While a wealth of data exists regarding COVID-19 case counts and hospitalizations, the record of the policies intended to curb those numbers remains surprisingly fragmented. Most existing Non-Pharmaceutical Intervention (NPI) datasets focus on national or state-level mandates and often only cover the initial months of the pandemic. However, in the United States, many critical decisions—such as closing local businesses or mandating masks in schools—were enacted at the county level.
Collecting this information is challenging due to "information decay." Local government websites update frequently, and official announcements made via social media or local newspapers often become unsearchable or disappear entirely over time. Without a centralized, validated record of these local actions, researchers cannot accurately model which interventions were effective in specific community contexts.
Methodology
To address this gap, researchers utilized a "human-in-the-loop" approach to data collection and validation, employing a team of trained researchers to capture ephemeral data from social media, government websites, and local news archives.
The workflow prioritized diversity in search strategies, encouraging team members to develop unique discovery processes to ensure that mandates missed by one method might be captured by another. The process involved three main phases:
- Discovery: Identifying mandates through varied digital channels.
- Documentation: Capturing permanent screenshots of sources to combat "link rot."
- Validation: A rigorous two-step verification process to ensure the correctness and completeness of every entry.
Dataset Characteristics and Results
The resulting dataset, hosted on Zenodo, covers 120 counties and independent cities in Virginia, representing 90% of the state's population from March 9, 2020, to March 23, 2022. The primary output includes 509 validated mandates across six categories: business closures, K-12 school closures, college/university closures, religious closures, lockdowns, and mask mandates.
Key findings from the dataset include:
- Temporal Depth: The average mandate length was approximately 226 days, providing a significantly longer window for longitudinal study than previous datasets.
- Human Necessity: The validation phase revealed an initial error rate of 18.46%, highlighting that automated web-scraping alone is insufficient for capturing the nuances of local policy.
- Policy Nuance: By utilizing metadata tags such as "Hybrid," "All Virtual," or "Optional," the dataset captures complexities that binary (Open/Closed) models miss.
Discussion and Future Directions
The fragility of digital history is a central theme of this research. Many local mandates were only announced on social media; as accounts are deleted, that policy history is lost. This dataset serves as a critical archive for Virginia, allowing researchers to control for state-wide mandates and focus purely on how county-level variations impacted health outcomes.
However, limitations remain. The dataset tracks when a mandate was ordered, but not public compliance. Furthermore, the manual nature of this "human-in-the-loop" framework presents scalability challenges for a national-level effort. Future research will aim to integrate this policy data with mobility metrics to measure intervention effectiveness and identify which specific counties acted as outliers in the state's overall pandemic trajectory.
Conclusion
This comprehensive, human-validated dataset provides a granular look at how local policies evolved far beyond the initial "first wave." By preserving decaying digital records, this work enables a deeper understanding of NPI effectiveness and provides a framework for future pandemic preparedness.
Summary updated