The U.S. COVID-19 Trends and Impact Survey, 2020-2021: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing and vaccination

Joshua A. Salomon, Alex Reinhart, Alyssa Bilinski, Eu Jing Chua, Wichada La Motte‐Kerr, Minttu M. Rönn, Marissa B Reitsma, Katherine Morris, Sarah LaRocca, Tamer H. Farag, Frauke Kreuter, Roni Rosenfeld, Ryan J. TibshiranimedRxiv 2021 · DOI

Abstract

The U.S. COVID-19 Trends and Impact Survey (CTIS) is a massive, cross-sectional, internet-based survey that has operated continuously since April 6, 2020. By inviting a random sample of active Facebook users daily, CTIS collects critical data on symptoms, risks, mitigating behaviors, mental health, testing, and vaccination. With over 20 million responses in its first year, the survey enables high-resolution tracking of trends across fine demographic and geographic scales. This paper describes the survey’s methodology and demonstrates how large-scale online surveys provide real-time indicators that bypass the delays and backlogs inherent in traditional public health reporting, offering a vital supplement to official surveillance systems.

Introduction

Traditional public health surveillance during the COVID-19 pandemic has relied heavily on clinical data, such as confirmed cases, hospitalizations, and mortality rates. While these metrics are essential, they are often hampered by significant reporting lags and are susceptible to artificial fluctuations caused by holiday backlogs or changes in testing availability. Furthermore, clinical data rarely captures the behavioral and socioeconomic drivers of transmission—the "invisible" variables like mask-wearing habits, social distancing, or the economic pressures that force symptomatic individuals to continue working.

The U.S. COVID-19 Trends and Impact Survey (CTIS) was designed to fill these gaps. By providing a continuous, real-time stream of data, the CTIS captures the human dimension of the pandemic across every U.S. county, serving as both an early warning system and a behavioral map of the nation’s response.

Methodology and Data Privacy

The CTIS represents a unique collaboration between academia and industry, specifically Carnegie Mellon University (CMU) and Facebook. The survey’s methodology is built on four pillars:

  1. Sampling: Each day, Facebook invites a new, stratified random sample of its active adult users (18+) to participate via a prompt in their News Feed.
  2. Privacy-First Design: To ensure participant trust, Facebook identifies the users but never sees their survey responses. Users are directed to an external Qualtrics site managed by CMU, ensuring a firewall between social media identity and health data.
  3. Weighting: To correct for the inherent biases of an online sample, researchers apply a two-step weighting process. This adjusts for non-response bias (using Facebook profile attributes like age and gender) and coverage bias (post-stratifying the sample to match the geographic and demographic distribution of the general population based on the 2019 American Community Survey).
  4. Indicators: The survey tracks specific metrics, including "COVID-like illness" (CLI), "CLI-in-community," and highly specific symptoms such as anosmia (loss of smell or taste).

Key Insights: Beyond Official Reporting

The Anosmia Signal

One of the most significant findings from the CTIS was the utility of self-reported anosmia. As a highly specific symptom of COVID-19, loss of smell or taste proved to be a remarkably accurate proxy for hospitalization trends. Unlike general symptoms like fever or cough, which can be confounded by other seasonal illnesses, the anosmia signal provided a "clean" indicator that closely tracked the velocity of the pandemic.

Stability During Holidays

Official case counts often exhibit a "holiday dip," where reported numbers plummet during festive periods due to laboratory closures, only to spike artificially days later. Because the CTIS is automated and continuous, its data remained stable through holidays, providing a more reliable measure of actual community spread during periods of high social activity.

Socioeconomic Transmission Risks

The survey illuminated high-risk "transmission pockets" that clinical data cannot see. For instance, in January 2021, the survey found that over 15% of respondents in food preparation and serving occupations reported working outside the home while experiencing atypical symptoms. This highlights the economic necessity that often overrides public health guidance.

Validation and Results

The scale of the CTIS is unprecedented in public health, averaging over 350,000 participants per week. The survey’s validity is supported by a strong correlation (0.83) between survey-reported positive tests and official state-level surveillance reports. Furthermore, the diagnostic power of specific survey items was confirmed: 34% of those who tested positive for COVID-19 reported a loss of smell or taste, compared to only 1.2% of those who tested negative.

In terms of policy application, the CTIS mapped vaccine acceptance and hesitancy down to the county level. This granular data revealed that while vaccine intent was high among older populations, it varied significantly by race, education, and geography among younger cohorts, allowing for more targeted public health messaging.

Limitations and Future Directions

Despite its scale, the CTIS is subject to selection bias; the sample tends to over-represent college-educated individuals and excludes those without Facebook access. Additionally, self-reported data can be influenced by social desirability bias, particularly regarding protective behaviors like mask-wearing. Current weighting adjustments also do not fully account for education level, a known source of bias.

Looking forward, the authors argue that large-scale digital surveying should become a permanent fixture of global health infrastructure. Efforts are currently underway to integrate CTIS data into deep-learning frameworks to improve the accuracy of "hotspot" predictions, potentially providing a lead time of one to three weeks over traditional clinical data.

Summary updated

Abstract

Abstract The U.S. COVID-19 Trends and Impact Survey (CTIS) is a large, cross-sectional, Internet-based survey that has operated continuously since April 6, 2020. By inviting a random sample of Facebook active users each day, CTIS collects information about COVID-19 symptoms, risks, mitigating behaviors, mental health, testing, vaccination, and other key priorities. The large scale of the survey – over 20 million responses in its first year of operation – allows tracking of trends over short timescales and allows comparisons at fine demographic and geographic detail. The survey has been repeatedly revised to respond to emerging public health priorities. In this paper, we describe the survey methods and content and give examples of CTIS results that illuminate key patterns and trends and help answer high-priority policy questions relevant to the COVID-19 epidemic and response. These results demonstrate how large online surveys can provide continuous, real-time indicators of important outcomes that are not subject to public health reporting delays and backlogs. The CTIS offers high value as a supplement to official reporting data by supplying essential information about behaviors, attitudes toward policy and preventive measures, economic impacts, and other topics not reported in public health surveillance systems. Significance statement The U.S. COVID-19 Trends and Impact Survey (CTIS) has operated continuously since April 6, 2020, collecting over 20 million responses. The largest public health survey ever conducted in the United States, CTIS was designed to facilitate detailed demographic and geographic analyses, track trends over time, and accommodate rapid response to emerging priorities. Using examples of CTIS results illuminating trends in symptoms, risks, mitigating behaviors, testing and vaccination in relation to evolving high-priority policy questions over 12 months of the pandemic, we illustrate the value of online surveys for tracking patterns and trends in COVID outcomes as an adjunct to official reporting, and showcase unique insights that would not be visible through traditional public health reporting.
BibTeX
@misc{salomon2021u,
  author = {Joshua A. Salomon and Alex Reinhart and Alyssa Bilinski and Eu Jing Chua and Wichada La Motte‐Kerr and Minttu M. Rönn and Marissa B Reitsma and Katherine Morris and Sarah LaRocca and Tamer H. Farag and Frauke Kreuter and Roni Rosenfeld and Ryan J. Tibshirani},
  title = {{The U.S. COVID-19 Trends and Impact Survey, 2020-2021: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing and vaccination}},
  year = {2021},
  doi = {10.1101/2021.07.24.21261076},
  howpublished = {medRxiv},
}
Figure