Abstract
The U.S. COVID-19 Trends and Impact Survey (CTIS) is a massive, cross-sectional, internet-based survey that has operated continuously since April 6, 2020. By inviting a random sample of active Facebook users daily, CTIS collects critical data on symptoms, risks, mitigating behaviors, mental health, testing, and vaccination. With over 20 million responses in its first year, the survey enables high-resolution tracking of trends across fine demographic and geographic scales. This paper describes the survey’s methodology and demonstrates how large-scale online surveys provide real-time indicators that bypass the delays and backlogs inherent in traditional public health reporting, offering a vital supplement to official surveillance systems.
Introduction
Traditional public health surveillance during the COVID-19 pandemic has relied heavily on clinical data, such as confirmed cases, hospitalizations, and mortality rates. While these metrics are essential, they are often hampered by significant reporting lags and are susceptible to artificial fluctuations caused by holiday backlogs or changes in testing availability. Furthermore, clinical data rarely captures the behavioral and socioeconomic drivers of transmission—the "invisible" variables like mask-wearing habits, social distancing, or the economic pressures that force symptomatic individuals to continue working.
The U.S. COVID-19 Trends and Impact Survey (CTIS) was designed to fill these gaps. By providing a continuous, real-time stream of data, the CTIS captures the human dimension of the pandemic across every U.S. county, serving as both an early warning system and a behavioral map of the nation’s response.
Methodology and Data Privacy
The CTIS represents a unique collaboration between academia and industry, specifically Carnegie Mellon University (CMU) and Facebook. The survey’s methodology is built on four pillars:
- Sampling: Each day, Facebook invites a new, stratified random sample of its active adult users (18+) to participate via a prompt in their News Feed.
- Privacy-First Design: To ensure participant trust, Facebook identifies the users but never sees their survey responses. Users are directed to an external Qualtrics site managed by CMU, ensuring a firewall between social media identity and health data.
- Weighting: To correct for the inherent biases of an online sample, researchers apply a two-step weighting process. This adjusts for non-response bias (using Facebook profile attributes like age and gender) and coverage bias (post-stratifying the sample to match the geographic and demographic distribution of the general population based on the 2019 American Community Survey).
- Indicators: The survey tracks specific metrics, including "COVID-like illness" (CLI), "CLI-in-community," and highly specific symptoms such as anosmia (loss of smell or taste).
Key Insights: Beyond Official Reporting
The Anosmia Signal
One of the most significant findings from the CTIS was the utility of self-reported anosmia. As a highly specific symptom of COVID-19, loss of smell or taste proved to be a remarkably accurate proxy for hospitalization trends. Unlike general symptoms like fever or cough, which can be confounded by other seasonal illnesses, the anosmia signal provided a "clean" indicator that closely tracked the velocity of the pandemic.
Stability During Holidays
Official case counts often exhibit a "holiday dip," where reported numbers plummet during festive periods due to laboratory closures, only to spike artificially days later. Because the CTIS is automated and continuous, its data remained stable through holidays, providing a more reliable measure of actual community spread during periods of high social activity.
Socioeconomic Transmission Risks
The survey illuminated high-risk "transmission pockets" that clinical data cannot see. For instance, in January 2021, the survey found that over 15% of respondents in food preparation and serving occupations reported working outside the home while experiencing atypical symptoms. This highlights the economic necessity that often overrides public health guidance.
Validation and Results
The scale of the CTIS is unprecedented in public health, averaging over 350,000 participants per week. The survey’s validity is supported by a strong correlation (0.83) between survey-reported positive tests and official state-level surveillance reports. Furthermore, the diagnostic power of specific survey items was confirmed: 34% of those who tested positive for COVID-19 reported a loss of smell or taste, compared to only 1.2% of those who tested negative.
In terms of policy application, the CTIS mapped vaccine acceptance and hesitancy down to the county level. This granular data revealed that while vaccine intent was high among older populations, it varied significantly by race, education, and geography among younger cohorts, allowing for more targeted public health messaging.
Limitations and Future Directions
Despite its scale, the CTIS is subject to selection bias; the sample tends to over-represent college-educated individuals and excludes those without Facebook access. Additionally, self-reported data can be influenced by social desirability bias, particularly regarding protective behaviors like mask-wearing. Current weighting adjustments also do not fully account for education level, a known source of bias.
Looking forward, the authors argue that large-scale digital surveying should become a permanent fixture of global health infrastructure. Efforts are currently underway to integrate CTIS data into deep-learning frameworks to improve the accuracy of "hotspot" predictions, potentially providing a lead time of one to three weeks over traditional clinical data.
Summary updated