TL;DR
The US COVID-19 Trends and Impact Survey (CTIS) is a massive, continuously running online survey that leverages Facebook's user base to provide real-time insights into COVID-19 symptoms, behaviors, and vaccination trends. This paper details its methodology and demonstrates its value in supplementing traditional public health data, offering granular insights into the pandemic's impact.
What Problem Does This Paper Address?
The COVID-19 pandemic created an urgent need for rapid, detailed, and geographically specific data to inform public health policy. Traditional surveillance systems, which track cases, hospitalizations, and deaths, often suffer from reporting delays and backlogs, especially around holidays. Furthermore, these systems typically don't capture crucial information about individual behaviors, attitudes towards policies, mental health, or economic impacts. This gap leaves policymakers without the timely and comprehensive data needed for effective decision-making during a rapidly evolving public health crisis. The CTIS aims to fill this void by providing continuous, real-time data on a wide range of COVID-19-related outcomes.
Methods
The CTIS employs a large-scale, cross-sectional, internet-based survey methodology.
- Sampling and Recruitment: The survey recruits a random sample of Facebook active users (ages 18+) daily. Facebook uses stratified random sampling within US states to select users who see a survey invitation in their News Feed.
- Data Collection: Users who click the invitation are directed to a survey hosted on Qualtrics, administered by Carnegie Mellon University (CMU). Facebook does not access individual responses. The survey is available in multiple languages.
- Survey Design: The survey instrument is dynamic, with content revised to address emerging public health priorities. Core items on symptoms, contacts, and demographics are consistently included, while new modules on topics like mask-wearing, vaccination intent, and economic impact have been added over time.
- Weighting: Analytic weights are crucial for addressing potential biases. These weights are developed in two steps:
- Nonresponse Bias Adjustment: Facebook calculates inverse probabilities of survey completion based on user demographics (age, gender, location) and other correlates of nonresponse.
- Coverage Bias Adjustment: Weights are post-stratified to match the survey sample's age, gender, and state/territory distribution to that of the general US population.
- Analysis: The study analyzes both aggregated macrolevel data and individual-level data, stratifying by demographic characteristics (age, race/ethnicity, gender) and geographic divisions (region, state, county).
Datasets
- US COVID-19 Trends and Impact Survey (CTIS): This is the primary dataset introduced and analyzed in the paper. It collected over 20 million responses in its first year of operation (April 2020 - April 2021).
- American Community Survey (ACS) 2019 Supplemental Estimates: Used for comparing the representativeness of the CTIS sample.
- Department of Health and Human Services (HHS) Data: Used for comparing reported COVID-19 hospitalizations.
- Johns Hopkins University Center for Systems Science and Engineering Data: Used for comparing reported COVID-19 cases and mortality.
- CDC Social Vulnerability Index (SVI): Used for stratifying analyses by county-level vulnerability.
Key Insights and Ideas
- Real-time Surveillance Power: Large online surveys like CTIS can provide continuous, real-time indicators of public health outcomes, bypassing the delays inherent in traditional reporting systems. This is crucial for agile policy responses.
- Behavioral Insights: CTIS goes beyond case counts to capture critical behavioral data (e.g., mask-wearing, social distancing, vaccine intent) and attitudinal data, offering a more holistic view of the pandemic's impact.
- Granular Geographic and Demographic Detail: The survey's scale and sampling strategy allow for fine-grained analysis at the state and county levels, revealing regional disparities and demographic differences in symptoms, risks, and behaviors.
- Symptom-Specific Indicators: The survey identified specific symptoms (like loss of smell/taste) that were highly predictive of positive COVID-19 tests, offering valuable syndromic surveillance signals.
- Dynamic Adaptation: The survey's ability to rapidly incorporate new questions in response to evolving public health needs (e.g., adding vaccination questions) makes it a highly adaptable tool.
Key Results
- High Correspondence with Surveillance Data: Despite potential limitations of an internet-based, voluntary sample, CTIS indicators (like "CLI in Community") showed strong correlations (0.83 for reported diagnoses vs. surveillance) and mirrored temporal trends of official data, often with less susceptibility to reporting anomalies.
- Symptom Differences in Positive Cases: Respondents who tested positive for COVID-19 were significantly more likely to report loss of smell or taste (34% vs. 1.2% for others).
- Behavioral Shifts: The survey captured sharp increases in risk-reducing behaviors (reduced contacts, increased mask use) during surges (Nov-Dec 2020) followed by relaxation as cases fell (Jan-April 2021).
- Vaccine Hesitancy Patterns: Fig. 5A and 5B revealed variations in vaccine acceptance and hesitancy by age, race/ethnicity, gender, and importantly, by county, highlighting the need for targeted interventions.
- Occupational Risk Differences: Table 3 showed significant variation in testing positive and working with symptoms across different occupation groups, with food preparation and serving roles showing higher risk.
Limitations and Future Work
- Representativeness: While weighting adjusts for some biases, the sample may still overrepresent college-educated individuals compared to the general US population. Researchers can perform further post-stratification adjustments.
- Self-Report Bias: Outcome measures rely on self-reports, which can be subject to recall bias, social desirability bias, and other measurement errors. However, the paper argues that trends are likely robust even if point estimates are biased.
- Facebook User Base: The reliance on Facebook users means the sample is not fully representative of the entire US population.
- Future Work: The paper suggests that similar online surveys will play a crucial role in future epidemic and pandemic preparedness, supplementing traditional systems with difficult-to-gather information. The CTIS data is publicly available at aggregate levels and through a data use agreement for microdata, enabling further research.
Summary updated