Twitter Sentiment Predicts Affordable Care Act Marketplace Enrollment

Charlene A. Wong, Maarten Sap, Andrew D. Schwartz, Robert Town, Tom Baker, Lyle Ungar, Raina M. MerchantJournal of Medical Internet Research 2015 · DOI

Abstract This correlation indicates Twitter's potential as a real-time monitoring strategy for future marketplace enrollment periods; marketplaces could systematically track Twitter sentiment to more rapidly identify enrollment changes and potentially emerging issues. As a repository of free and accessible consumer-generated opinions, this study reveals a novel role for Twitter in the health policy landscape.

Introduction

Evaluating the impact of major health policies, such as the Affordable Care Act (ACA), traditionally relies on two primary metrics: public opinion polls and official enrollment numbers. While robust, both metrics suffer from a significant "reporting lag" that can span weeks or even months. In the high-stakes, rapidly changing political environment surrounding health care reform, policymakers and health administrators require faster feedback loops to identify emerging issues or shifts in public behavior.

This study explores whether social media—specifically Twitter—can serve as a real-time proxy for public sentiment that correlates with real-world health insurance purchasing decisions. By analyzing nearly one million tweets, researchers aimed to determine if digital chatter could act as a "barometer" for health policy success.

Methodology

The research team employed a retrospective, cross-sectional design focusing on March 2014, the final month of the first ACA open enrollment period.

Data Collection and Sentiment Analysis

Using the Twitter Search API, the team captured 977,303 tweets containing keywords such as "ACA," "#ACA," "Obamacare," and "#Obamacare," as well as tweets directed at official marketplace handles like @HealthCaregov. To establish a baseline, a random sample of approximately 977,000 general tweets from the same period was also collected.

To quantify the "mood" of these tweets, the researchers applied the NRC sentiment lexicon, a comprehensive list of over 54,000 words with pre-assigned sentiment weights. A weighted sentiment score was calculated for each tweet based on word frequency. This automated analysis was validated against human raters, showing a strong correlation (r=.26; P<.001) and high inter-rater reliability.

Geocoding and Statistical Modeling

Because enrollment is tracked at the state level, the team matched tweets to specific U.S. states using GPS coordinates or self-reported location data in user profiles. This process successfully geocoded approximately 46% of the corpus.

Linear regression was then used to test the correlation between a state’s average Twitter sentiment and its enrollment rate (the percentage of eligible individuals who selected a plan). The model adjusted for potential confounders, including whether a state expanded Medicaid or operated its own insurance exchange.

Results

The study revealed that digital sentiment is a significant predictor of tangible economic behavior.

Predictive Power

The core finding was that a 0.10 increase in the sentiment score was associated with an 8.7% increase in enrollment at the state level (95% CI 1.32-16.13; P=.02). This correlation remained robust even when controlling for state-level policy differences, such as Medicaid expansion (P=.02) or the use of state-based versus federal marketplaces (P=.03).

Branding and Sentiment Bias

The analysis also highlighted a distinct split in branding. Tweets using the term "ACA" or "#ACA" had significantly higher sentiment scores than those using "Obamacare" or "#Obamacare." This aligns with political observations that the "Obamacare" label was frequently used more pejoratively in digital spaces.

Furthermore, ACA-related tweets were generally more negative than the random sample of general tweets. This suggests that social media users are more likely to use the platform to debate or complain about policy than to praise it. However, despite this overall negative bias, the relative positivity between states remained a reliable predictor of enrollment rates.

Discussion

The findings suggest that Twitter sentiment can serve as a leading indicator for health policy outcomes. However, several limitations must be considered.

Demographic and Geocoding Bias

Twitter users are not a perfectly representative sample of the U.S. population; they tend to be younger and more tech-savvy. This "digital divide" may influence the relationship between who is tweeting and who is actually eligible for marketplace plans. Additionally, more than half of the tweets could not be geocoded, which may introduce selection bias if users who share their location differ significantly from those who do not.

Correlation vs. Causality

While the study demonstrates a strong correlation, it does not establish causality. It remains unclear whether positive sentiment drives enrollment, or if a smoother, more successful enrollment process in certain states drives more positive sentiment on social media.

Future Directions

This study focused on a single month (March 2014). Future research is required to determine if this correlation holds across different enrollment cycles or for other types of health policies. Nonetheless, the results indicate that marketplaces could systematically track Twitter sentiment to more rapidly identify enrollment changes and emerging issues, providing a novel tool for real-time policy monitoring.

Summary updated

Abstract

This correlation indicates Twitter's potential as a real-time monitoring strategy for future marketplace enrollment periods; marketplaces could systematically track Twitter sentiment to more rapidly identify enrollment changes and potentially emerging issues. As a repository of free and accessible consumer-generated opinions, this study reveals a novel role for Twitter in the health policy landscape.
BibTeX
@article{wong2015twitter,
  author = {Charlene A. Wong and Maarten Sap and Andrew D. Schwartz and Robert Town and Tom Baker and Lyle Ungar and Raina M. Merchant},
  title = {{Twitter Sentiment Predicts Affordable Care Act Marketplace Enrollment}},
  journal = {Journal of Medical Internet Research},
  year = {2015},
  doi = {10.2196/jmir.3812},
}
Figure