Deep Learning Reveals Patterns of Diverse and Changing Sentiments Towards COVID-19 Vaccines Based on 11 Million Tweets

Hanyin Wang, Meghan R. Hutch, Yikuan Li, Adrienne S. Kline, Sebastian Otero, Leena B. Mithal, Emily S. Miller, Andrew Naidech, Yuan LuoarXiv 2022 · DOI · arXiv

Abstract

This study utilizes deep learning (XLNet) and computer vision to analyze over 11 million COVID-19 vaccine-related tweets from March 2020 to March 2022. By integrating sentiment analysis with demographic extraction and geospatial parsing, the researchers identified that positive social media sentiment serves as a leading indicator for real-world vaccine uptake. The analysis also revealed significant information gaps for pregnant populations and quantified demographic disparities in vaccine hesitancy. The findings suggest that real-time "infodemiology" can provide critical early-warning signals for public health officials.

Introduction

Public health officials often struggle to gauge public sentiment in real-time. Traditional methods, like surveys, are frequently limited by small sample sizes, high costs, and significant time lags. Furthermore, these methods often fail to capture the nuances of how different subpopulations—stratified by race, age, or pregnancy status—react to shifting news cycles.

The authors sought to bridge this gap by using "infodemiology"—the study of online information to inform public health. By analyzing a massive, longitudinal dataset from Twitter, they aimed to understand not just what people were saying, but who was saying it and how those digital attitudes translated into physical actions (vaccine uptake).

Methods

The researchers built a sophisticated multi-stage pipeline to process 11.2 million tweets from March 2020 to March 2022:

  • Sentiment Analysis: They used XLNet, a state-of-the-art generalized autoregressive pretraining model. They fine-tuned it first on a general Twitter dataset (SemEval) and then on a custom set of 7,700 hand-annotated tweets, achieving an accuracy of 0.92.
  • Demographic Extraction: To identify the "who," the team employed the deepface framework. This allowed them to estimate age, gender, and race/ethnicity from user profile images.
  • Geospatial Parsing: They used the mordecai package to resolve free-text location strings from user profiles into structured geographic data.
  • Pregnancy Sub-analysis: Using regular expressions, they isolated a subset of tweets specifically related to vaccination during pregnancy to compare against general population trends.

Results

Demographic Composition

The study analyzed 11,211,672 COVID-19 vaccine-related tweets from 2,203,681 unique users. The demographic analysis revealed significant skews in the active Twitter population discussing vaccines. As shown in Figure 1, the user base was approximately 89% male and 80.2% White, reflecting known demographic biases on the platform.

Demographic distribution of users and tweets by race, age, and gender.

Sentiment as a Leading Indicator

A key finding was that "crests" in positive sentiment on Twitter preceded upswings in actual vaccine administration. This suggests that social media monitoring could serve as an early-warning system for public health demand. Conversely, negative sentiment was highly reactive to specific events; for example, the highest peak of negative sentiment occurred on April 13, 2021, following reports of rare blood clotting cases associated with the Johnson & Johnson vaccine.

The Pregnancy Information Gap

Because pregnant individuals were excluded from initial clinical trials, there was a massive delay in positive sentiment and vaccine uptake in this group. While positive sentiment in the general population dominated by early 2021, the pregnancy sub-group did not peak until August 2021, coinciding with official CDC recommendations based on new safety data.

Demographic Disparities

Vaccine hesitancy varied across demographic lines. Male users and Asian subpopulations (in the English-speaking dataset) showed higher percentages of negative sentiment, often citing concerns about clinical trial validity and unpredictable long-term risks.

Discussion and Limitations

The authors identify several constraints to this approach. The study only considered English-language tweets, limiting generalizability in non-English speaking regions. Additionally, demographic detection from profile pictures is not 100% accurate and can be influenced by lighting or non-personal photos. Finally, Twitter users are not a perfect proxy for the general global population, as they tend to be younger and more tech-savvy.

Despite these limitations, the study demonstrates that this pipeline can be deployed in near-real-time for future pandemics to provide early signals of vaccine hesitancy and allow for more tailored, evidence-based public health messaging.

Summary updated

BibTeX
@misc{wang2022deep,
  author = {Hanyin Wang and Meghan R. Hutch and Yikuan Li and Adrienne S. Kline and Sebastian Otero and Leena B. Mithal and Emily S. Miller and Andrew Naidech and Yuan Luo},
  title = {{Deep Learning Reveals Patterns of Diverse and Changing Sentiments Towards COVID-19 Vaccines Based on 11 Million Tweets}},
  year = {2022},
  eprint = {2207.10641},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2207.10641},
}
Figure