Stance Detection in COVID-19 Tweets

Kyle Glandt, Sarthak Khanal, Yingjie Li, Doina Caragea, Cornelia Caragea

Introduction: Beyond Sentiment Analysis

Public health officials rely on understanding public sentiment to gauge "pandemic fatigue"—the point at which adherence to health mandates declines. However, traditional sentiment analysis, which classifies text as "positive" or "negative," often fails to capture the specific perspective an author holds toward a policy. This perspective is known as stance.

For instance, a user might express frustration with the discomfort of a mask (negative sentiment) while still strongly advocating for their use (in-favor stance). To address the lack of specialized benchmarks for pandemic-era mandates, researchers developed the COVID-19-Stance dataset.

Examples of tweets annotated for target, stance, opinion type, and sentiment.

The COVID-19-Stance Dataset

The primary contribution of this work is a manually annotated dataset of 6,133 tweets. These tweets are categorized into four controversial targets:

  1. Anthony S. Fauci, M.D.
  2. Keeping Schools Closed
  3. Stay at Home Orders
  4. Wearing a Face Mask

The dataset captures the evolution of public discourse from March to August 2020, showing a significant surge in activity during the summer months as mandates became more polarized.

Temporal distribution of tweets across the four targets from March to August 2020.

In addition to the labeled data, the researchers curated a "noisy" unlabeled set of 29,640 tweets to facilitate semi-supervised learning.

Methodology: Leveraging Transformers

The study establishes high-performance baselines by moving beyond traditional neural networks (like BiLSTMs and CNNs) to state-of-the-art Transformer models:

  • COVID-Twitter-BERT: This model was pre-trained on a massive corpus of COVID-related tweets, providing a superior linguistic foundation for pandemic-specific terminology compared to general-purpose BERT.
  • Semi-Supervised Learning (BERT-NS): Using a "Noisy Student" approach, a "teacher" model trained on labeled data predicted labels for the 29,000 unlabeled tweets. A "student" model was then trained on the combined set to improve robustness.
  • Domain Adaptation (BERT-DAN): A Dual-view Adaptation Network was employed to transfer knowledge from older stance datasets (such as SemEval 2016) to help the model recognize the general structure of argumentative language.

Key Insights and Results

The researchers found that stance is often implicit, requiring models to "read between the lines" rather than relying on explicit declarations. Sarcasm and humor emerged as significant challenges, often leading to misclassifications.

Performance Highlights: * BERT-based models significantly outperformed traditional RNNs and CNNs. * The best models achieved F1 scores between 0.717 (for "Keeping Schools Closed") and 0.832 (for "Anthony S. Fauci, M.D."). * "Keeping Schools Closed" was the most difficult target to classify, likely due to the emotionally charged and complex language used by stakeholders.

Limitations and Future Directions

While the models show strong performance, they still struggle with the linguistic "noise" of social media, such as sarcasm and a reliance on specific hashtags rather than semantic understanding. Future work will explore multi-task learning frameworks that incorporate sentiment and opinion type (explicit vs. implicit) to further refine stance detection accuracy.

Summary updated

BibTeX
@inproceedings{glandt2021stance,
  author = {Kyle Glandt and Sarthak Khanal and Yingjie Li and Doina Caragea and Cornelia Caragea},
  title = {{Stance Detection in COVID-19 Tweets}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.127},
}
Figure