Emotion Analysis of Writers and Readers of Japanese Tweets on Vaccinations

Patrick John Ramos, Kiki Ferawati, Kongmeng Liew, Eiji Aramaki, Shoko WakamiyaACL 2022

TL;DR

This paper investigates the emotions expressed in Japanese tweets about COVID-19 vaccinations, distinguishing between the emotions of tweet writers and their readers. By fine-tuning a BERT model, they found surprise and fear to be the most prominent emotions, and observed weak positive correlations between certain positive emotions (joy, anticipation, trust) and vaccination rates in Japan.

What Problem Does This Paper Address?

Understanding public sentiment towards COVID-19 vaccinations is crucial for effective pandemic control. Vaccine hesitancy, often fueled by negative emotions, can hinder vaccine uptake. While social media offers a rich source of public opinion, existing research on COVID-19 vaccination sentiment in Japanese Twitter data has primarily focused on the emotions of the writers of tweets. This paper addresses the gap by recognizing that the emotions of readers of tweets may differ significantly from those of writers, and that these reader emotions can also provide valuable insights into vaccine hesitancy. By analyzing both writer and reader emotions and correlating them with actual vaccination data, the study aims to offer a more comprehensive understanding of public sentiment surrounding vaccinations.

Methods

The core of this research lies in leveraging and fine-tuning a BERT (Bidirectional Encoder Representations from Transformers) model for emotion analysis. Specifically, the authors fine-tuned a Japanese BERT model (cl-tohoku/bert-base-japanese-v2) on the WRIME dataset, which provides emotion intensity scores for eight of Plutchik's emotions (joy, sadness, anticipation, surprise, anger, fear, disgust, trust) for both tweet writers and readers. The fine-tuning process aimed to predict these intensity scores directly, using mean squared error loss.

For the analysis of vaccine-related tweets, this fine-tuned "emotion analysis BERT" was used in an inference-only mode. The model processed tweets containing keywords related to COVID-19 vaccines and outputted predicted emotion intensity scores for both writers and readers.

Datasets

  • WRIME Dataset: Used for fine-tuning the BERT model. This dataset contains 43,200 Japanese tweets annotated with Plutchik's eight emotions by writers and three reader annotators. Emotion intensity is rated on a scale of 0 to 3.
  • Vaccine-related Tweet Dataset: A newly collected dataset of 20,254 Japanese tweets containing keywords like "vaccine," "Moderna," "Pfizer," or "Omicron," scraped from December 2021. This dataset was used for emotion analysis inference.
  • Our World in Data: Used for vaccination statistics in Japan during December 2021, including the number of vaccinations administered, new people vaccinated, and people fully vaccinated.

Key Insights and Ideas

  • Writer vs. Reader Emotions: The study highlights that writer and reader emotions can differ. For instance, a writer might express fear, while a reader might respond with anger out of disagreement. This distinction is crucial for a nuanced understanding of public sentiment.
  • Surprise and Fear Dominate: The most intensely predicted emotions for vaccine-related tweets were surprise (for writers) and fear (for readers). This suggests that uncertainty and apprehension are significant emotional responses to vaccination discussions online.
  • Weak Correlation with Vaccination Rates: While a direct, strong correlation between specific emotions and vaccination uptake wasn't found, there was a weak positive correlation between certain positive emotions (writer joy, writer/reader anticipation, writer/reader trust) and the number of vaccinations. This hints that positive sentiment might play a subtle role in encouraging vaccination.

Key Results

  • The fine-tuned BERT model achieved a training Mean Squared Error (MSE) of 0.356.
  • On the vaccine-related tweet dataset, surprise was the most intense emotion predicted for writers, while fear was the most intense for readers.
  • A correlation analysis revealed weak positive correlations (p < 0.05) between:
  • Writer joy and vaccinations (r = 0.36)
    • Writer anticipation and vaccinations (r = 0.40)
    • Writer trust and vaccinations (r = 0.40)
    • Reader anticipation and vaccinations (r = 0.44)
    • Reader trust and vaccinations (r = 0.39)
  • No significant correlations were found between emotion scores and the number of people vaccinated or fully vaccinated.

Limitations and Future Work

  • Topic Ambiguity: The collected vaccine-related tweet dataset does not guarantee that the tweets are about vaccinations or that the inferred emotions are directed towards vaccinations. The keywords might appear in unrelated contexts.
  • Aspect-Based Analysis: The current emotion analysis is not aspect-based, meaning it doesn't pinpoint what specific aspect of vaccination the emotion is directed towards. Future work could involve aspect-based emotion analysis to understand emotions related to vaccine brands, side effects, or government policies.
  • Correlation with Other Metrics: The authors suggest extending correlation analyses to other COVID-19 metrics, such as infection rates and reproduction rates, to gain a broader understanding of public sentiment's impact.
  • Understanding Emotion Differences: Further research could explore why writer and reader emotions differ, as observed in the study.

Summary updated

BibTeX
@inproceedings{ramos2022emotion,
  author = {Patrick John Ramos and Kiki Ferawati and Kongmeng Liew and Eiji Aramaki and Shoko Wakamiya},
  title = {{Emotion Analysis of Writers and Readers of Japanese Tweets on Vaccinations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure