COVID-19 Sensing: Negative Sentiment Analysis on Social Media in China via BERT Model

Tianyi Wang, Ke Lu, Kam-Pui Chow, Qing ZhuIEEE Access 2020 · OpenReview

TL;DR

This paper leverages a fine-tuned BERT model to analyze over a million COVID-19 related posts from Sina Weibo, classifying sentiment and identifying key public concerns. The findings reveal that negative sentiment often centers around the virus's origin, symptoms, impact on work, and public health control measures, offering insights for governments to manage public anxiety.

What Problem Does This Paper Address?

The COVID-19 pandemic presented unprecedented challenges, not only in public health but also in managing public sentiment and anxiety. Understanding how people react and what they worry about is crucial for effective public health responses. Social media platforms like Sina Weibo serve as a vast repository of public opinion, but extracting meaningful insights from this deluge of information requires sophisticated analysis. This paper aims to bridge this gap by analyzing a large corpus of Chinese social media posts to understand public sentiment and concerns related to COVID-19, particularly focusing on negative sentiment which can indicate areas of significant public distress.

Methods

The researchers employed a two-pronged approach:

  1. Sentiment Classification: They fine-tuned a pre-trained Chinese BERT (Bidirectional Encoder Representations from Transformers) model for sentiment classification. BERT is a powerful deep learning model for natural language processing, known for its ability to understand context. By fine-tuning it on a specific dataset, they trained it to categorize Weibo posts into three sentiment classes: positive, neutral, and negative.
  2. Topic Extraction and Analysis: To understand what people were talking about within each sentiment category, they used the TF-IDF (Term Frequency-Inverse Document Frequency) model. TF-IDF is a statistical measure used to evaluate the importance of a word to a document in a collection. This allowed them to extract the most significant keywords (topics) from posts, particularly those with negative sentiment.

They then performed trend analysis on the sentiment over time and thematic analysis on the extracted topics to identify patterns and key concerns.

Datasets

The study utilized a dataset of 999,978 randomly selected COVID-19 related Weibo posts collected from January 1, 2020, to February 18, 2020. This dataset was initially sourced by crawling posts related to 230 COVID-19 key phrases. A subset of these posts (120,000) was manually labeled for sentiment (positive, neutral, negative) by volunteers, with the final training and testing sets derived from this labeled data.

Key Insights and Ideas

  • BERT's Efficacy: The fine-tuned BERT model achieved a respectable 75.65% accuracy in sentiment classification, outperforming several other baseline NLP algorithms. This highlights BERT's capability in understanding nuanced sentiment in social media text.
  • Negative Sentiment as a Barometer: Posts with negative sentiment are particularly valuable for understanding public concerns and potential social disruptions. The study found that negative sentiment often spikes around significant events, like the confirmation of human-to-human transmission.
  • Four Pillars of Public Concern: The analysis revealed that negative sentiment regarding COVID-19 primarily revolved around four key themes:
  • Origin: Discussions about where the virus came from (e.g., "Gamey Food," "Bat," "Conspiracy Theory").
  • Symptom: Concerns about physical manifestations of the illness (e.g., "Fever," "Cough").
  • Production Activity: Worries related to work and economic life (e.g., "Go to Work," "Resume Work," "School New Semester Beginning").
  • Public Health Control: Reactions to government measures and information transparency (e.g., "Temperature Taking," "Coronavirus Cover-up," "City Shutdown").
  • Information Transparency is Key: The study suggests that transparent information sharing and scientific guidance are crucial for alleviating public concerns and managing depression related to the pandemic.

Key Results

  • Sentiment Distribution: Out of the analyzed posts, 56.2% were neutral, 27.4% were positive, and 16.4% were negative.
  • Topic Prevalence: The most prominent negative sentiment topics included "Gamey Food" (3.08%), "Bat" (2.70%), "Fever" (2.13%), and "Go to Work" (1.94%).
  • Trend Correlation: A significant surge in negative sentiment posts was observed around January 20, 2020, coinciding with the confirmation of human-to-human transmission. This trend generally mirrored the increase in confirmed COVID-19 cases.
  • Distinctive Negative Topics: The study identified 11 topics that were distinctively associated with negative sentiment, further categorized into the four themes mentioned above.

Limitations and Future Work

  • Data Bias: The study acknowledges that using only Sina Weibo data might introduce bias, as it doesn't capture sentiments expressed on other social media platforms.
  • Topic Selection Threshold: Focusing only on topics appearing in more than 1% of posts might lead to overlooking less frequent but potentially important concerns.
  • Future Directions: The authors suggest improving the classification model's accuracy and expanding the analysis to include data from multiple social media platforms and different countries to gain a more comprehensive understanding. They also propose developing an online platform for real-time sentiment monitoring during future crises.

Summary updated

Abstract

Coronavirus disease 2019 (COVID-19) poses massive challenges for the world. Public sentiment analysis during the outbreak provides insightful information in making appropriate public health responses. On Sina Weibo, a popular Chinese social media, posts with negative sentiment are valuable in analyzing public concerns. 999,978 randomly selected COVID-19 related Weibo posts from 1 January 2020 to 18 February 2020 are analyzed. Specifically, the unsupervised BERT (Bidirectional Encoder Representations from Transformers) model is adopted to classify sentiment categories (positive, neutral, and negative) and TF-IDF (term frequency-inverse document frequency) model is used to summarize the topics of posts. Trend analysis and thematic analysis are conducted to identify characteristics of negative sentiment. In general, the fine-tuned BERT conducts sentiment classification with considerable accuracy. Besides, topics extracted by TF-IDF precisely convey characteristics of posts regarding COVID-19. As a result, we observed that people concern four aspects regarding COVID-19, the virus Origin (Gamey Food, 3.08%; Bat, 2.70%; Conspiracy Theory, 1.43%), Symptom (Fever, 2.13%; Cough, 1.19%), Production Activity (Go to Work, 1.94%; Resume Work, 1.12%; School New Semester Beginning, 1.06%) and Public Health Control (Temperature Taking, 1.39%; Coronavirus Cover-up, 1.26%; City Shutdown, 1.09%). Results from Weibo posts provide constructive instructions on public health responses, that transparent information sharing and scientific guidance might help alleviate public concerns.
BibTeX
@inproceedings{wang2020covid,
  author = {Tianyi Wang and Ke Lu and Kam-Pui Chow and Qing Zhu},
  title = {{COVID-19 Sensing: Negative Sentiment Analysis on Social Media in China via BERT Model}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=L1lFmjEkwXF},
}
Figure