Detecting rumors in social media using emotion based deep learning approach

Drishti Sharma, Abhishek SrivastavaPeerJ Comput. Sci 2024 · DOI

TL;DR

This paper introduces SEMTEC, a novel deep learning approach for detecting social media rumors by integrating textual content with emotional and sentiment analysis. By focusing on the primary tweet and its inherent emotional cues, SEMTEC achieves high accuracy, outperforming existing methods and offering a more nuanced understanding of rumor propagation.

What Problem Does This Paper Address?

The rapid dissemination of unverified information, or rumors, on social media poses significant societal risks, from misinformation campaigns to real-world harm. While existing research has explored various machine learning and deep learning techniques for rumor detection, many approaches have focused on propagation patterns or relied solely on textual content. This paper argues that the emotional and sentiment expressed within a tweet are crucial, yet often underutilized, features for accurately identifying rumors, especially in real-time scenarios where analyzing propagation chains is challenging. The authors aim to bridge this gap by developing a model that leverages these semantic aspects of the primary tweet itself.

Methods

The proposed SEMTEC (Sentiment and EMotion driven TransformEr Classifier) method is a deep learning approach that combines several key components:

  • Data Refinement: A robust pre-processing pipeline is employed to clean and normalize textual data, handling contractions, removing noise (HTML tags, URLs), standardizing characters, and ensuring proper capitalization and lowercasing.
  • Textual Feature Extraction: The BERT (Bidirectional Encoder Representations from Transformers) architecture is used to extract contextual embeddings from the pre-processed text, capturing the semantic meaning of each token.
  • Sentiment Feature Extraction: The TextBlob library, a lexicon-based sentiment analysis tool, is utilized to extract sentiment polarity (positive, negative, neutral) from the tweets.
  • Emotion Feature Extraction: A Bi-directional LSTM (Long Short-Term Memory) network, trained on an emotion dataset, is employed to extract emotion tags (joy, sadness, anger, fear, love, surprise) from the tweets.
  • Feature Concatenation and Classification: The extracted textual, sentiment, and emotion features are concatenated to form a comprehensive representation. This combined feature vector is then fed into a classifier (a neural network with dense layers and softmax activation) to predict whether the tweet is a rumor or not.

Datasets

The authors utilized two datasets for their experiments:

  • PHEME: A publicly available dataset containing tweets related to nine real-world events, annotated with rumor and non-rumor labels.
  • Twitter24: A novel dataset curated by the authors from real-time tweets by popular user accounts on Twitter (X), focusing on information circulating in India. This dataset was manually labeled and verified using fact-checking websites.
  • Emotion dataset for NLP: Used to train the Bi-directional LSTM model for emotion extraction.
  • EmoPHEME: A derived dataset from PHEME, annotated with emotion labels, created as a byproduct of this research to facilitate further emotion-related sentiment analysis.

Key Insights and Ideas

  • Emotion as a Rumor Indicator: The study strongly suggests that tweets conveying high emotional states, such as fear or anger, are more readily accepted as true, making them potent vehicles for rumor spread. Conversely, true stories might evoke different emotional responses.
  • Primary Tweet Significance: SEMTEC emphasizes the importance of analyzing the primary tweet itself for rumor detection, especially in real-time scenarios, rather than solely relying on propagation patterns or follow-up comments.
  • Synergy of Features: The combination of textual context, sentiment, and emotion provides a richer representation of a tweet's content, leading to significantly improved rumor detection accuracy compared to methods that use only one or two of these features.
  • Transformer and RNN Complementarity: The use of BERT for contextual embeddings and Bi-LSTMs for emotion extraction demonstrates the power of combining different deep learning architectures to capture diverse aspects of textual data.

Key Results

  • On the PHEME dataset, SEMTEC achieved an accuracy of approximately 92%, outperforming existing methods by around 0.7% in accuracy.
  • On the novel Twitter24 dataset, SEMTEC exceeded standard methods' accuracy by approximately 2%.
  • The ablation study showed that incorporating both emotion and sentiment tags significantly enhanced performance compared to using only textual features or a combination of text and sentiment.
  • SEMTEC demonstrated superior performance across Precision, Recall, and F1-score metrics on the PHEME dataset, with values around 0.91, 0.92, and 0.92, respectively.

Limitations and Future Work

  • Propagation Feature Neglect: The current SEMTEC method primarily focuses on the root or main tweet and does not explicitly utilize propagation features (e.g., how a tweet spreads). The authors suggest this could be an area for future extension.
  • Real-time Processing Delay: The multi-module pipeline (preprocessing, feature extraction) might introduce a slight delay in real-time processing, which could be a concern for immediate rumor detection.
  • Low-Resource Languages: The current work focuses on English tweets. Future work aims to extend rumor detection to low-resource languages, specifically Hindi, by building new datasets and models.
  • Semantic Web Integration: The authors plan to leverage semantic web technologies and knowledge graphs (like Wikidata and DBpedia) to enhance real-time tweet validation and rumor detection.
  • Emoji and Other Features: Exploring the use of emojis and other essential features for rumor detection in low-resource languages is also a future direction.

Summary updated

BibTeX
@article{sharma2024detecting,
  author = {Drishti Sharma and Abhishek Srivastava},
  title = {{Detecting rumors in social media using emotion based deep learning approach}},
  journal = {PeerJ Comput. Sci.},
  year = {2024},
  doi = {10.7717/peerj-cs.2202},
}
Figure