FacTweet: Profiling Fake News Twitter Accounts

Bilal Ghanem, Simone Paolo Ponzetto, Paolo Rosso

Introduction

The proliferation of misinformation on social media has led to significant research into automated fake news detection. Traditional approaches often focus on verifying individual claims or specific tweets. However, this granular focus faces two primary challenges: the "mixing" strategy, where deceptive accounts blend legitimate news with misinformation to build credibility and evade detection, and the high cost of annotating individual tweets.

This paper introduces FacTweet, a system designed to profile Twitter accounts by identifying their "hidden signature"—the unique behavioral patterns and sequential posting styles that distinguish deceptive accounts from factual news outlets. By shifting the focus from the message to the messenger, FacTweet provides a robust pre-filtering step in the fight against digital deception.

Methodology

FacTweet shifts the analysis from isolated tweets to chronological "chunks" of an account's timeline. The core hypothesis is that deceptive accounts—whether they spread propaganda, hoaxes, or clickbait—follow specific temporal patterns that are invisible when tweets are viewed in isolation.

Architecture and Feature Engineering

The model processes a user's timeline by sorting tweets chronologically and dividing them into chunks. The authors found that a chunk size of 20 tweets yielded the optimal balance between context and computational efficiency. Each tweet within a chunk is represented by a dense vector incorporating three main categories of features:

  • Semantic and Lexical Features: Emotion (15 types), Sentiment (positive/negative), and Morality (10 categories like "fairness" or "betrayal").
  • Stylistic Features: Punctuation usage, hashtags, mentions, and uppercase ratios, which often signal the "tone" of an account.
  • Word Embeddings: Pre-trained GloVe embeddings to capture broader linguistic context and semantic relationships.

These vectors are fed into a Long Short-Term Memory (LSTM) network, which models the temporal dependencies between tweets. An attention mechanism is applied to the LSTM outputs to help the model focus on the most informative tweets within a chunk, followed by a dropout layer for regularization and a softmax classifier for final account categorization into four types: Propaganda (P), Clickbait (C), Hoax (H), and Factual (R).

The FacTweet architecture: Timeline chunks are processed through an LSTM with attention for account-level classification.

Dataset

The researchers curated a dataset of 176 Twitter accounts, comprising 144 non-factual accounts and 32 factual accounts from verified news outlets. Despite the relatively small number of unique accounts—reflecting the difficulty of obtaining high-quality ground truth labels for account-level intent—the dataset contains over 520,000 tweets. This scale provides a rich temporal history for each profile, allowing the model to learn complex behavioral signatures.

Experimental Results

The evaluation demonstrates that modeling tweets in sequential chunks significantly outperforms tweet-level classification. FacTweet achieved an Accuracy of 0.74 and a Macro-F1 of 0.565, outperforming all baselines, including standard Bag-of-Words and specialized tweet-level classifiers like Tweet2vec.

Comparison of FacTweet against baselines and tweet-level approaches, showing the superiority of chunk-level modeling.

As shown in the results table, the best tweet-level approach (Logistic Regression using all features) only managed a Macro-F1 of 0.39. This gap proves that the sequential "chunk" approach is far superior for identifying the intent of an account. Ablation studies further revealed that word embeddings and stylistic features were the most critical components; removing word embeddings led to a sharp decline in performance, highlighting the importance of semantic context in identifying deceptive narratives.

Conclusion and Future Work

FacTweet demonstrates that account-level profiling using sequential modeling is a powerful tool for misinformation detection. By identifying the "hidden signature" of deceptive accounts, the model can flag suspicious actors even when they attempt to mask their intent with legitimate content.

Future research will focus on expanding the dataset to include more diverse account "personalities," exploring multilingual capabilities using cross-lingual embeddings, and incorporating explicit timestamp information to analyze the "rhythm" of posting as an additional behavioral feature.

Summary updated

Abstract

We present an approach to detect fake news in Twitter at the account level using a neural recurrent model and a variety of different semantic and stylistic features. Our method extracts a set of features from the timelines of news Twitter accounts by reading their posts as chunks, rather than dealing with each tweet independently. We show the experimental benefits of modeling latent stylistic signatures of mixed fake and real news with a sequential model over a wide range of strong baselines.
BibTeX
@inproceedings{ghanem2020factweet,
  author = {Bilal Ghanem and Simone Paolo Ponzetto and Paolo Rosso},
  title = {{FacTweet: Profiling Fake News Twitter Accounts}},
  booktitle = {SLSP},
  year = {2020},
  doi = {10.1007/978-3-030-59430-5\_3},
}
Figure