Leveraging Community and Author Context to Explain the Performance and Bias of Text-Based Deception Detection Models

Galen Weld, Ellyn Ayton, Tim Althoff, Maria Glenski

Introduction

Automated deception detection typically focuses on the linguistic content of a message—analyzing syntax, sentiment, and lexical features. While state-of-the-art NLP models often report high aggregate performance metrics, these single scores can mask significant underlying biases. This research explores how standard models behave across different sub-populations on social media, specifically examining how the context of the author and the community (subreddit) influences model accuracy and confidence.

Methodology

The researchers utilized a standard, high-performing "ContentOnly" Long Short-Term Memory (LSTM) neural network as a probe. The model was trained to classify news as either Deceptive or Credible based solely on text.

To analyze the impact of context, the team computed hundreds of metadata characteristics for both Authors and Communities using historical Reddit data from 2016. These features included:

  • Factualness and Bias: Based on labels from Media Bias/Fact Check (MBFC), measuring how often a user or community links to low-fact or extreme-bias sources.
  • Engagement Metrics: Average upvotes and downvotes for posts.
  • Activity Patterns: The Gini coefficient of contributor activity within a community.

Ground truth labels were derived from established datasets (Volkova et al., 2017) and MBFC, covering categories like propaganda, satire, conspiracy, and varying levels of factualness.

The Deception of Aggregate Metrics

While the model achieved an overall Macro F1 score of approximately 0.80, this figure is misleading. A Kernel Density Estimation (KDE) of F1 scores across individual communities reveals a bimodal distribution, showing that for a significant portion of the test set, the model performs poorly.

Figure 5: KDE of F1 scores across communities compared to the Macro F1. The distribution shows a significant number of communities where performance falls well below the aggregate average.

As shown in the figure above, the aggregate F1 score (dashed line) fails to represent the "blind spots" where the model's F1 drops below 0.40 for nearly 20% of communities.

Contextual Performance Variance

The model's performance varies wildly depending on the subreddit environment. By grouping subreddits into quartiles based on model performance, distinct patterns emerge regarding the "type" of community where the model succeeds or fails.

Figure 3: Subreddit word clouds grouped by performance quartiles. Quartile 4 (highest performance) contains polarized or low-credibility subreddits, while Quartile 1 (lowest performance) contains general news and regional subreddits.

In Quartile 4 (highest performance), we find subreddits like The_Donald, conspiracy, and HillaryForPrison. Conversely, Quartile 1 (lowest performance) includes general news hubs like news, canada, and unitedkingdom. This suggests that the model is highly effective at identifying the specific linguistic "style" of deception prevalent in low-credibility or highly polarized circles but struggles when deceptive content is posted in more "reputable" or neutral contexts.

Key Findings and Bias

  1. Author History as a Predictor: An author’s history of posting deceptive content is a powerful predictor of model success (correlation of 0.85). The model essentially learns to identify the stylistic markers of "junk news" authors.
  2. The "Reputable" Failure: There is a negative correlation between an author's historical factualness and model performance. As the perceived reliability of an author increases, the model's ability to catch their deceptive posts decreases.
  3. The Popularity Paradox: Surprisingly, the model is more confident in correctly identifying deceptive posts when they are highly upvoted. This implies that "viral" misinformation may possess distinct, easily identifiable linguistic markers that the model can readily exploit.

Conclusion

This study demonstrates that text-based deception detection is not context-neutral. Models are prone to an algorithmic bias where they over-perform on known "low-credibility" sources while being easily fooled by deceptive content from otherwise "trustworthy" authors. Future work must move toward context-aware models that integrate author and community metadata to mitigate these blind spots and provide a more robust defense against misinformation.

Summary updated

BibTeX
@inproceedings{weld2021leveraging,
  author = {Galen Weld and Ellyn Ayton and Tim Althoff and Maria Glenski},
  title = {{Leveraging Community and Author Context to Explain the Performance and Bias of Text-Based Deception Detection Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure