Leveraging Community and Author Context to Explain the Performance and Bias of Text-Based Deception Detection Models

Galen Weld, Ellyn Ayton, Tim Althoff, Maria Glenski

TL;DR

This paper investigates why deception detection models perform differently across various online communities and authors. By analyzing community and author characteristics, they reveal that model performance is not uniform and can disproportionately affect certain user groups, highlighting the need for more nuanced evaluation beyond aggregate metrics.

What Problem Does This Paper Address?

The proliferation of deceptive news on social media poses a significant challenge to public trust. While Natural Language Processing (NLP) models offer automated solutions for detecting such content, their evaluation has traditionally relied on aggregate performance metrics like F1 scores. This approach can mask critical issues: a model might perform well overall but fail significantly on specific sub-populations of users or communities. This paper addresses the crucial question of whether deception detection models exhibit biases and performance disparities across different online communities and authors, and seeks to explain these variations by incorporating contextual information about who is posting and where.

Methods

The authors leverage an existing LSTM-based deception detection model, referred to as the "ContentOnly model," which relies solely on text and lexical features. To explain its performance and biases, they introduce a novel approach by computing hundreds of community and author characteristics. These characteristics are derived from two primary sources: annotations of news sources for their factualness and bias (from Media Bias/Fact Check and Volkova et al.), and the historical posting behavior of users and communities on Reddit. They then analyze the correlation between these contextual features and the model's performance (measured by F1 score, precision, and recall) and its prediction confidence. This allows them to identify which community and author attributes are associated with better or worse model performance.

Datasets

The study utilizes data collected from Twitter and Reddit, specifically from the 2016 time period, aligning with annotations from Volkova et al. (2017). For Twitter, they used posts from official news source accounts. For Reddit, they collected link-posts from the Pushshift monthly archives. The ground truth for deception was established using annotations from Volkova et al. (2017), which categorize news sources as "Credible" or "Deceptive." Additionally, they incorporated annotations from Media Bias/Fact Check (MBFC) for factualness and political bias. The data was down-sampled to mitigate class imbalance and then split into training and testing sets.

Key Insights and Ideas

  • Beyond Aggregate Metrics: The paper strongly advocates for moving beyond overall F1 scores. Even a model with a high aggregate F1 score can have drastically poor performance on specific communities, indicating a potential for unfairness or unreliability.
  • Author Matters More Than Community (for Prediction): While both author and community characteristics are correlated with model performance, author characteristics were found to be a stronger predictor of high model performance, particularly for authors with a history of submitting low-factual or deceptive content.
  • Community Acceptance Amplifies Model Confidence: The model's confidence in its predictions is significantly correlated with community acceptance, especially for highly accepted posts. This effect is more pronounced for deceptive content, suggesting that the model is more certain when a post is both deceptive and popular within its community.
  • Bias and Factualness as Predictors: The study found that model performance is correlated with the prevalence of certain types of content within communities and by authors. Specifically, a higher proportion of "right bias," "low factual," and "deceptive" links are associated with stronger model performance, while "left bias" and "high factual" content are linked to weaker performance.

Key Results

The ContentOnly model achieved an overall F1 score of 0.79 on the test set. However, the study revealed significant performance variability across communities, with 20% of communities having an F1 score below 0.40. Author characteristics were found to be stronger predictors of model performance than community characteristics. For instance, the author's tendency to submit low-factual or deceptive content was strongly correlated with better model performance. Conversely, communities and authors with a higher tendency to engage with high-factual and center-biased news sources showed weaker model performance. Correlations between aggregated community/author characteristics and performance metrics were generally stronger than post-level correlations, with maximum correlation values reaching 0.70 for communities and 0.85 for authors.

Limitations and Future Work

The authors acknowledge that their analysis focuses exclusively on posts and could be extended to include comments. They also note that while they aimed to mitigate bias, their coverage of non-news links might underrepresent the true extent. Future work could explore how to extend these contextual analyses to other NLP tasks beyond deception detection and investigate the impact of model design choices on performance across different sub-populations. The paper emphasizes the importance of considering the intended application of a deception detection model when evaluating its performance, as different use cases (e.g., flagging for human review vs. automated enforcement) may require different precision-recall trade-offs.

Summary updated

BibTeX
@inproceedings{weld2021leveraging,
  author = {Galen Weld and Ellyn Ayton and Tim Althoff and Maria Glenski},
  title = {{Leveraging Community and Author Context to Explain the Performance and Bias of Text-Based Deception Detection Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure