Structurizing Misinformation Stories via Rationalizing Fact-Checks

Shan Jiang, Christo WilsonACL 2021

TL;DR

This paper tackles the challenge of understanding the diverse landscape of misinformation by proposing a novel method to automatically categorize misinformation stories based on the rationales used in fact-checks. By leveraging "rationalized neural models" and a large dataset of Snopes.com articles, they identify ten distinct types of misinformation and analyze their evolution over time, revealing shifts towards more politicized and digitally manipulated content.

What Problem Does This Paper Address?

The proliferation of misinformation online is a significant societal concern, yet existing research often treats "misinformation" as a monolithic concept. This broad definition obscures the nuances between different types of false or misleading content, such as political conspiracies, doctored images, or urban legends. The paper aims to address this by moving beyond a coarse understanding of misinformation and instead seeks to "structurize" it into fine-grained story types. This is crucial for developing more targeted and effective strategies to combat its spread.

Methods

The core of the paper's methodology lies in using "rationalized neural models" to extract "rationales" from fact-checking articles. Rationales are defined as key phrases within a fact-check that explain why a piece of information is deemed false or misleading. The authors experiment with both "hard" and "soft" rationalization techniques.

  • Hard Rationalization: An end-to-end model that first generates rationales and then uses them to make a prediction. This method can be sensitive to hyperparameter tuning.
  • Soft Rationalization: A multi-phase approach where a model first predicts the outcome, then generates importance scores for tokens, which are then binarized into rationales. This method is found to be more robust and achieves better performance.

A key innovation is the introduction of domain knowledge as weak supervision. By providing the models with a lexicon of known misinformation types (e.g., "conspiracy," "satire"), the rationalization process is guided to extract more relevant and specific phrases, improving the quality of the identified rationales.

Datasets

The primary dataset used in this research is a collection of fact-checking articles from Snopes.com, spanning from 1994 to early 2021. This dataset was preprocessed to extract the main article content and verdicts. The authors also utilized two public datasets for evaluating their rationalization methods:

  • Movie Reviews Dataset (Zaidan et al., 2007): Annotated with rationales for sentiment analysis.
  • Personal Attacks Dataset (Carton et al., 2018): Annotated with rationales for identifying personal attacks.

Key Insights and Ideas

  • Fact-checks as a rich source of misinformation typology: The paper demonstrates that the rationales embedded within fact-checks are not just explanations but also implicit categorizations of misinformation types.
  • The evolution of misinformation is towards the political and digital: The study reveals a significant increase in the prevalence of politically charged misinformation, such as conspiracy theories and fabricated content, alongside a rise in digitally manipulated media.
  • Soft rationalization with domain knowledge is key: The research highlights that soft rationalization, augmented with domain knowledge, is the most effective approach for extracting meaningful rationales from fact-checks.
  • Misinformation trends mirror societal shifts: The observed increase in politicized misinformation and digital manipulation can be linked to broader societal trends like increased political polarization and the accessibility of advanced digital editing tools.

Key Results

The paper successfully identifies ten distinct types of misinformation stories by clustering extracted rationales. These clusters empirically confirm existing theorized types (e.g., satire, urban legends) and also reveal novel categories like "clickbait with inflammatory and sensational language" and "entirely fictional content."

  • Evolutionary Trends: Over the last decade, heavily politicized misinformation types (digitally altered images, fabricated content, conspiracy theories) have nearly doubled in prevalence, while less politicized types (legends, hoaxes, mistakes) have decreased.
  • Event-Specific Trends:
    • 2016 vs. 2020 US Elections: Fabricated and misleading content was the most prevalent type in both elections. The 2016 election saw more hoaxes and satires, while the 2020 election showed a similar prevalence of these types.
  • H1N1 vs. COVID-19 Pandemics: The H1N1 pandemic was associated with more legends and tales, whereas the COVID-19 pandemic attracted a higher proportion of conspiracy theories.
  • Model Performance: Soft rationalization with domain knowledge achieved the best rationale extraction performance (Pr(z)) on both the movie reviews and personal attacks datasets, maintaining high prediction accuracy (F1(y)). For the fact-check dataset, the best-performing soft rationalization model used a domain knowledge lexicon with 12 words.

Limitations and Future Work

  • Observational Study Limitations: The research acknowledges that, like observational studies, its computational approach is prone to bias and confounding.
  • Snopes.com Specificity: While Snopes.com is a comprehensive source, it may not be representative of the global misinformation ecosystem. Future work could involve characterizing misinformation comparatively on a global scale.
  • High-Profile Bias: Fact-checkers often focus on high-profile misinformation, meaning low-prevalence, long-tail stories might be underrepresented. Investigating these may require different data sources beyond fact-checks.
  • Lack of Ground Truth for Rationales: The extracted misinformation types are not validated with ground-truth labels due to the current lack of empirical knowledge. Future work aims to leverage these findings to create annotation guidelines and obtain human-identified types for further analysis.

Summary updated

BibTeX
@inproceedings{jiang2021structurizing,
  author = {Shan Jiang and Christo Wilson},
  title = {{Structurizing Misinformation Stories via Rationalizing Fact-Checks}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure