What to Annotate: Retrieving Lexical Markers of Conspiracy Discourse from an Italian-English Corpus of Telegram Data

Costanza Marini, Elisabetta JezekCOLING 2024

TL;DR

This paper tackles the challenge of automatically identifying conspiracy theory (CT) discourse online by creating and analyzing a novel bilingual (English-Italian) corpus of Telegram comments. They identify key linguistic markers, like specific types of vocabulary and conceptual metaphors, that can be used to train future AI systems for CT detection.

What Problem Does This Paper Address?

Conspiracy theories (CTs) are increasingly prevalent and can distort public perception of reality. Automatically identifying CT discourse is crucial for understanding and potentially mitigating its spread, especially on social media platforms. While previous research has explored CT language, there's a lack of comprehensive, cross-linguistic studies that define shared annotation schemes for CT discourse. This paper aims to bridge this gap by identifying statistically significant linguistic markers that are common across different CTs and languages, paving the way for more robust automatic detection systems.

Methods

The researchers employed a corpus-driven linguistic analysis using the Sketch Engine online platform. They focused on two key functions: keywords and terms extraction. These functions compare the frequency of words and multi-word expressions in a specific "focus corpus" (their CT data) against a larger "reference corpus" (general web data). Words and phrases that appear significantly more often in the focus corpus are identified as statistically relevant. This approach allowed them to pinpoint linguistic features that are characteristic of CT discourse. They also utilized a user-friendly, open-source annotation tool called Taguette for their data annotation process.

Datasets

The primary dataset introduced is the Complotto corpus, a 40,000-token English-Italian bilingual corpus of conspiracy-oriented Telegram comments. This corpus consists of 658 comments (317 Italian, 341 English) extracted from three Italian and three English Telegram channels dedicated to specific CTs: Flat Earth, vaccine conspiracies, and climate change hoaxes. The data was carefully annotated by two human annotators to distinguish between "conspiratorial" and "non-conspiratorial" comments, achieving a high level of inter-annotator agreement.

Key Insights and Ideas

  • Linguistic Fingerprints of CTs: CT discourse isn't just about the content; it has distinct linguistic patterns. Identifying these patterns is key to automatic detection.
  • "Insider vs. Outsider" Mentality: The language often reflects a clear division between an "insider" group (those who know the "truth") and an "outsider" group (the "system" or "elites" perpetuating deception). This is reflected in pronoun usage like "noi" (we) and "ci" (us) in Italian.
  • The "Hidden Truth" Narrative: CTs frequently employ vocabulary that suggests an alternative, hidden truth exists behind the official narrative. This includes terms related to debunking and revealing deception.
  • Institutions as Abusers: A recurring conceptual metaphor identified is "INSTITUTIONS ARE ABUSERS," where governments, scientists, and other authorities are portrayed as actively deceiving or harming the public.
  • Evidentiality and Epistemic Modality: The language used often signals uncertainty, the source of information, or the speaker's degree of belief, which are crucial for understanding how CT claims are presented.
  • Telegram as a Rich Source: Telegram channels, with their longer, more exhaustive comments and ingroup dynamics, provide a valuable source for studying CT discourse compared to more fragmented platforms like Twitter.

Key Results

The analysis of the Complotto corpus revealed statistically significant linguistic markers of CT discourse, including:

  • Evidentiality and epistemic modality markers: Words and phrases that indicate the source or certainty of information.
  • Debunking vocabulary: Terms that suggest an alternative truth is being hidden (e.g., "mentire" - to lie, "manipolazione" - manipulation).
  • The conceptual metaphor "INSTITUTIONS ARE ABUSERS": This metaphor, often manifested as "INSTITUTIONS ARE LIARS" or "DECEIVERS," was strongly supported by the data.
  • Presentative discourse markers: Words like "ecco" (here) in Italian and "here" in English were found to be more frequent, often used to introduce evidence or explanations.

Limitations and Future Work

The authors acknowledge that their current corpus is limited to three specific CTs and two languages. They are actively working on expanding the corpus to include a third language, German, to further enhance the cross-linguistic analysis. The identified linguistic features are proposed as valuable tags for future fine-grained semantic annotation tasks, aiming to develop more sophisticated automatic CT identification systems.

Summary updated

BibTeX
@inproceedings{marini2024what,
  author = {Costanza Marini and Elisabetta Jezek},
  title = {{What to Annotate: Retrieving Lexical Markers of Conspiracy Discourse from an Italian-English Corpus of Telegram Data}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure