You are right. I am ALARMED - But by Climate Change Counter Movement

Shraey Bhatia, Jey Han Lau, Timothy BaldwinarXiv 2020 · OpenReview

Abstract: The world is facing the challenge of climate crisis. Despite the consensus in the scientific community about anthropogenic global warming, the web is flooded with articles spreading climate misinformation. These articles are carefully constructed by climate change counter movement (CCCM) organizations to influence the narrative around climate change. We revisit the literature on climate misinformation in social sciences and repackage it to introduce it to the community of NLP. Despite considerable work in the detection of fake news, there is no misinformation dataset available that is specific to the domain of climate change. We try to bridge this gap by scraping and releasing articles with known climate change misinformation.

Introduction

The global scientific community maintains a near-unanimous consensus on anthropogenic global warming. However, the digital information ecosystem is frequently saturated with climate misinformation. These narratives are often systematically produced by Climate Change Counter Movement (CCCM) organizations to delay policy action and foster public skepticism. This paper addresses a significant gap in Natural Language Processing (NLP) research: the lack of domain-specific datasets for climate misinformation. While general "fake news" detection is well-studied, climate-specific disinformation possesses unique topical markers and stylistic signatures that general models often overlook.

Methodology and Argument Framing

The researchers adopted a source-centric approach to data collection, identifying CCCM organizations through social science literature and the DeSmogBlog database. A key contribution of this work is the categorization of misinformation into two distinct "frames":

  1. Science Frame: Arguments that directly challenge scientific facts (e.g., questioning if CO2 is harmful).
  2. Policy Frame: Arguments targeting the economic or social feasibility of climate action (e.g., attacking the cost of renewables).
Common CCCM arguments categorized by Science and Policy frames.

To ensure model robustness, the authors implemented an organization-level split. Models are trained on data from one set of organizations and evaluated on entirely different ones, preventing the model from simply learning the specific web layout or idiosyncratic style of a single source.

The Climate Misinformation Dataset

The released dataset comprises articles from 15 CCCM organizations across English-speaking nations.

Training and Evaluation

The training set includes 1,168 articles from 12 organizations, exhibiting significant variance in length and style. The test set (510 documents) is designed to be a rigorous benchmark for domain knowledge and "common sense." It includes articles from unseen CCCM organizations, reputable news outlets (The Guardian, BBC), debunking sites (Sceptical Science), and satirical news (The Beetota Advocate).

Composition of the test set by document source.

Discussion: The Satire Challenge and Historical Parallels

The inclusion of satire serves as a "hard case" for NLP models. Satirical content often mimics the sensationalist style of disinformation; models relying purely on stylistic features (like superlatives or melodrama) are prone to misclassifying satire as misinformation.

Furthermore, the authors note that CCCM strategies often mirror the "Tobacco Industry Playbook." This involves attacking individual researchers, questioning established methodologies, and demanding unattainable "scientific standards" to create a false sense of scientific uncertainty.

Conclusion

This work introduces the first dedicated NLP dataset for climate change misinformation, characterizing it not as random noise but as a "carefully crafted strategy." By providing a benchmark that includes scientific consensus, policy debate, and satire, the authors offer a vital tool for developing models capable of navigating the complex narrative landscape of the climate crisis. Future work should aim to expand this dataset beyond English-speaking regions and provide extensive benchmarking with state-of-the-art transformer models.

Summary updated

Abstract

The world is facing the challenge of climate crisis. Despite the consensus in scientific community about anthropogenic global warming, the web is flooded with articles spreading climate misinformation. These articles are carefully constructed by climate change counter movement (cccm) organizations to influence the narrative around climate change. We revisit the literature on climate misinformation in social sciences and repackage it to introduce in the community of NLP. Despite considerable work in detection of fake news, there is no misinformation dataset available that is specific to the domain.of climate change. We try to bridge this gap by scraping and releasing articles with known climate change misinformation.
BibTeX
@misc{bhatia2020you,
  author = {Shraey Bhatia and Jey Han Lau and Timothy Baldwin},
  title = {{You are right. I am ALARMED - But by Climate Change Counter Movement}},
  year = {2020},
  eprint = {2004.14907},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.14907},
}

Loading related content...

Figure