Generative Debunking of Climate Misinformation

Francisco Zanartu, Yulia Otmakhova, John Cook, Lea FrermannACL 2024

TL;DR

This paper introduces "Generative Debunking," a novel approach using large language models (LLMs) to automatically generate "truth sandwich" debunkings of climate misinformation. By combining LLMs with techniques for classifying contrarian claims and detecting fallacies, the system aims to combat the spread of false information at scale, a critical challenge in climate communication.

What Problem Does This Paper Address?

Climate misinformation poses significant threats, including reduced climate literacy, increased polarization, and the undermining of factual communication efforts. While psychological research has identified effective debunking strategies, such as the "fact-myth-fallacy-fact" structure, implementing these interventions at scale is challenging. The rapid and widespread dissemination of false information on social media outpaces manual fact-checking. This paper addresses the need for an automated solution to detect and correct climate misinformation efficiently and effectively, aiming to achieve the "holy grail of fact-checking."

Methods

The core of this work is the development of a generative debunking framework that leverages large language models (LLMs). The authors explore three prompting strategies of increasing complexity, tested with different LLMs:

  • Single Generic Prompt (GPT-4): A comprehensive, end-to-end instruction for the LLM, requiring it to perform fallacy detection, formatting, and fact retrieval based on its internal knowledge.
  • Single Prompt with Myth-Specific External Information (Palm2): This approach augments the generic prompt by dynamically incorporating external information and examples relevant to the input myth. It utilizes the FLICC model for fallacy prediction and retrieves specialized examples from a curated dataset.
  • Structured Prompt with Context (Mixtral): This most complex approach breaks down the debunking generation into four distinct prompts, each focusing on a specific component: Fact (using a ReAct agent with internet search), Myth (paraphrasing), Fallacy (detection and explanation), and a final Fact (reinforcement). This modular approach allows for more targeted and potentially higher-quality output.

The system integrates existing tools for fallacy detection (FLICC model) and contrarian claim classification (CARDS classifier).

Datasets

The study utilizes several datasets:

  • FLICC test set: A dataset of 256 samples across 12 logical fallacies, used for model evaluation.
  • CLIMATE-FEVER dataset: A collection of 1,535 real-world climate change-related claims, with manually annotated evidence sentences. The study specifically uses the false claims from this dataset.
  • CARDS-examples dataset: A newly developed dataset of 62 instances of misinformation, each with gold-standard truth-sandwich debunkings and fallacy labels, created by a climate debunking expert. This dataset is used for in-context learning.

Key Insights and Ideas

  • The "Truth Sandwich" is Key: The fact-myth-fallacy-fact structure is psychologically grounded and effective for debunking, leveraging primacy and recency effects for better recall.
  • LLMs Can Generate Debunkings: The study demonstrates that LLMs, particularly powerful ones like GPT-4 and Mixtral, can be prompted to generate structured and coherent debunkings.
  • Prompting Strategy Matters: The complexity of the prompting strategy significantly impacts the quality of the generated debunkings, with structured and context-aware prompts yielding better results, especially for open-source LLMs.
  • Fact Generation is Hard: Generating accurate, relevant, and "sticky" facts that effectively counter misinformation remains a significant challenge for LLMs. Non-expert annotators can be easily swayed by seemingly plausible but irrelevant facts.
  • Human Evaluation is Difficult: Evaluating the quality of generated debunkings, especially the factual components, is challenging for non-experts, leading to low inter-annotator agreement and a tendency to overestimate their own judgment. Domain expertise is crucial for accurate evaluation.

Key Results

The study found that GPT-4 with a simple prompt and Mixtral with a structured, context-sensitive prompt performed promisingly. Specifically:

  • GPT-4 and Mixtral Show Promise: These models, when combined with structured prompts, demonstrated the most effective debunking generation.
  • Fallacy Detection is Stronger: Human annotators showed better agreement on the fallacy detection and explanation components of the debunkings compared to the factual components.
  • Palm2 Struggles: The open-source Palm2 model, when using a single context-sensitive prompt, produced less coherent and less compliant debunkings, often generating generic information.
  • Non-Expert vs. Expert Agreement: Non-expert annotators often rated outputs more optimistically than the expert, particularly for Mixtral's outputs, which tended to include more specific (though not always relevant) facts.
  • All models adhered to the fact-myth-fallacy-fact structure 100% of the time.

Limitations and Future Work

The authors acknowledge several limitations:

  • Fact Quality: The generated facts are not always reliably high-quality, lacking specificity and direct relevance to the myth.
  • Evaluation Challenges: The evaluation process, particularly for facts, is difficult for non-experts, highlighting the need for expert involvement or supervision.
  • Prompt Design: The study did not systematically disentangle the effects of individual prompt design decisions or exhaustively combine all prompts with all LLMs.
  • Limited Scope: The evaluation was based on a small sample size and a limited annotator pool.
  • New vs. Known Myths: The models' performance on novel or less common myths, which may lack training data, was not systematically studied.

Future work should focus on:

  • Improving the specificity and relevance of generated facts.
  • Developing more robust evaluation methodologies, potentially involving expert oversight.
  • Systematically analyzing the impact of prompt design choices.
  • Exploring combinations of LLMs and prompting strategies.
  • Investigating the models' ability to distinguish input myths from facts (though this was outside the scope of the current study).
  • Integrating online misinformation detection capabilities.

Summary updated

BibTeX
@inproceedings{zanartu2024generative,
  author = {Francisco Zanartu and Yulia Otmakhova and John Cook and Lea Frermann},
  title = {{Generative Debunking of Climate Misinformation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure