Abstract
The proliferation of climate change misinformation necessitates scalable, automated fact-checking systems that provide not just veracity labels, but also human-readable explanations. This paper presents a Retrieval-Augmented Generation (RAG) framework that treats climate claim verification as a query-focused multi-document summarization task. By integrating a Binary Passage Retriever (BPR) with a modified PRIMERA model featuring specialized global attention on claim tokens, the system retrieves evidence from peer-reviewed scientific literature to generate natural language explanations. Experimental results on the Climate-Fever dataset demonstrate that this approach significantly improves veracity classification accuracy and explanation quality over strong baselines.
Introduction
The rapid spread of climate change misinformation manipulates public perception and hinders critical policy action. While expert-led fact-checking platforms provide high-quality rebuttals, the sheer volume of online claims makes manual verification impossible to scale. Existing automated systems often provide binary "True/False" labels, which lack the nuance required to effectively counter misinformation. To be persuasive, a system must provide a human-readable explanation grounded in credible scientific evidence.
Generating such explanations presents three primary challenges: identifying relevant literature from vast databases, synthesizing information across multiple technical documents, and ensuring the explanation directly addresses the specific claim (the "query").
Methods
The proposed system utilizes a Retrieval-Augmented Generation (RAG) framework consisting of two main components: a retriever and a generator.
The Retriever
The system employs the Binary Passage Retriever (BPR), an efficient dense passage retrieval method. BPR uses hashing to rapidly identify the top-k (specifically, 5) most relevant passages from a curated knowledge base. This knowledge base includes scientific abstracts from PubMed and comprehensive reports from the Intergovernmental Panel on Climate Change (IPCC).
The Generator
For the generation stage, the authors adapt PRIMERA, a state-of-the-art encoder-decoder model pre-trained for multi-document summarization. To make the model "query-focused," the standard attention mechanism was modified. While the base PRIMERA model applies "global attention" only to document separators, this work adds global attention to the claim words themselves. This architectural choice forces the model to continuously refer back to the original claim while processing retrieved evidence, ensuring the output is a direct explanation of the claim rather than a generic summary.
Results
The model, designated P-full, was evaluated against a strong Fusion-in-Decoder (FiD) baseline using the Climate-Fever dataset, which contains 1,535 real-world climate claims.
As shown in the results, P-full achieved a veracity classification accuracy of 60%, outperforming the baseline's 55%. In terms of explanation quality, P-full consistently surpassed the baseline across lexical (ROUGE) and semantic (BERTScore) metrics, achieving a ROUGE-1 score of 0.33 compared to 0.25 for FiD.
Ablation studies confirmed the importance of the specialized global attention. Removing attention on claim tokens (the -claim setting) resulted in a performance drop across all metrics, validating that explicit query-focusing reduces "claim-copying" errors and improves consistency.
Discussion and Limitations
Analysis revealed that the retriever remains a significant bottleneck; in 75% of cases where the model provided an "insufficient explanation," the retriever had failed to find relevant passages. Furthermore, while the model excels at synthesizing information, it still struggles with complex claims requiring multi-step logical reasoning. Future work will focus on improving retriever precision and enhancing the model's reasoning capabilities to further reduce hallucinations and improve the synthesis of disparate data points.
Summary updated