Abstract
Existing approaches construct appropriate interaction models to explore semantic conflicts between claims and relevant articles, which provides practical solutions for interpretable claim verification. However, these conflicts are not necessarily all about questioning the false part of claims, which makes considerable semantic conflicts difficult to be used as evidence to explain the results of claim verification. In this paper, we propose evidence inference networks (EVIN), which focus on the conflicts questioning the core semantics of claims and serve as evidence for interpretable claim verification. Specifically, EVIN first captures the core semantic segments of claims and the users' principal opinions in relevant articles. Then, it finely-grained identifies the semantic conflicts contained in each relevant article from these opinions. Finally, it constructs coherence modeling to match the conflicts that queries the core semantic fragments of claims as explainable evidence. Experiments on two widely used datasets demonstrate that EVIN not only achieves satisfactory performance but also provides explainable evidence for end-users.
Introduction
In the era of social media, the rapid spread of false claims necessitates automated verification systems that are not only accurate but also interpretable. While current models attempt to provide explanations by identifying semantic conflicts between claims and external articles, they often fail to distinguish between useful evidence and irrelevant noise. For instance, in a claim regarding disinfecting face masks in a microwave, a user comment regarding the destruction of the mask's internal structure is high-quality evidence, whereas a comment questioning the cleanliness of the microwave for future food use is irrelevant to the claim's veracity.
EVIN addresses this by identifying the core semantics of a claim and matching them specifically with relevant refutations, effectively "purifying" the evidence pool.
Methodology
EVIN utilizes a four-level hierarchical architecture designed to distill meaningful evidence from noisy social media environments:
- Input Encoding Layer: Employs BiLSTMs and pre-trained BERT embeddings to represent claims and relevant articles.
- Co-interactive Shared Layer: Captures the core segments of the claim and the holistic crowd opinion using a Gate Affine Absorption module. This gating mechanism adaptively filters features to focus on segments of the claim that users are actively debating.
- Fine-grained Conflict Discovery Layer: Uses cross-attention to pinpoint localized semantic conflicts within individual articles.
- Evidence-aware Coherence Layer: Measures the coherence between core claim segments and discovered conflicts to prioritize high-quality evidence.
Experimental Evaluation
The model was evaluated on two benchmark datasets: Snopes (4,341 claims) and PolitiFact (3,568 claims). EVIN consistently outperformed state-of-the-art baselines like DeClarE and AIFN. On Snopes, EVIN achieved a 0.842 micF1 score, while on PolitiFact, it reached 0.561 micF1. Ablation studies confirmed that the "coherence" and "gate" modules are critical, with their removal resulting in performance drops of up to 4.1%.
Key Insights and Discussion
- Sparsity as a Signal: The authors observed that for false claims, attention weights are sparser and more focused on specific "questioning voices." This suggests that false claims are typically debunked by specific, high-impact pieces of information.
- Interpretability: Qualitative analysis through word clouds demonstrated that EVIN successfully captures refuting keywords such as "rumor," "suspected," and "uncovered" when analyzing false claims, providing human-readable justifications for its predictions.
Conclusion and Future Work
While EVIN provides a robust framework for interpretable verification, it currently treats evidence as a static pool. Future research could incorporate temporal dynamics to track how evidence evolves over time. Additionally, addressing the "cold start" problem—where new claims have little social engagement—through few-shot learning or external knowledge bases like Wikidata remains a promising direction.
Summary updated