Abstract
Fact Verification requires fine-grained natural language inference capability that finds subtle clues to identify the syntactical and semantically correct but not well-supported claims. This paper presents Kernel Graph Attention Network (KGAT), which conducts more fine-grained fact verification with kernel-based attentions. Given a claim and a set of potential evidence sentences that form an evidence graph, KGAT introduces node kernels, which better measure the importance of the evidence node, and edge kernels, which conduct fine-grained evidence propagation in the graph, into Graph Attention Networks for more accurate fact verification. KGAT achieves a 70.38% FEVER score and significantly outperforms existing fact verification models on FEVER, a large-scale benchmark for fact verification. Our analyses illustrate that, compared to dot-product attentions, the kernel-based attention concentrates more on relevant evidence sentences and meaningful clues in the evidence graph, which is the main source of KGAT's effectiveness.
Introduction
In an era of sophisticated misinformation, the ability to automatically verify claims against a trusted corpus is more critical than ever. Fact-checking is rarely about finding a single "smoking gun" sentence; it often requires connecting subtle clues across multiple pieces of evidence. For example, verifying a claim about a musician's specific role in a band might require synthesizing information from separate sentences regarding their membership and their instrumentation.
Existing systems face two major hurdles: noise from automated evidence retrieval and the need for fine-grained reasoning. Standard "dot-product" attention mechanisms often spread focus too broadly across tokens, failing to capture the specific, fine-grained clues needed for a definitive verdict. This paper introduces the Kernel Graph Attention Network (KGAT) to address these limitations.
Methodology
KGAT treats fact verification as a graph reasoning problem, utilizing kernel-based attention to aggregate information more precisely.
Graph Construction
The model builds a graph where each node represents a pair consisting of the claim and one retrieved evidence sentence. Every node is connected to every other node, allowing information to flow between different pieces of evidence.
Edge Kernels (Propagation)
To facilitate communication between nodes, KGAT uses Edge Kernels. Instead of a simple similarity score, it builds a translation matrix of word-to-word interactions and applies Gaussian kernels. This allows the model to capture multi-level interactions (e.g., exact matches vs. soft semantic matches) and propagate only the most relevant clues across the graph.
Node Kernels (Selection)
Not all evidence is equally relevant. KGAT uses Node Kernels to calculate how strongly each evidence sentence supports or refutes the claim. This acts as a sophisticated filtering system to weight the importance of each evidence node before the final verdict.
Experimental Setup
The researchers evaluated KGAT on FEVER (Fact Extraction and VERification), a large-scale benchmark containing 185,455 annotated claims derived from 5.4 million Wikipedia documents. Claims are labeled as Supported, Refuted, or Not Enough Info.
Results and Analysis
KGAT achieved a state-of-the-art 70.38% FEVER score, significantly outperforming previous leaders like GEAR and BERT-based baselines.
Key insights from the analysis include:
- Concentrated Attention: Kernel-based attention produces much lower entropy than standard attention, meaning the model's focus is highly concentrated on meaningful keywords rather than irrelevant stop words.
- Multi-Hop Reasoning: KGAT excels on claims requiring multiple evidence pieces. By using edge kernels to propagate specific clues between sentences, it fills reasoning gaps that single-sentence models miss.
- De-noising: The node kernels make the model robust to noisy, irrelevant sentences pulled in during initial search. KGAT’s top-ranked sentence covered nearly 80% of the ground truth evidence, a significant improvement over standard attention models.
Conclusion
KGAT demonstrates that fine-grained reasoning via kernel-based attention is highly effective for fact verification. While the model currently focuses on the reasoning stage, future work could integrate node kernels directly into the retrieval stage to improve initial evidence quality. Additionally, improving the interpretability of these kernel-based decisions remains a key area for future research.
Summary updated