Consistent Multi-Granular Rationale Extraction for Explainable Multi-hop Fact Verification

Jiasheng Si, Yingjie Zhu, Deyu ZhouarXiv 2023 · arXiv

TL;DR

Deep learning models for multi-hop fact verification often operate as black boxes, making it hard to see how they connect multiple pieces of evidence. This paper introduces CURE (Consistent mUlti-granular Rationale Extraction), a framework that simultaneously extracts sentence-level and token-level explanations. By enforcing consistency between these two granularities, CURE delivers highly precise, faithful, and noise-free explanations without compromising the model's predictive accuracy.

What Problem Does This Paper Address?

To verify complex claims, fact-checking systems must perform "multi-hop" reasoning—linking multiple disparate pieces of evidence together. While modern neural networks excel at this, explaining why a model reached a specific conclusion remains a challenge.

A popular explainability technique is erasure search: finding the smallest subset of the input text (the "rationale") that can be kept while still preserving the model's original prediction. However, existing erasure methods only look at a single granularity:

  • Sentence-level rationales are too coarse, often retaining irrelevant words and noise evidence alongside the facts.
  • Token-level rationales lack structural context, frequently selecting scattered words from completely irrelevant sentences due to spurious correlations.

This single-granularity focus leads to redundant or contradictory explanations. This paper addresses this gap by asking: How can we extract token-level and sentence-level rationales simultaneously so that they complement and validate each other?

An example of single-granular rationale extraction issues from the HoVer dataset.

Methods

The authors propose CURE, a post-hoc explainability framework. It takes a frozen, pretrained multi-hop fact verification model (specifically, a Transformer-XH architecture paired with a Capsule Network) and trains two lightweight, parameterized explainer networks on top of it.

Figure 2: The overall architecture of our CURE. (a): the multi-granular rationale extraction, (b): the veracity prediction model

The core innovation lies in how these explainers are trained using differentiable masking (via the Hard Concrete reparameterization trick) guided by three diagnostic properties:

  1. Fidelity: Ensures the extracted rationales are sufficient on their own to preserve the original model's prediction. This is enforced by minimizing the Euclidean distance between the prediction logits of the original text and the masked text.
  2. Consistency: Uses symmetric Jensen-Shannon Divergence to align the token-level and sentence-level explainers. This forces the token explainer to select words primarily from the sentences that the sentence explainer deemed important, preventing it from picking random tokens from noise sentences.
  3. Salience: Uses supervised signals as prior knowledge to guide the search space. It leverages human-annotated sentence labels and computes token-level pseudo-labels using Layered Integrated Gradients.

Datasets

The framework was evaluated on three challenging multi-hop fact verification benchmarks:

  • HoVer: A dataset specifically designed for many-hop fact extraction and claim verification, where each claim is paired with 5 retrieved text snippets.
  • LIAR-PLUS: A benchmark containing political claims, each associated with 10 pieces of evidence.
  • PolitiHop: A dataset focusing on political fact-checking with 5 pieces of evidence per claim.

All three datasets include human annotations for sentence-level rationales mixed with distracting noise evidence.

Key Insights and Ideas

  • Granular Synergy: Training token and sentence explainers together acts as a powerful mutual regularizer. Sentence-level context prevents the token explainer from hallucinating importance, while fine-grained token importance helps sharpen sentence selection.
  • Pruning Spurious Correlations: Without consistency constraints, token-level explainers suffer from "under-aggressive pruning"—they fail to ignore tokens in completely irrelevant sentences. Intersecting the two granularities effectively shields the explanation from data artifacts and noise.
  • Anchoring on Titles: Qualitatively, CURE demonstrates a strong tendency to retain document titles. The model recognizes that titles serve as critical semantic "bridges" necessary for linking multi-hop evidence across different documents.

Key Results

CURE significantly outperformed classical post-hoc explainers (like LIME, SHAP, and DiffMask) across several metrics:

  • Superior Consistency: CURE achieved a consistency score of 0.9522 on the HoVer dataset, whereas baseline methods often hovered near zero or even showed negative consistency (meaning they picked tokens from sentences they claimed were irrelevant).
  • High Faithfulness: CURE achieved the lowest Fidelity loss across all three datasets (e.g., 0.2287 on HoVer), proving that its concise snippets truly reflect the internal reasoning of the verification model.
  • Human Approval: In a blind evaluation by human annotators rating explanations on a 1–5 scale, CURE scored the highest on average across Correctness, Faithfulness, and Non-redundancy.
Figure 3: Human evaluation results in a likert scale of 1 to 5, where 1 means strongly disagree and 5 means strongly agree. Average denotes the average score of three criteria. The inner-rater agreement measured by Krippendorff’s α is 0.88.

Limitations and Future Work

Despite its success, CURE has a few limitations:

  • Supervision Dependency: The "Salience" property relies on human-annotated sentence-level rationale labels during training. This makes it difficult to deploy CURE in fully unsupervised scenarios where rationale labels are missing.
  • Sensitivity to Sparsity: The framework uses an $L_0$ regularization loss to keep explanations concise. However, overemphasizing sparsity can occasionally strip away context that is minorly helpful to the verification model, making the framework sensitive to hyperparameter tuning.

For future work, the authors plan to extend this multi-granular approach to generate counterfactual explanations (i.e., demonstrating how changing a specific fact would alter the model's verdict).

Summary updated

Abstract

The success of deep learning models on multi-hop fact verification has prompted researchers to understand the behavior behind their veracity. One possible way is erasure search: obtaining the rationale by entirely removing a subset of input without compromising the veracity prediction. Although extensively explored, existing approaches fall within the scope of the single-granular (tokens or sentences) explanation, which inevitably leads to explanation redundancy and inconsistency. To address such issues, this paper explores the viability of multi-granular rationale extraction with consistency and faithfulness for explainable multi-hop fact verification. In particular, given a pretrained veracity prediction model, both the token-level explainer and sentence-level explainer are trained simultaneously to obtain multi-granular rationales via differentiable masking. Meanwhile, three diagnostic properties (fidelity, consistency, salience) are introduced and applied to the training process, to ensure that the extracted rationales satisfy faithfulness and consistency. Experimental results on three multi-hop fact verification datasets show that the proposed approach outperforms some state-of-the-art baselines.
BibTeX
@misc{si2023consistent,
  author = {Jiasheng Si and Yingjie Zhu and Deyu Zhou},
  title = {{Consistent Multi-Granular Rationale Extraction for Explainable Multi-hop Fact Verification}},
  year = {2023},
  eprint = {2305.09400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.09400},
}
Figure