Towards a Framework for Evaluating Explanations in Automated Fact Verification

Neema Kotonya, Francesca Toni

This position paper from researchers at Imperial College London and Dataminr addresses a critical bottleneck in Explainable AI (XAI): we are getting better at making models "talk," but we still struggle to measure if what they are saying actually makes sense.

Focusing on Automated Fact Verification (AFV)—where AI must determine if a claim is true or false based on evidence—the authors propose a formal mathematical framework to evaluate the quality of the explanations these models produce.

TL;DR

The paper introduces a formal hierarchy for evaluating AI-generated explanations, moving from simple text summaries to complex logical arguments. By defining specific properties like "dialectical faithfulness" and "non-circularity," the authors provide a mathematical toolkit to measure whether an explanation is logically sound and aligned with a model's confidence.

What Problem Does This Paper Address?

As Large Language Models (LLMs) become more integrated into high-stakes tasks like fact-checking, their "black box" nature becomes a liability. To build trust, these models must provide rationales—short, coherent justifications for their predictions.

However, the field currently lacks a unified way to evaluate these rationales. Some models output "heat maps" of important words, others generate summaries, and some produce logical chains. Without a formal framework, it is nearly impossible to compare these different formats or determine if an explanation is actually "good" beyond just looking "plausible" to a human reader.

Methods

The authors categorize explanations into three increasingly complex structures and define formal properties for each:

  1. Free-form Explanations: Simple sequences of propositions (e.g., a summary of evidence). The key metric here is Coherence—ensuring the explanation doesn't contradict itself.
  2. Deductive Explanations: Propositions connected by a formal relation (like a "Chain of Thought"). The paper defines metrics for Relevance (is every step connected to the conclusion?), Non-circularity, and Non-redundancy (is the explanation as lean as possible?).
  3. Argumentative Explanations: The most sophisticated tier, where explanations are modeled as "bipolar argumentation frameworks." This involves arguments that can support or attack one another, mimicking a human debate.

The paper goes a step further by providing quantitative metrics (mathematical formulas) for these properties, such as $Rel_{STRONG}$ for direct logical connections and $Acc(E)$ for the "acceptability" of an argument.

Datasets

As this is a position paper focused on establishing a theoretical framework, the authors do not introduce a new training dataset or perform a large-scale empirical benchmark. Instead, they use illustrative examples derived from:

  • Snopes: Real-world fact-checking claims (e.g., a viral post about Alan Turing).
  • General Fact-Checking Scenarios: Synthetic examples used to demonstrate how the mathematical metrics would penalize circular or redundant logic.

Key Insights and Ideas

  • The "Argumentative Turn": The authors argue that the best explanations aren't just lists of facts; they are structures of conflict and support. By using symbolic AI (Argumentation Theory) to evaluate neural model outputs, they bridge the gap between "black box" deep learning and classical logic.
  • Dialectical Faithfulness: This is a standout concept. It suggests that an explanation’s strength should mirror the model’s confidence. If a model is 99% sure a claim is false, its explanation should be an unattacked, "strong" argument. If the model has low confidence, a "faithful" explanation should show internal conflict or attacking arguments.
  • Structural Evaluation vs. Human Intuition: Current evaluation often relies on "human-in-the-loop" checks, which are subjective and slow. This framework allows for an automated, structural check of an explanation's logical integrity.

Key Results

The primary result is the formalization of six specific metrics:

  • $Coh(E)$: Measures the violation of coherence in free-form text.
  • $Rel_{WEAK}$ & $Rel_{STRONG}$: Quantifies how much of a deductive chain actually contributes to the final verdict.
  • $Red(E)$: A redundancy score that penalizes "fluff" in the explanation.
  • $Acc(E)$: Measures if an argument "defends itself" against counter-arguments.
  • $Cir(E)$: Detects circular reasoning where an argument essentially tries to prove itself.

Limitations and Future Work

  • Subjectivity of Reasoning: The authors acknowledge that "human reasoning" isn't a monolith. Cultural and generational differences mean that what one person finds a "convincing" explanation, another might find confusing.
  • Computational Implementation: While the formulas are provided, implementing them at scale for real-time LLM monitoring remains a challenge for future engineering.
  • Scope: The framework is currently tailored to Fact Verification. The authors suggest that future work should test how well these metrics translate to other tasks like medical diagnosis or legal reasoning.

Summary updated

BibTeX
@inproceedings{kotonya2024towards,
  author = {Neema Kotonya and Francesca Toni},
  title = {{Towards a Framework for Evaluating Explanations in Automated Fact Verification}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure