Towards a Framework for Evaluating Explanations in Automated Fact Verification

Neema Kotonya, Francesca Toni

Abstract

As Large Language Models (LLMs) become increasingly opaque, the need for "rationalizing explanations"—justifications for specific predictions—has never been greater. This paper proposes a formal framework to categorize these explanations into three levels of structural complexity: free-form, deductive, and argumentative. We introduce a suite of mathematical metrics to evaluate their coherence, relevance, and logical integrity, providing a unified approach to assessing the quality of justifications in automated fact verification.

1. Introduction

In the high-stakes domain of automated fact verification, it is insufficient for a model to simply label a claim as "True" or "False." Users require an understanding of the underlying reasoning to build trust and ensure the system is not "right for the wrong reasons." However, the field currently lacks a unified methodology to define and evaluate these explanations. Existing models produce outputs ranging from simple summaries to complex logical chains or debates of conflicting evidence. Without a formal framework, researchers cannot systematically compare these formats or measure critical properties such as redundancy, circularity, or relevance.

2. A Structural Taxonomy of Explanations

The proposed framework provides a formal taxonomy and a set of mathematical properties for three distinct classes of explanations, categorized by their structural complexity.

Figure 1: The three levels of explanation complexity defined in the framework: (a) Free-form, (b) Deductive, and (c) Argumentative.
  • Free-form Explanations: These consist of a sequence of natural language propositions. The primary evaluation metric is Coherence, ensuring the explanation does not contain internal contradictions.
  • Deductive Explanations: Propositions are connected by binary relations, forming a logical chain similar to "Chain-of-Thought" prompting. Metrics include Weak/Strong Relevance, Non-circularity, and Non-redundancy.
  • Argumentative Explanations: The most complex type, modeled as a "bipolar argumentation framework" where arguments can either support or attack one another. This allows for contrastive explanations that reflect conflicting evidence.

3. Evaluation Metrics and Key Concepts

The paper formalizes these structures into quantitative metrics (e.g., $Rel_{WEAK}$, $Red(E)$, $Acc(E)$). A standout concept is Dialectical Faithfulness, which argues that an explanation's strength should match the model's confidence. An explanation that appears entirely certain for a low-confidence prediction is considered "unfaithful."

Furthermore, by treating deductive and argumentative explanations as graphs, the framework leverages graph theory to detect circular reasoning—a common failure mode in generative AI that is difficult to identify using standard NLP metrics like ROUGE or BLEU.

4. Illustrative Application

To demonstrate the framework's utility, the authors analyze hand-constructed examples and real-world fact-check justifications. Figure 2 illustrates a deductive explanation for a claim regarding the lifespan of a daffodil plant.

Figure 2: A formal representation of a deductive explanation for a verified claim about daffodil plants.

In this instance, the claim ($p_3$) is verified through a chain of propositions ($p_1, p_2$) and their relations ($\mathcal{R}$). The framework allows for the formal verification of the logical path taken to reach the verdict, ensuring that the explanation is both relevant and non-redundant.

5. Discussion and Future Work

While the framework provides a rigorous logical foundation, the authors acknowledge that human subjectivity remains a factor; a logically perfect chain may not always be the most "understandable" to every user. Future work will involve applying these metrics to large-scale benchmarks of state-of-the-art LLMs to assess their performance on properties like Dialectical Faithfulness and Non-redundancy.

The implementation of these metrics is available in the authors' GitHub repository.

Summary updated

BibTeX
@inproceedings{kotonya2024towards,
  author = {Neema Kotonya and Francesca Toni},
  title = {{Towards a Framework for Evaluating Explanations in Automated Fact Verification}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure