Axiomatic Foundations of Explainability

Leila Amgoud, Jonathan Ben-NaimIJCAI 2022 · DOI

TL;DR

This paper tackles the under-studied formal properties of explainability methods in machine learning. It introduces a set of axioms to characterize different types of explanations, distinguishing between "sufficient" (abductive) and "necessary" (counterfactual) reasons. The work proposes new families of explainers based on these axioms and analyzes their properties, revealing that explainers using incomplete data (like Anchors and LIME) can violate crucial axioms like relevance, leading to potentially incorrect explanations.

What Problem Does This Paper Address?

As machine learning models become more prevalent in critical applications like healthcare and finance, understanding why they make certain decisions is paramount for trust and adoption. While many "explainability" methods exist, their underlying principles and formal guarantees are often unclear, making it difficult to compare them or ensure their reliability. This paper addresses this gap by proposing a formal, axiomatic framework to define and analyze explainability functions, particularly those that provide global explanations for classes rather than just individual instances. The authors aim to bring theoretical rigor to the field, clarifying the properties that constitute good explanations and identifying the trade-offs inherent in different approaches.

Methods

The core of this paper is the introduction of a set of formal axioms designed to capture desirable properties of "class explainers." These explainers aim to provide reasons for a classifier assigning a particular class, independent of specific instances. The authors define key concepts like "literals" (attribute-value assignments), "instances" (complete attribute assignments), and "classifiers" (functions mapping instances to classes).

They then propose several axioms, including:

  • Success: An explainer always provides an explanation.
  • Explainability: Explanations are non-empty.
  • Irreducibility: Reasons do not contain unnecessary information.
  • Feasibility: Reasons are consistent and can occur in reality.
  • Representativity: Explanations can justify any instance's prediction.
  • Relevance: Explanations only include literals that impact the prediction.
  • Coreness: Explanations only include literals whose absence would change the prediction.
  • Exhaustivity: Explanations include all relevant literals.
  • Completeness: Explanations include all core literals.
  • Coherence: Explanations for different classes are mutually exclusive.

These axioms are used to categorize explainers into two main families: sufficiency explainers (based on abductive reasoning, providing reasons that guarantee a class) and necessity explainers (based on counterfactual reasoning, providing reasons that are required for a class). The paper then defines specific explainers within these families (e.g., aAbd, mAbd, aCtf, mCtf, xCtf) and rigorously analyzes which axioms they satisfy. A significant portion of the work is dedicated to analyzing explainers that operate on incomplete information (i.e., using only a subset of available instances), abstracting concepts like Anchors and LIME.

Datasets

Not discussed in the paper. The paper focuses on theoretical foundations and axiomatic properties rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • Sufficient vs. Necessary Reasons: The paper clearly delineates two fundamental types of explanations: sufficient reasons (abductive, "if these conditions hold, then class X is predicted") and necessary reasons (counterfactual, "if these conditions hadn't held, class X would not have been predicted"). This distinction is crucial for understanding the different guarantees an explainer can offer.
  • Axioms as a Formal Language: The proposed axioms provide a formal language to discuss and compare explainability methods. They move beyond ad-hoc metrics to establish principled criteria for evaluating explanation quality.
  • Incompatibility of Coreness and Exhaustivity: The paper highlights that explainers aiming to be exhaustive (including all relevant literals) and those focusing on core literals (essential for the prediction) can be fundamentally incompatible. This reveals a core trade-off in explanation design.
  • The Peril of Incomplete Information: A critical finding is that explainers relying on subsets of data (like Anchors and LIME) can violate the "Relevance" axiom. This means they might provide reasons that are not truly relevant to the classifier's decision-making process, potentially leading to misleading or incorrect explanations. This is a significant theoretical critique of popular methods.
  • Characterizing Explainers: The paper demonstrates how specific combinations of axioms can uniquely characterize certain types of explainers, providing a formal basis for understanding their behavior. For instance, mAbd (minimal abductive explainer) is characterized by Feasibility, Irreducibility, Representativity, and Coherence.

Key Results

  • The paper formally defines two families of explainers: sufficiency (abductive) and necessity (counterfactual).
  • It establishes that sufficiency explainers are characterized by Feasibility, Representativity, and Coherence, while necessity explainers are characterized by Success, Irreducibility, Feasibility, Relevance, Coreness, and Completeness (for aCtf and xCtf).
  • The minimal abductive explainer (mAbd) is uniquely characterized by Irreducibility, Feasibility, Representativity, and Coherence.
  • Explainers based on incomplete information (abstracting Anchors/LIME) are shown to violate Relevance, Coreness, Completeness, Exhaustivity, Representativity, and Coherence, indicating a fundamental limitation.

Limitations and Future Work

  • Incomplete Information Explainers: The paper acknowledges that explainers using incomplete information (like Anchors and LIME) violate several key axioms, particularly Relevance, leading to potentially erroneous explanations. This is a significant limitation of these popular methods.
  • Future Work: The authors plan to extend their axiomatic framework to deal with other types of explanations, such as conversational ones. They also suggest exploring the implications of these axioms for practical explainability systems.

Summary updated

BibTeX
@inproceedings{amgoud2022axiomatic,
  author = {Leila Amgoud and Jonathan Ben-Naim},
  title = {{Axiomatic Foundations of Explainability}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/90},
}
Figure