TL;DR
Faithfulness metrics—designed to measure how accurately an explanation reflects a model's behavior—frequently disagree, ranking the same explanations in opposite orders. This paper introduces a "faithfulness functional" to decompose these metrics into three core components, arguing that faithfulness is an "incompletely specified" requirement that must be justified through the specific explanatory, institutional, and normative context of its use.
What Problem Does This Paper Address?
In the world of eXplainable AI (XAI), "faithfulness" is the gold standard. It asks: Does this explanation actually describe how the model works, or is it just a plausible-sounding story?
The problem is that the XAI community has developed dozens of different ways to measure faithfulness (e.g., Infidelity, Sufficiency, Comprehensiveness). Researchers have discovered a sobering reality: these metrics often disagree. An explanation might be "highly faithful" according to one metric but "unfaithful" according to another. This "disagreement problem" leaves developers and auditors in a bind—which metric should they trust?
This paper moves beyond just documenting the disagreement. It seeks to understand why it happens and provides a framework for choosing the right metric for a given situation.
Methods
The authors' primary contribution is a faithfulness functional. They argue that almost any faithfulness metric for local feature-attribution explanations can be broken down into three mathematical components:
- Comparison Distribution ($\nu_{x,e}$): This defines which inputs the model is tested against. For example, do we perturb the features the explanation says are important, or the ones it says are unimportant?
- Assessment Rule ($\ell_{x,e}$): This defines what a "favorable" model response looks like. If we remove an "important" feature, should the model's output score drop significantly (signed change) or just change at all (absolute change)?
- Aggregation Rule ($\mathcal{A}$): This defines how to combine many individual tests into a single score (e.g., taking the mean or calculating a Pearson correlation).
By using this functional, the authors reconstructed eight prominent faithfulness metrics from the literature (summarized in Table 1 of the paper), showing that their differences lie entirely in how they specify these three components.
Datasets
This paper is primarily theoretical and conceptual. It does not introduce new training datasets. However, it uses:
- A Synthetic Example: A small, 4-feature binary classification setup used to mathematically prove "rank non-equivalence" (that different metrics can reverse the ranking of explanations).
- Industry Case Study: An analysis of a real-world procedure for generating Adverse Action Notices (AANs) in credit lending, proposed by researchers at Wells Fargo.
Key Insights and Ideas
- Faithfulness is Incompletely Specified: The authors argue that the general requirement of "faithfulness" is too vague to be a single mathematical formula. Instead, it's a shared concept that requires specific choices to be made before it can be measured.
- Three Sources of Disagreement: The paper distinguishes between three reasons why metrics might differ, as illustrated in Figure 1:
- (a) Theoretical Contestation: Disagreeing on what faithfulness even means.
- (b) Incomplete Specification: Agreeing on the core concept but making different choices about how to apply it in a specific setting. The authors argue this is the primary driver of disagreement in XAI.
- (c) Indicator Variation: Agreeing on the specific measurement goal but having different (perhaps noisy) ways of calculating it.
- Contextual Justification: Since faithfulness is incomplete, choosing a metric isn't a neutral technical decision. It requires justification based on:
- Explanatory considerations: The form of the explanation (e.g., a top-k list of reasons).
- Institutional considerations: Legal requirements (like the Equal Credit Opportunity Act for AANs).
- Normative considerations: The ultimate purpose (e.g., helping a consumer improve their credit score).
Key Results
- Proof of Rank Non-Equivalence: The authors formally prove that common metrics like Infidelity, Sufficiency, and Comprehensiveness are pairwise rank-non-equivalent.
- Metric Reconstruction: The paper successfully maps eight major faithfulness metrics into the new functional framework, providing a "Rosetta Stone" for comparing XAI evaluations.
- AAN Case Study: By reconstructing the Wells Fargo B-Shap procedure, the authors show that while some choices (like the reference point for comparison) are well-justified by the legal and educational goals of credit denial notices, other choices (like how to handle feature interactions) remain open and require further normative argument.
Limitations and Future Work
The authors acknowledge that their functional currently focuses on local feature-attribution and rationale-based explanations. Extending this framework to other explanation forms, such as counterfactuals or global explanations, is a clear path for future research.
Furthermore, while the paper provides a framework for how to justify choices, it doesn't provide a "one-size-fits-all" answer for which metric is best. The burden of justification remains with the practitioners and auditors to look closely at their specific application context.
Summary updated