TL;DR
This paper tackles the challenge of generating reliable explanations for complex AI models, particularly when explanations are derived from limited data samples. They introduce a novel framework using argumentation theory to create explanation functions that offer a trade-off between guaranteeing an explanation for every instance and ensuring the explanation's correctness.
What Problem Does This Paper Address?
Modern AI models, especially deep neural networks, are often "black boxes," making it difficult to understand why they make specific predictions. Explainable AI (XAI) aims to address this opacity. One popular type of explanation is "abductive explanations," which identify a minimal set of features and their values sufficient for a prediction. However, generating these explanations exhaustively across all possible feature combinations is computationally infeasible for complex models. Existing methods that use samples of data to generate explanations (like LIME and Anchors) have been shown to produce incorrect explanations. This paper addresses the fundamental incompatibility between generating explanations from samples and ensuring both their existence for every instance and their correctness.
Methods
The paper proposes a parameterized family of argumentation-based explanation functions. The core idea is to represent the decision-making process of a classifier as an argumentation system. This involves:
- Argument Generation: Creating arguments that support a specific class prediction, where each argument consists of a set of literals (feature-value pairs) and the predicted class. These arguments are derived from a sample of instances.
- Attack Identification: Defining "attacks" between arguments when their supports are consistent but their conclusions (predicted classes) differ. This captures conflicts between explanations.
- Stable Semantics: Using Dung's stable semantics to find sets of arguments that can be jointly accepted, representing compatible explanations.
- Selection Functions and Inference Rules: Introducing parameters (selection functions and inference rules) to define how to choose among stable extensions and how to extract accepted arguments. These parameters allow for different strategies to balance competing desirable properties.
The paper then formally analyzes these functions, showing how different combinations of selection functions and inference rules lead to functions that either prioritize the existence of explanations (success) or their correctness (coherence).
Datasets
The paper uses several datasets for experimental analysis:
- Adult
- Diabetes
- Lending
- Titanic
- Recidivism
These datasets are standard benchmarks often used in XAI research.
Key Insights and Ideas
- Incompatibility of Success and Coherence: The paper proves a fundamental impossibility result: no explanation function derived from a sample can simultaneously guarantee an explanation for every instance (success) and ensure that explanations for different classes are inconsistent (coherence). This highlights a core trade-off in sample-based explanation generation.
- Argumentation as a Framework: Framing explanations as arguments and conflicts as attacks provides a principled way to manage competing explanations and derive robust insights.
- Parameterized Functions for Trade-offs: The proposed parameterized family of functions allows researchers to explicitly choose the desired trade-off between success and coherence, offering flexibility based on application needs.
- Universal vs. Existential Inference: The choice of inference rule (universal vs. existential) is critical. Universal inference guarantees coherence but may sacrifice success, while existential inference can lead to more explanations but risks incorrect ones.
Key Results
- Theoretical Guarantees: The paper provides theoretical guarantees for its proposed explanation functions, showing which combinations satisfy success and/or coherence.
- Experimental Validation: Experiments on various datasets demonstrate that:
- The proposed functions that guarantee coherence (using universal inference) explain a significantly higher proportion of instances compared to existing methods that aim for correctness.
- The function
gMax,∀(which prioritizes correctness) is shown to be very cautious, often explaining very few instances. - Functions that maximize the number of explained instances (
gMix,∀) or explained classes (gCardc,∀) show promising performance in practice.
Limitations and Future Work
- Trade-off Remains: The fundamental trade-off between success and coherence persists. The paper offers ways to navigate this trade-off but doesn't eliminate it.
- Future Work: The authors suggest exploring other attack relations to minimize the distance between plausible and absolute explanations. They also propose incorporating feature weights to assign a basic weight to arguments, reflecting feature importance. This could lead to more nuanced and informative explanations.
Summary updated