TL;DR
This paper tackles the problem of generating too many abductive explanations for machine learning models, making them uninterpretable. They propose a novel approach by incorporating "explainee preferences" to filter and generate only the most relevant explanations, significantly reducing their number and improving their utility.
What Problem Does This Paper Address?
Machine learning models, especially complex ones like random forests, can produce a vast number of "abductive explanations" (reasons why a prediction was made). These explanations, while technically correct, can be exponentially numerous, even for simple models like decision trees. This sheer volume makes it impossible for users (explainees) to process them effectively, defeating the purpose of explainable AI (XAI). The challenge is to move beyond simply generating any explanation to generating useful explanations that align with what the user actually cares about.
Methods
The core innovation lies in introducing "explainee preference models." Instead of generating all possible explanations, the authors propose to first model what makes an explanation "preferred" by the user. They explore several preference models:
- Dichotomous Preferences:
- Feature Subset Preference: Explanations must only use features from a predefined set
S. This is useful for discarding technical or irrelevant features.
- Feature Subset Preference: Explanations must only use features from a predefined set
- Constraint Satisfaction: Explanations must satisfy a given logical constraint
C. This can enforce regulatory requirements or specific user-defined rules. - Gradual Preferences:
- Inclusion-Preferred Explanations: Explanations are ranked based on a feature prioritization. Explanations that use "higher-ranked" features are preferred.
- Minimum-Weight Explanations: Each feature is assigned a weight (cost). Explanations are preferred if they have the lowest total weight, allowing for a trade-off between "expensive" and "cheap" features.
For each preference model, they develop algorithms to compute "preferred majoritary reasons" for random forests. Notably, while generating all minimum-size explanations can be intractable, their methods for preferred majoritary reasons are often polynomial-time, leveraging techniques like weighted partial MAXSAT solvers for the minimum-weight case.
Datasets
The authors evaluated their methods on 22 standard binary classification datasets from Kaggle, OpenML, and UCI repositories. These datasets vary in size and feature count, providing a diverse testbed for their algorithms. No new benchmarks were introduced; the focus was on applying their methods to existing, well-known datasets.
Key Insights and Ideas
- The "Too Many Explanations" Problem is Real: Even for "intelligible" models like decision trees, the number of explanations can be astronomically large, rendering them useless.
- User Preferences are Key: The utility of an explanation is not solely intrinsic but heavily depends on the user's needs and understanding. Modeling these preferences is crucial for effective XAI.
- Preferred Explanations Drastically Reduce Complexity: By focusing on preferred explanations, the authors demonstrate that the number of relevant reasons can be significantly reduced, making enumeration feasible in cases where it was previously impossible.
- Flexibility in Preference Modeling: The paper offers a spectrum of preference models, from simple feature inclusion to more nuanced weighted trade-offs, catering to various user needs.
- Tractability for Random Forests: While some explanation tasks for random forests are intractable, their proposed methods for preferred majoritary reasons are often computationally efficient.
Key Results
The experimental results, summarized in Table 2, show that their approach is practical. For many datasets and weight functions, they could compute all minimum-weight majoritary reasons within a 60-second time limit per instance. This was particularly effective when using weight functions derived from SHAP values, feature importance, or word frequencies, which drastically reduced the number of reasons compared to a uniform weight function (equivalent to minimum-size reasons). The paper highlights that using these preference-based approaches makes the enumeration of reasons feasible in scenarios where it would otherwise be out of reach.
Limitations and Future Work
- Defining "Preferred" is Subjective: The paper acknowledges that defining what constitutes a "preferred" explanation is a difficult task with no universal consensus. The proposed models are a step towards formalizing this, but human evaluation is still needed.
- Human-Grounded Evaluation: The authors suggest that future work should focus on more human-grounded evaluation of explanations, moving beyond purely functional or computational aspects.
- Application-Specific Evaluation: They express interest in applying their methods to specific applications, like sale prediction, to further assess the practical utility and user acceptance of preferred explanations.
- Approximation for Hard Cases: For computationally intensive cases (like minimum-weight majoritary reasons), they propose using anytime MAXSAT solvers to derive approximations, though they note that the weight of the approximated reason might not be close to the optimal.
Summary updated