Twin-Systems to Explain Artificial Neural Networks using Case-Based Reasoning: Comparative Tests of Feature-Weighting Methods in ANN-CBR Twins for XAI

Eoin M. Kenny, Mark T. KeaneIJCAI 2019 · DOI

TL;DR

This paper introduces "twin-systems" to tackle the explainability challenge in Artificial Intelligence (AI), specifically for "black box" Artificial Neural Networks (ANNs). By creating an interpretable "twin" model (Case-Based Reasoning, CBR) that mirrors the ANN's behavior, the authors demonstrate a novel way to understand ANN predictions. They also introduce and evaluate a new feature-weighting method, COLE, which proves most effective in this twinning approach.

What Problem Does This Paper Address?

As AI systems, particularly ANNs, become increasingly prevalent in decision-making across various domains, their lack of transparency poses a significant problem. Users and regulators alike are demanding explanations for AI-driven outcomes, a field known as Explainable AI (XAI). ANNs, with their complex, non-linear structures, are notoriously difficult to interpret, making it hard to understand why they make a particular prediction. This paper addresses this "black box" problem by proposing a "twin-system" approach, where a less interpretable ANN is paired with a more transparent CBR system. The goal is to leverage the interpretability of CBR to explain the ANN's decisions.

Methods

The core of the paper is the "twin-system" framework, specifically focusing on ANN-CBR twins. This involves:

  • Twinning: Creating a CBR system that acts as a "twin" to an ANN. Both systems operate on the same dataset.
  • Feature Weighting: Extracting feature weights from the ANN to inform the CBR system's reasoning process. This is crucial because the importance of features in explaining a prediction can vary.
  • Novel Evaluation Methodology: A new method is proposed to quantitatively assess the "agreement" between the ANN and its CBR twin. High agreement signifies that the extracted feature weights accurately represent the ANN's decision-making process.
  • COLE (Contributions Oriented Local Explanations): A novel feature-weighting method is introduced. It leverages saliency map techniques (like LRP, IG, and DeepLIFT) to estimate feature contributions, which are then used to weight the CBR's k-NN search. This method aims to capture how features directly contribute to the ANN's classification.
  • Comparative Testing: The paper competitively tests several existing and the new COLE feature-weighting methods in two experiments.

Datasets

The paper utilizes several publicly available datasets for evaluation:

  • Experiment 1 (MLP-CBR Twins):
    • Connect-4
    • Thyroid Disease
    • Nursery
    • Adult Census
    • Default of Credit
    • Bank Marketing
    • Breast Cancer
  • Experiment 2 (CNN-CBR Twins):
    • MNIST
    • MNIST-Fashion
    • CIFAR-10
    • CIFAR-10* (using transfer learning with InceptionResNetV2)

These datasets are chosen to challenge unweighted k-NN baselines and represent both tabular and image data.

Key Insights and Ideas

  • Twin-Systems as a Promising XAI Approach: The paper strongly advocates for the twin-system framework as a viable method for achieving explainability, particularly by pairing ANNs with CBR systems.
  • Feature Weighting is Key: The effectiveness of ANN-CBR twins hinges on accurately extracting feature weights from the ANN. Different weighting methods yield significantly different levels of agreement between the twin systems.
  • Local vs. Global Weighting: Local feature-weighting methods generally outperform global methods, suggesting that feature importance is often context-dependent rather than a fixed, universal value.
  • Contribution-Based Weighting Excels: The novel COLE method, which focuses on feature contributions, consistently achieves the highest agreement scores, indicating its superior ability to capture the ANN's decision logic.
  • CNN Explanations are Plausible: For image data, the ANN-CBR twins, particularly with COLE, provide qualitatively convincing explanations, showing nearest neighbors that visually justify the CNN's classifications.

Key Results

  • MLP-CBR Twins: The COLE method (specifically C-DeepLIFT) achieved the highest mean agreement score of 0.99, significantly outperforming other methods and the baseline unweighted k-NN (0.91). Local weighting methods generally showed better performance than global ones.
  • CNN-CBR Twins: The C-DeepLIFT weighted k-NN achieved a very high mean agreement score of 0.996, demonstrating strong fidelity to the CNN's predictions. This performance was particularly notable in complex multi-class classification tasks.
  • Qualitative Evidence: Figures in the paper illustrate how CNN-CBR twins can provide intuitive explanations for image classifications, showing nearest neighbors that visually resemble the queried image and highlighting key discriminating features.

Limitations and Future Work

  • Focus on Agreement: The current evaluation methodology primarily focuses on predictive agreement between the twin systems, rather than directly assessing the "goodness" or user-understandability of the explanations themselves.
  • Computational Cost: While the paper notes that deriving weights is a one-time cost and subsequent queries are fast, the initial computation for large datasets can still be significant (e.g., ~6613 seconds for CIFAR-10).
  • Generalizability of COLE: The authors plan to generalize COLE to be applicable to machine learning models beyond just ANNs, such as Support Vector Machines (SVMs).
  • User Studies: Future work will involve user studies to assess the explanatory value of the retrieved cases from the CBR twin.
  • Broader Twin-System Applications: The paper suggests exploring other combinations of "black box" and "white box" models for twinning.

Summary updated

BibTeX
@inproceedings{kenny2019twin,
  author = {Eoin M. Kenny and Mark T. Keane},
  title = {{Twin-Systems to Explain Artificial Neural Networks using Case-Based Reasoning: Comparative Tests of Feature-Weighting Methods in ANN-CBR Twins for XAI}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/376},
}
Figure