Counterfactual Explanation Based on Gradual Construction for Deep Networks

Sin-Han Kang, Honggyu Jung, Dong-Ok Won, Seong-Whan LeearXiv 2020 · DOI · OpenReview

TL;DR

This paper introduces a novel "gradual construction" method for generating counterfactual explanations for deep neural networks. By iteratively masking important features and composing new values that align with the training data's logit distribution, the method produces more human-interpretable and less adversarial explanations with fewer feature modifications.

What Problem Does This Paper Address?

Deep neural networks, despite their impressive performance, are often "black boxes," making it difficult to understand why they make certain decisions. Counterfactual explanations aim to address this by identifying how input features should be changed to alter a prediction to a desired target class. However, existing methods often modify features without considering the underlying data distribution, leading to explanations that are unrealistic or even adversarial. This paper tackles the challenge of generating counterfactual explanations that are not only interpretable but also grounded in the learned patterns of the training data, thus avoiding unrealistic or misleading outputs.

Methods

The core of the proposed method is a "gradual construction" process that iteratively refines a counterfactual explanation. This process involves two key steps:

  • Masking Step: This step identifies the most influential features in the input data that, if modified, would lead to the target class. It uses the directional derivative of the network's output with respect to the input to find features with high sensitivity.
  • Composition Step: Once important features are identified, this step optimizes their new values. Crucially, it ensures that the output score of the perturbed data is close to the logit distribution of the training data that belongs to the target class. This is achieved by minimizing a loss function that balances increasing the target class probability with aligning the perturbed data's logit score to the training data's distribution.

This iterative process of masking and composition continues until the desired classification probability for the target class is achieved.

Datasets

The paper evaluates its method on a variety of datasets:

  • Text: IMDB sentiment analysis dataset.
  • Image: MNIST handwritten digit dataset.
  • Tabular: HELOC (Home Equity Line of Credit) and UCI Credit Card datasets.

No new benchmarks were introduced; existing, well-established datasets were used.

Key Insights and Ideas

  • Logit Distribution Matters: The most significant insight is the importance of considering the logit distribution of the training data during the composition step. This prevents the generation of adversarial examples and ensures that the counterfactual explanations are more aligned with how the model learned from real data.
  • Gradual Construction for Interpretability: Iteratively refining the explanation through masking and composition leads to more focused and understandable changes, rather than drastic, potentially nonsensical alterations.
  • Minimality and Explainability: The method aims to achieve both minimality (fewest feature changes) and explainability (human-understandable reasons for the change), which are often in tension in existing methods.
  • Domain Agnosticism: The proposed method is designed to be domain-agnostic, applicable to text, image, and tabular data.

Key Results

  • Human-Friendly Explanations: The method consistently produces more human-acceptable and interpretable counterfactual explanations compared to state-of-the-art methods across various domains.
  • Fewer Feature Modifications: The proposed approach often requires fewer feature modifications to achieve the desired counterfactual explanation, contributing to its minimality property.
  • Reduced Adversarial Examples: By incorporating the logit distribution of training data, the method significantly reduces the generation of adversarial examples, which are common in other counterfactual explanation techniques.
  • Quantitative Metrics: The paper presents quantitative evaluations using L1, L2, and coherence metrics on tabular datasets, showing favorable results for their method in terms of minimality and consistency. For instance, on the HELOC dataset, their method achieved a lower L1 metric (1.05 ± 0.10) compared to others, indicating fewer feature changes.

Limitations and Future Work

  • Combining Factual and Counterfactual Explanations: While the proposed method excels at generating counterfactual explanations, it doesn't simultaneously provide factual explanations (i.e., why the original prediction was made). The authors suggest that combining their approach with methods like LORE [32] or ABELE [31] could offer a more comprehensive understanding.
  • Grammatical Correctness (Text Domain): In the text domain, the method might produce grammatically incorrect but semantically relevant word changes. Future work could incorporate linguistic constraints to ensure grammatical correctness.
  • Non-linearity and Local Decision Boundaries: For tabular data, the non-linearity of feature spaces can lead to different local decision boundaries, potentially impacting the coherence of explanations. Further research into methods that provide both factual and counterfactual explanations simultaneously is suggested.

Summary updated

Abstract

To understand the black-box characteristics of deep networks, counterfactual explanation that deduces not only the important features of an input space but also how those features should be modified to classify input as a target class has gained an increasing interest. The patterns that deep networks have learned from a training dataset can be grasped by observing the feature variation among various classes. However, current approaches perform the feature modification to increase the classification probability for the target class irrespective of the internal characteristics of deep networks. This often leads to unclear explanations that deviate from real-world data distributions. To address this problem, we propose a counterfactual explanation method that exploits the statistics learned from a training dataset. Especially, we gradually construct an explanation by iterating over masking and composition steps. The masking step aims to select an important feature from the input data to be classified as a target class. Meanwhile, the composition step aims to optimize the previously selected feature by ensuring that its output score is close to the logit space of the training data that are classified as the target class. Experimental results show that our method produces human-friendly interpretations on various classification datasets and verify that such interpretations can be achieved with fewer feature modification.
BibTeX
@misc{kang2020counterfactual,
  author = {Sin-Han Kang and Honggyu Jung and Dong-Ok Won and Seong-Whan Lee},
  title = {{Counterfactual Explanation Based on Gradual Construction for Deep Networks}},
  year = {2020},
  eprint = {2008.01897},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2008.01897},
}

Loading related content...

Figure