Generating Natural Counterfactual Visual Explanations

Wenqi Zhao, Satoshi Oyama, Masahito KuriharaIJCAI 2020 · DOI

TL;DR

This paper introduces a novel method for generating natural counterfactual visual explanations for image classification models. By modifying attribute descriptions and using text-to-image GANs, they create visual examples that help users understand why a model made a certain prediction, leading to improved classification accuracy in human studies.

What Problem Does This Paper Address?

As machine learning models become more prevalent in real-world applications, understanding their decision-making processes (interpretability) is crucial for building trust, ensuring safety, and debugging. For image classification, a key challenge is explaining why a model classified an image as a certain category. Existing methods often focus on highlighting important regions within an image. This paper tackles the problem from a "counterfactual thinking" perspective: what minimal changes to an input image would lead the model to predict a different category? Specifically, they aim to generate visual explanations that are more intuitive and natural than just highlighting pixels.

Methods

The core of their approach involves generating counterfactual visual explanations through a two-step process:

  1. Predefined Description Texts: They start by creating detailed, predefined text descriptions for bird classes, focusing on specific attributes (bill, head, breast, etc.). This is crucial because the quality of the counterfactual explanation depends on the richness of these attribute descriptions. They deliberately exclude attributes like size, which are hard to accurately represent visually.
  2. Generating Counterfactual Visual Explanations:
    • They identify counterfactual features – attributes present in a target class (e.g., class B) but absent in the original class (e.g., class A).
  3. These features are used to modify the text description of class A, creating a "counterfactual text" that incorporates features of class B.
  4. A text-to-image Generative Adversarial Network (GAN) is then employed to generate a new image based on this modified text. They use established GAN models like StackGAN and AttnGAN.
  5. The generated counterfactual image is fed back into the classification model. The "best" counterfactual image is selected based on how strongly it shifts the prediction probability towards the target class (class B) while decreasing it for the original class (class A). This selection is guided by the logarithm of the probability ratio.

Datasets

The paper mentions applying their method to a "fine-grained image classification dataset" for birds. Specific dataset names are not provided, but the context suggests a dataset with detailed bird species classifications.

Key Insights and Ideas

  • Counterfactual Thinking for Visual Explanations: Shifting the focus from "what's important in this image?" to "what would need to change for it to be something else?" offers a more intuitive way to understand model behavior.
  • Text-to-Image GANs as Explanation Generators: Leveraging GANs to generate entirely new images based on modified textual descriptions is a powerful way to create natural-looking counterfactuals, overcoming limitations of simple image manipulation or region swapping.
  • Attribute-Based Explanations: The method relies heavily on well-defined textual attributes, highlighting the importance of rich, structured data for interpretability.
  • Improved Human Classification: The generated visual explanations can directly aid human users in classification tasks, suggesting their potential for practical applications beyond model debugging.

Key Results

The paper reports a significant improvement in classification accuracy when human crowd workers were provided with their counterfactual visual explanations.

  • Classification accuracy without explanations: 0.81
  • Classification accuracy with explanations: 0.92

This demonstrates that their method effectively enhances human understanding and classification performance.

Limitations and Future Work

  • Reliance on Predefined Text: The quality of explanations is heavily dependent on the availability and detail of predefined attribute descriptions. If these are insufficient or inaccurate, the counterfactuals will suffer.
  • GAN Model Limitations: The quality of the generated counterfactual images is limited by the capabilities of the text-to-image GAN models used. Generating complex or unique shape features can be challenging.
  • Future Work: The authors plan to focus on generating more complex and unique shape features to improve the quality of counterfactual visual explanations. They also aim to conduct more extensive crowdsourcing experiments to further validate their findings.

Summary updated

BibTeX
@inproceedings{zhao2020generating,
  author = {Wenqi Zhao and Satoshi Oyama and Masahito Kurihara},
  title = {{Generating Natural Counterfactual Visual Explanations}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/742},
}
Figure