Explaining Image Classifiers Generating Exemplars and Counter-Exemplars from Latent Representations

Riccardo Guidotti, Anna Monreale, Stan Matwin, Dino PedreschiAAAI 2020 · DOI

Introduction

As deep learning models become more pervasive in critical fields like medicine, their "black box" nature becomes a liability. Understanding why a model classified an X-ray as "malignant" is essential for trust and safety. Traditional explanation methods often fall short: gradient-based methods are model-specific and limited to heatmaps, while model-agnostic methods like LIME often produce "nonsense" images by perturbing pixels directly, leading to unstable and unreliable explanations.

The ABELE Framework

The authors introduce ABELE (Adversarial Black box Explainer generating Latent Exemplars), a model-agnostic method that operates in a latent space rather than pixel space. This approach ensures that synthetic "neighboring" images used for explanations remain realistic and semantically meaningful, avoiding the "out-of-distribution" problem that plagues pixel-masking methods.

Methodology

ABELE's process involves several sophisticated stages centered around a Latent Local Rule-based Explainer (llore):

  1. Latent Mapping: An Adversarial Autoencoder (AAE) learns a compressed, structured representation of the dataset.
  2. Neighborhood Generation: Synthetic instances are generated in the latent space near the target image. Because these points are in the learned latent space, they correspond to realistic images when decoded.
  3. Local Surrogate Model: A simple, interpretable decision tree is trained on this neighborhood to mimic the black-box model's behavior locally.
  4. Exemplar Extraction: The system identifies "exemplars" (similar images with the same class) and "counter-exemplars" (similar images with a different class).
The Latent Local Rule-based Explainer (llore) architecture within ABELE.

Key Insights: The Power of Contrast

A core strength of ABELE is its ability to provide contrastive explanations. Humans often understand decisions better through comparison. For example, showing a "9" that is classified as a "4" because the top loop isn't closed (a counter-exemplar) is more intuitive than a simple heatmap. This contrastive approach highlights exactly what features keep an image in its class or push it into another.

Visualizing exemplars and counter-exemplars helps clarify decision boundaries.

Results and Evaluation

The researchers evaluated ABELE on MNIST, Fashion-MNIST, and CIFAR-10 datasets against Random Forests (RF) and Deep Neural Networks (DNN). Key findings include:

  • Fidelity: ABELE's decision tree surrogate mimicked black-box behavior more accurately than LIME, particularly for RF models.
  • Robustness: ABELE significantly outperformed LIME in stability tests. In a "local Lipschitz" test for the MNIST dataset, ABELE showed a robustness score of 0.38, while LIME lagged at 1.5 (where lower is better).
  • Relevance: Masking experiments confirmed that areas highlighted by ABELE's saliency maps were truly critical to the model's decision; hiding these areas caused classification accuracy to drop sharply.

Conclusion

By leveraging latent spaces and adversarial training, ABELE provides stable, realistic, and contrastive explanations that move beyond simple heatmaps. While it requires more computational overhead to train the initial autoencoder, the resulting transparency and robustness offer a significant step forward for interpretable AI. Future work aims to extend this framework to tabular, text, and time-series data, as well as specialized fields like medical radiography.

Summary updated

BibTeX
@inproceedings{guidotti2020explaining,
  author = {Riccardo Guidotti and Anna Monreale and Stan Matwin and Dino Pedreschi},
  title = {{Explaining Image Classifiers Generating Exemplars and Counter-Exemplars from Latent Representations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i09.7116},
}
Figure