Introduction
As deep learning models become more pervasive in critical fields like medicine, their "black box" nature becomes a liability. Understanding why a model classified an X-ray as "malignant" is essential for trust and safety. Traditional explanation methods often fall short: gradient-based methods are model-specific and limited to heatmaps, while model-agnostic methods like LIME often produce "nonsense" images by perturbing pixels directly, leading to unstable and unreliable explanations.
The ABELE Framework
The authors introduce ABELE (Adversarial Black box Explainer generating Latent Exemplars), a model-agnostic method that operates in a latent space rather than pixel space. This approach ensures that synthetic "neighboring" images used for explanations remain realistic and semantically meaningful, avoiding the "out-of-distribution" problem that plagues pixel-masking methods.
Methodology
ABELE's process involves several sophisticated stages centered around a Latent Local Rule-based Explainer (llore):
- Latent Mapping: An Adversarial Autoencoder (AAE) learns a compressed, structured representation of the dataset.
- Neighborhood Generation: Synthetic instances are generated in the latent space near the target image. Because these points are in the learned latent space, they correspond to realistic images when decoded.
- Local Surrogate Model: A simple, interpretable decision tree is trained on this neighborhood to mimic the black-box model's behavior locally.
- Exemplar Extraction: The system identifies "exemplars" (similar images with the same class) and "counter-exemplars" (similar images with a different class).
Key Insights: The Power of Contrast
A core strength of ABELE is its ability to provide contrastive explanations. Humans often understand decisions better through comparison. For example, showing a "9" that is classified as a "4" because the top loop isn't closed (a counter-exemplar) is more intuitive than a simple heatmap. This contrastive approach highlights exactly what features keep an image in its class or push it into another.
Results and Evaluation
The researchers evaluated ABELE on MNIST, Fashion-MNIST, and CIFAR-10 datasets against Random Forests (RF) and Deep Neural Networks (DNN). Key findings include:
- Fidelity: ABELE's decision tree surrogate mimicked black-box behavior more accurately than LIME, particularly for RF models.
- Robustness: ABELE significantly outperformed LIME in stability tests. In a "local Lipschitz" test for the MNIST dataset, ABELE showed a robustness score of 0.38, while LIME lagged at 1.5 (where lower is better).
- Relevance: Masking experiments confirmed that areas highlighted by ABELE's saliency maps were truly critical to the model's decision; hiding these areas caused classification accuracy to drop sharply.
Conclusion
By leveraging latent spaces and adversarial training, ABELE provides stable, realistic, and contrastive explanations that move beyond simple heatmaps. While it requires more computational overhead to train the initial autoencoder, the resulting transparency and robustness offer a significant step forward for interpretable AI. Future work aims to extend this framework to tabular, text, and time-series data, as well as specialized fields like medical radiography.
Summary updated