Interpret the Predictions of Deep Networks via Re-Label Distillation

Yingying Hua, Shiming Ge, Daichi Zhang

TL;DR

This paper introduces re-label distillation, a local interpretability method that probes the "black box" of deep neural networks (DNNs) by generating synthetic variations of an input image. By training a simple, linear student model to mimic how a complex DNN classifies these synthetic images, the authors create highly accurate saliency maps that pinpoint exactly which features drove the original prediction.

What Problem Does This Paper Address?

Deep neural networks are notoriously opaque. While they achieve state-of-the-art accuracy, their complex, non-linear decision-making processes are often incomprehensible to humans. This lack of transparency is a major hurdle for deploying AI in high-stakes fields like medicine or security, where understanding why a model failed is as important as its overall success.

Existing "post-hoc" explanation methods, such as Grad-CAM or RISE, often produce blurry saliency maps or struggle to accurately capture the model's local decision boundary. The authors aim to bridge this gap by creating a more intuitive and precise way to map the relationship between specific input pixels and the final classification.

Methods

The proposed framework, illustrated in Figure 2, consists of three main stages:

Fig. 2. Overview of our approach. For a given image, we first use a pre-trained VAE to generate some synthetic images by perturbing the latent vector with random noise. Then, we re-label these synthetic images through a pre-trained CNN into one of two classed by identifying whether their predictions shift. Finally, we train a two-class linear model by distilling the soft logits from CNN with these re-labeled synthetic images. Therefore, the weights of the trained linear model can mark the location of the important features contributed to its prediction, which could generate a saliency map to interpret the prediction of the image.
  1. Self-supervised Image Synthesis: For a specific input image, the researchers use a pre-trained Variational Autoencoder (VAE). By adding random noise to the image's latent representation, the VAE generates a set of "synthetic images" that are semantically similar to the original but vary slightly in their features.
  2. Re-labeling: These synthetic images are fed into the "teacher" model (the black-box DNN we want to explain). Each synthetic image is assigned a binary label: 1 if the DNN's prediction matches the original image's classification, and 0 if the prediction shifts. This process effectively maps the DNN's local decision boundary.
  3. Re-label Distillation: A simple, two-class linear model (the "student") is trained to approximate the teacher's behavior on these synthetic images. The loss function combines "soft" knowledge (matching the teacher's raw output logits) and "hard" labels (matching the binary re-labels).

Because the student model is linear, its learned weights ($w$) directly represent the importance of each pixel. These weights are then visualized as a saliency map to explain the prediction.

Datasets

The authors evaluated their approach using the standard ImageNet dataset. They used two widely recognized architectures as their "black-box" teacher models:

  • ResNet50
  • VGG16

Key Insights and Ideas

  • Local Boundary Probing: Instead of just looking at gradients (which can be noisy), this method probes the model's behavior in a neighborhood around the input. As shown in the t-SNE plot in Figure 3, synthetic images are naturally "sortable" in the label domain, meaning they clearly delineate where the model changes its mind.
  • Distillation as Interpretation: The core "aha moment" is treating interpretability as a knowledge distillation problem. By forcing a complex model's local logic into a simple linear student, the inherent interpretability of linear models is transferred to the complex one.
  • VAE vs. Random Noise: Using a VAE to generate perturbations is more effective than random pixel masking because it creates variations that are more likely to lie on the natural data manifold, providing a more realistic probe of the model's logic.
Fig. 3. An example of t-SNE [12] plots. Points are colored by their reconstructed labels. The plots indicate that the synthetic images are sortable and could be projected into the label do- main to represent the boundary knowledge of deep networks.

Key Results

The authors compared their method against eight state-of-the-art approaches, including RISE, Grad-CAM, and LIME.

  • Qualitative Superiority: As seen in Figure 4, the re-label distillation saliency maps (column h) are significantly more focused and accurate. For example, in the "Beagle" image, the method clearly highlights the dog's face and body while ignoring irrelevant background grass, whereas other methods are much noisier.
  • Quantitative Performance: The method was tested using Deletion (how fast does confidence drop when salient pixels are removed?) and Insertion (how fast does confidence rise when salient pixels are added?) metrics.
  • On ResNet50, their method achieved a Deletion score of 0.0627 (lower is better), significantly outperforming RISE (0.1076) and Grad-CAM (0.1421).
  • On VGG16, their Insertion score reached 0.7981 (higher is better), a substantial lead over RISE (0.6663).
  • Sensitivity: Figure 5 demonstrates that the model's confidence is highly sensitive to the features identified by this method, confirming that it has successfully captured the most influential pixels.
Fig. 4. The qualitative comparisons with state-of-the-art methods, including (a) Linear approximation, (b) RISE [13], (c) Exci- tation backprop [14], (d) Extremal perturbations [15], (e) Grad-CAM [16], (f) Score-CAM [17], (g) Occlusion sensitivity [18], (h) Re-Label distillation (ours). Our saliency maps mark the target area of the image more accurately and show the degree of importance to the predictions more clearly, which means our method generates a better visual explanation than others.
Fig. 5. Quantitative results on ResNet50 and VGG16. With the deletion (left) or insertion (right) of the salient features, the obvious change in class probability validates the signifi- cance of these features contributed to the models’ predictions.

Limitations and Future Work

  • Computational Overhead: The process requires generating 1,000 synthetic images and training a linear model for every image one wishes to explain. This is computationally more expensive than simple gradient-based methods like Grad-CAM.
  • Generative Model Dependency: The quality of the explanation is partially tied to the VAE's ability to generate meaningful variations. If the VAE cannot represent certain features of the input image well, the explanation might be incomplete.
  • Future Directions: The authors suggest that this work provides a foundation for future research into more efficient and universal explainable AI (XAI) frameworks.

Summary updated

Abstract

Interpreting the predictions of a black-box deep network can facilitate the reliability of its deployment. In this work, we propose a re-label distillation approach to learn a direct map from the input to the prediction in a self-supervision manner. The image is projected into a VAE subspace to generate some synthetic images by randomly perturbing its latent vector. Then, these synthetic images can be annotated into one of two classes by identifying whether their labels shift. After that, using the labels annotated by the deep network as teacher, a linear student model is trained to approximate the annotations by mapping these synthetic images to the classes. In this manner, these re-labeled synthetic images can well describe the local classification mechanism of the deep network, and the learned student can provide a more intuitive explanation towards the predictions. Extensive experiments verify the effectiveness of our approach qualitatively and quantitatively.
BibTeX
@inproceedings{hua2021interpret,
  author = {Yingying Hua and Shiming Ge and Daichi Zhang},
  title = {{Interpret the Predictions of Deep Networks via Re-Label Distillation}},
  booktitle = {ICME},
  year = {2021},
}
Figure