Understanding Instance-based Interpretability of Variational Auto-Encoders

Zhifeng Kong, Kamalika Chaudhuri

TL;DR

This paper extends instance-based interpretability—the ability to explain a model's behavior through specific training examples—to the unsupervised world of Variational Auto-Encoders (VAEs). The authors introduce VAE-TracIn, a computationally efficient method to identify which training samples most influence a VAE's likelihood for a given test point, enabling practical applications like unsupervised data cleaning.

What Problem Does This Paper Address?

In supervised learning, we can ask, "Which training images caused the model to label this test image as a 'cat'?" This is often answered using influence functions, which quantify how much a model's prediction would change if a specific training point were removed.

However, unsupervised learning (where there are no labels) presents two major hurdles for this kind of interpretability:

  1. Framing the Question: Without labels, what are we measuring influence on? The authors propose framing the counter-factual question as: "Which training samples are most responsible for increasing the likelihood of a test sample?"
  2. Computational Complexity: VAEs are deep generative models where the loss function involves an expectation over a latent space. Calculating standard influence functions requires inverting a massive Hessian matrix (second-order derivatives), which is computationally prohibitive for neural networks with millions of parameters.

Methods

The authors bridge this gap through a mix of theoretical analysis and a new algorithm:

  • Theoretical Grounding: They first analyze influence functions for classical unsupervised methods like k-Nearest Neighbors (k-NN), Kernel Density Estimation (KDE), and Gaussian Mixture Models (GMM) to build intuition on how "proponents" (helpful samples) and "opponents" (harmful samples) behave.
  • VAE-TracIn: To make influence tractable for VAEs, they adapt the "TracIn" algorithm. Instead of calculating the Hessian, TracIn estimates influence by "tracing" the gradient updates of a sample across several checkpoints saved during the training process.
  • Handling Expectations: Since the VAE loss is an expectation, they use an empirical average over $m$ samples from the encoder. They provide a mathematical proof (a concentration bound) showing that this empirical average is a reliable estimate of the true influence.

Datasets

The researchers evaluated their method on several standard computer vision benchmarks:

  • MNIST: Handwritten digits.
  • CIFAR-10: Natural images (animals, vehicles).
  • CIFAR Subclasses: Individual classes from CIFAR-10 (e.g., just the "Airplane" class) to test fine-grained influence.
  • EMNIST & CelebA: Used as "outlier" datasets to test the method's ability to perform unsupervised data cleaning.

Key Insights and Ideas

  • Self-Influence as a "Hardness" Metric: A sample’s "self-influence" (how much it influences its own likelihood) is highly correlated with its loss. High self-influence samples are often "atypical"—visually complicated, high-contrast, or noisy.
Scatter plots showing the correlation between VAE-TracIn self-influence and loss across different datasets.
  • Proponents vs. Opponents:
    • In MNIST, proponents are visually similar (the same digit).
  • In CIFAR, proponents often share background colors or have large norms in the latent space, meaning they impact a large region of the model's distribution.
Test samples and their strongest proponents identified by VAE-TracIn for MNIST, CIFAR, and CIFAR subclasses.
  • Unsupervised Data Cleaning: Because outliers or "noisy" data points tend to have very high self-influence, VAE-TracIn can be used to flag suspicious samples for human review without needing any labels.
  • GMM Intuition: Surprisingly, the authors found that in high-dimensional spaces (like GMMs), "opponents" can sometimes come from the same class if they are far from the test point, as they "pull" the cluster center away from that point.

Key Results

  • Sanity Check: VAE-TracIn passed a "self-influence" test, correctly identifying a training sample as its own strongest proponent nearly 100% of the time on MNIST and CIFAR subclasses.
  • Data Cleaning Performance: In experiments where "extra" samples from different datasets were mixed in, VAE-TracIn detected them with an Area Under the Curve (AUC) of 0.887 for MNIST and 0.760 for CIFAR.
  • Latent Space Proximity: On MNIST, the strongest proponents for a test sample were significantly closer in the latent space (mean distance ~0.94) compared to the average distance of the entire training set (~2.54).

Limitations and Future Work

  • Scalability: While VAE-TracIn is faster than Hessian-based methods, it still requires multiple training checkpoints and gradient calculations, which can be slow for massive, modern datasets.
  • Optimization: The authors suggest that using only the gradients of the last layer or selecting a specific subset of "optimal" checkpoints could further speed up the process.
  • Downstream Tasks: The paper opens the door for using these influence scores for "machine unlearning" (efficiently deleting data from a model) and debugging bias in generative models.

Summary updated

Abstract

Instance-based interpretation methods have been widely studied for supervised learning methods as they help explain how black box neural networks predict. However, instance-based interpretations remain ill-understood in the context of unsupervised learning. In this paper, we investigate influence functions [Koh and Liang, 2017], a popular instance-based interpretation method, for a class of deep generative models called variational auto-encoders (VAE). We formally frame the counter-factual question answered by influence functions in this setting, and through theoretical analysis, examine what they reveal about the impact of training samples on classical unsupervised learning methods. We then introduce VAE- TracIn, a computationally efficient and theoretically sound solution based on Pruthi et al. [2020], for VAEs. Finally, we evaluate VAE-TracIn on several real world datasets with extensive quantitative and qualitative analysis.
BibTeX
@inproceedings{kong2021understanding,
  author = {Zhifeng Kong and Kamalika Chaudhuri},
  title = {{Understanding Instance-based Interpretability of Variational Auto-Encoders}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=a5-37ER8qTI},
}
Figure