Designing Counterfactual Generators using Deep Model Inversion

Jayaraman J. Thiagarajan, Vivek Sivaraman Narayanaswamy, Deepta Rajan, Jia Liang, Akshay Chaudhari, Andreas Spanias

TL;DR

DISC (Deep Inversion for Synthesizing Counterfactuals) allows researchers to generate realistic "what-if" image explanations for a deep learning model using only the trained classifier itself—no training data or pre-trained GANs required. By combining strong image priors with a novel "manifold consistency" objective, DISC produces interpretable, localized changes that reveal how a model perceives different classes.

The Challenge of Data-Free Counterfactuals

Counterfactual (CF) explanations are a powerful tool for interpretability. If a model classifies an image as "not smiling," a counterfactual shows you the smallest possible change needed to make the model say "smiling."

Most current methods for generating these images rely on having access to the original training data or a pre-trained generative model (like a GAN). However, in many real-world scenarios—such as medical imaging or proprietary software—the training data is unavailable due to privacy or intellectual property concerns.

Progression of counterfactual generation from basic inversion to the full DISC method.

Without the "guardrails" of training data, standard techniques like Deep Model Inversion tend to produce unrealistic, "trippy" images or adversarial noise that satisfies the model's math but means nothing to a human. The challenge is: How do we generate realistic, semantically meaningful counterfactuals using only the trained classifier?

The DISC Approach

The authors propose DISC, which treats the trained classifier as a source of knowledge to be "inverted." They introduce three key innovations to make this work:

  1. Strong Image Priors: Instead of optimizing pixels directly, they use Deep Image Priors (DIP) (using a U-Net structure) and Implicit Neural Representations (INR). INRs, specifically using Fourier mapping and SIREN activations, help the model learn a continuous mapping from coordinates to colors, which acts as a natural stabilizer for image synthesis.
  2. Manifold Consistency: This is the "secret sauce." To prevent the generator from creating unrealistic "out-of-distribution" images, they incorporate a constraint that forces the counterfactual to stay near the data manifold. They explore two ways to do this:
  3. Direct Error Prediction (DEP): Training a small auxiliary network to predict the classifier's error; the generator then tries to minimize this predicted error.
  4. Deterministic Uncertainty Quantification (DUQ): Using RBF-based networks to measure how "certain" the model is about a sample.
  5. Progressive Optimization: They don't optimize the whole image at once. They gradually increase the number of layers in the generator being optimized, allowing the model to first capture coarse structures before moving to fine, discernible details.

Datasets

The authors evaluated DISC on two distinct types of data:

  • CelebA Faces: A standard benchmark for facial attributes. They focused on attributes like "smiling," "bald," and "young."
  • ISIC 2018 Skin Lesion Dataset: A high-stakes medical dataset where interpretability is crucial for clinical trust. They generated counterfactuals to show how a lesion might look if it were a different type of skin condition (e.g., changing a Melanoma to a Nevus).

Key Insights and Ideas

  • Latent Space > Image Space: Optimizing in the latent space of the classifier (Latent Space Optimization or LSO) produces much higher-quality results than manipulating raw pixels (ISO).
  • The Manifold is Mandatory: Without the manifold consistency objective, the inversion process can create "holes" or missing pixels. The consistency term acts like an automated "inpainting" tool that fills in the gaps to keep the image looking real.
Effect of manifold consistency on filling missing image regions.
  • INR for Localization: One of the most interesting findings is that Implicit Neural Representations (INR) are better than standard convolutional priors at localizing changes. For a "smiling" counterfactual, INR-based DISC focuses almost exclusively on the mouth and cheeks, whereas other methods might change the background or hair.
  • Robustness to Noise: DISC-generated explanations remain effective even when the input images are corrupted by blur or noise, suggesting the manifold consistency helps "clean" the signal.

Key Results

The authors introduced a metric called Classifier Discrepancy (CD) to measure how "faithful" the counterfactuals are. If you train a new classifier on your generated counterfactuals, does it behave like the original?

  • On CelebA, DISC (using INR + DEP) achieved a CD score of 0.08, significantly outperforming the baseline LSO + TV (0.31).
  • In medical imaging (ISIC), DISC produced changes consistent with the "ABCD" (Asymmetry, Border, Color, Diameter) signatures that doctors actually use to diagnose lesions.
  • Qualitatively, the difference is striking: while baselines often produce blurry or noisy images, DISC produces sharp, localized edits that look like natural photographic changes.

Limitations and Future Work

  • Single Attribute Focus: Currently, the method is designed to change one attribute at a time. The authors note that exploring multiple target attributes simultaneously is a logical next step.
  • Computational Cost: Like most inversion-based methods, DISC requires an optimization process for every query image, which is slower than using a feed-forward generative model (if one were available).
  • Static Data: The current framework is built for static images; the authors suggest that extending this to time-varying or video data would be a valuable future direction.

Summary updated

Abstract

Explanation techniques that synthesize small, interpretable changes to a given image while producing desired changes in the model prediction have become popular for introspecting black-box models. Commonly referred to as counterfactuals, the synthesized explanations are required to contain discernible changes (for easy interpretability) while also being realistic (consistency to the data manifold). In this paper, we focus on the case where we have access only to the trained deep classifier and not the actual training data. While the problem of inverting deep models to synthesize images from the training distribution has been explored, our goal is to develop a deep inversion approach to generate counterfactual explanations for a given query image. Despite their effectiveness in conditional image synthesis, we show that existing deep inversion methods are insufficient for producing meaningful counterfactuals. We propose DISC (Deep Inversion for Synthesizing Counterfactuals) that improves upon deep inversion by utilizing (a) stronger image priors, (b) incorporating a novel manifold consistency objective and (c) adopting a progressive optimization strategy. We find that, in addition to producing visually meaningful explanations, the counterfactuals from DISC are effective at learning classifier decision boundaries and are robust to unknown test-time corruptions.
BibTeX
@inproceedings{thiagarajan2021designing,
  author = {Jayaraman J. Thiagarajan and Vivek Sivaraman Narayanaswamy and Deepta Rajan and Jia Liang and Akshay Chaudhari and Andreas Spanias},
  title = {{Designing Counterfactual Generators using Deep Model Inversion}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=KiejlgrgNB},
}
Figure