Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples

Alejandro Barredo Arrieta, Javier Del Ser

TL;DR

This paper introduces a novel framework for auditing deep learning classifiers by generating "plausible" adversarial examples. By combining Generative Adversarial Networks (GANs) with multi-objective optimization, they create realistic counterfactuals that highlight a model's limitations and potential biases, making them understandable to non-experts.

What Problem Does This Paper Address?

Deep learning models have achieved remarkable performance, but their "black-box" nature makes it difficult to understand why they make certain decisions or where their failure points lie. This lack of interpretability hinders trust, especially for non-expert users. While adversarial attacks exist, they often produce unrealistic examples that don't intuitively reveal a model's weaknesses. This paper addresses the need for a method that generates adversarial examples which are not only effective at confusing a model but are also plausible – meaning they look like something a human could realistically create, thereby providing a more intuitive understanding of the model's boundaries.

Methods

The core of the proposed framework is a hybrid approach:

  • Attribute GAN (AttGAN): This GAN architecture, building on prior work, is adapted to generate images with specific attribute modifications. It uses an encoder-decoder structure and is trained with a combination of reconstruction loss and adversarial losses. A key addition is an attribute classifier that helps guide the generator to produce images with desired attribute changes.
  • Multi-Objective Optimization: To balance the competing goals of creating effective adversarial examples, the authors frame the problem as a multi-objective optimization task. The three objectives are:
  • Plausibility (fgan): Ensuring the generated image is realistic and indistinguishable from real data by a discriminator.
  • Adversarial Success (fadv): Confusing the target classifier (making it misclassify the generated image).
  • Intensity (fatt): Minimizing the magnitude of the attribute changes made to the original image.
  • Meta-Heuristic Search: To efficiently find Pareto-optimal solutions (i.e., attribute modifications that offer the best trade-offs between the three objectives), they employ multi-objective meta-heuristic algorithms, specifically testing NSGA-II, GDE3, and SMPSO, with SMPSO being selected as the best performer.

Datasets

  • Celebrity Dataset [26]: This dataset, containing celebrity face images with annotated facial attributes, was used for training and evaluating the framework. The authors downsized the images to 128x128 pixels for computational efficiency. Thirteen facial attributes were used to feed the GAN.

Key Insights and Ideas

  • Plausibility is Key for Explainability: The paper strongly argues that realistic, human-understandable adversarial examples are crucial for building trust and explaining model behavior to non-experts.
  • GANs as Attribute Editors: Leveraging GANs not just for generation but for targeted attribute manipulation provides a powerful tool for creating specific types of adversarial perturbations.
  • Multi-Objective Optimization for Balanced Auditing: Framing the problem with multiple, often conflicting, objectives (plausibility, adversarial success, intensity) allows for a more nuanced understanding of a model's vulnerabilities.
  • Uncovering Hidden Biases: The framework can reveal subtle biases in datasets or models that might not be apparent through standard evaluation metrics. For instance, a model's reliance on hair color for gender classification was exposed.

Key Results

  • The proposed framework successfully generated plausible counterfactuals for a human face classification task (distinguishing male/female).
  • In Experiment 1, the framework demonstrated that changes like adding blonde hair to male images or darkening hair on female images could confuse the target classifier. This highlighted a potential bias in the model towards hair color.
  • Experiment 2, where blonde females were removed from the training data, showed that the framework could still find counterfactuals, but the influence of hair color was less pronounced, suggesting the model learned to rely on other attributes.
  • The authors found that the "crossing point" between the classifier's confidence in the original class and the plausibility of the counterfactual often represented the most informative and balanced adversarial example.

Limitations and Future Work

  • Limitations: The paper acknowledges that the framework's effectiveness is tied to the quality and comprehensiveness of the attribute annotations in the dataset. The "black-box" nature of the audited model means only its logits are accessible, which is a practical constraint.
  • Future Work: The authors propose several directions:
  • Using the generated counterfactual information to retrain the audited model and improve its robustness or mitigate bias.
  • Exploring Reinforcement Learning as an alternative to meta-heuristics for learning attribute-change relationships.
  • Investigating how counterfactuals can be used to actively counteract bias in training data.

Summary updated

Abstract

The last decade has witnessed the proliferation of Deep Learning models in many applications, achieving unrivaled levels of predictive performance. Unfortunately, the black-box nature of Deep Learning models has posed unanswered questions about what they learn from data. Certain application scenarios have highlighted the importance of assessing the bounds under which Deep Learning models operate, a problem addressed by using assorted approaches aimed at audiences from different domains. However, as the focus of the application is placed more on non-expert users, it results mandatory to provide the means for him/her to trust the model, just like a human gets familiar with a system or process: by understanding the hypothetical circumstances under which it fails. This is indeed the angular stone for this research work: to undertake an adversarial analysis of a Deep Learning model. The proposed framework constructs counterfactual examples by ensuring their plausibility, e.g. there is a reasonable probability that a human could generate them without resorting to a computer program. Therefore, this work must be regarded as valuable auditing exercise of the usable bounds a certain model is constrained within, thereby allowing for a much greater understanding of the capabilities and pitfalls of a model used in a real application. To this end, a Generative Adversarial Network (GAN) and multi-objective heuristics are used to furnish a plausible attack to the audited model, efficiently trading between the confusion of this model, the intensity and plausibility of the generated counterfactual. Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
BibTeX
@inproceedings{arrieta2020plausible,
  author = {Alejandro Barredo Arrieta and Javier Del Ser},
  title = {{Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples}},
  booktitle = {IJCNN},
  year = {2020},
  url = {https://openreview.net/forum?id=m9ouaAFwaE},
}
Figure