On Black-Box Explanation for Face Verification

Domingo Mery, Bernardita MorrisWACV 2022

TL;DR

This paper introduces a novel set of "black-box" explanation methods for face verification systems, meaning they don't require access to the internal workings of the face recognition model. By analyzing how the matching score changes when parts of a probe image are altered, the authors generate saliency maps that highlight the most important facial regions for recognition, offering a stable and interpretable explanation.

What Problem Does This Paper Address?

Face verification systems, which aim to confirm if two images depict the same person, have become incredibly accurate. However, they often operate as "black boxes," making it difficult to understand why they make a particular decision. This lack of interpretability hinders our ability to trust these systems, debug them, and understand their potential failure modes. While many explanation techniques exist for deep learning models, they typically require access to the model's internal layers (e.g., gradients), which is often not possible for commercial or proprietary systems. This paper addresses this gap by proposing methods that explain face verification systems without needing to peek inside the model, treating them as true black boxes.

Methods

The core idea behind the proposed methods is to probe the face verification model by perturbing the probe image and observing the effect on the matching score with a reference (enrolled) image. The authors introduce six different approaches:

  • Single Removal (S0-): This method systematically removes small circular regions from the probe image and measures the drop in the matching score. Regions whose removal causes a significant score decrease are deemed important.
  • Greedy Removal (S1-): This iteratively removes the most impactful region from the probe image in each step, building a saliency map based on the cumulative score changes.
  • Single Aggregation (S0+): This method aggregates (adds) small circular regions from the probe image onto a black background and measures the increase in the matching score. Regions whose aggregation leads to a higher score are considered important.
  • Greedy Aggregation (S1+): Similar to Greedy Removal, this iteratively aggregates the most impactful region, building a saliency map based on score increases.
  • Sequential Removal/Aggregation (SEQ): This method combines Greedy Aggregation and Greedy Removal in a sequence to find image modifications that result in a matching score very close to the original.
  • Average Removal/Aggregation (AVG): This approach averages the saliency maps from the first four methods (S0-, S1-, S0+, S1+) to provide a more robust and comprehensive explanation.

A novel visualization technique is also proposed: contour visualization. Instead of overlaying heatmaps, which can obscure image details, this method superimposes contours representing boundaries of saliency values, highlighting important areas without altering the original image significantly.

Datasets

The paper uses the following datasets:

  • MORPH dataset: Used for experiments with synthetic images of varying quality and occlusions, as well as for analyzing explanations across different demographic groups.
  • VGGface2 and Casia-WebFace: These are standard datasets used for training face verification models, specifically mentioned in the context of the FaceNet algorithm.
  • Angela Merkel face images: A custom set of 18 images of the same person with different expressions, poses, and occlusions, used for evaluating explanations on real-world variations.

No new benchmarks were introduced; instead, existing datasets were used to evaluate the proposed explanation methods.

Key Insights and Ideas

  • Black-box explainability is feasible and valuable: The paper demonstrates that effective explanations can be generated for face verification systems without internal model access, opening up possibilities for explaining commercial systems.
  • Perturbation-based analysis is a powerful tool: By systematically altering parts of an image and observing the impact on the matching score, one can infer the importance of those parts to the model.
  • Saliency maps can reveal algorithmic biases: The explanations for the LBP method, for instance, show it focusing on irrelevant areas, suggesting potential issues with its decision-making process compared to deep learning methods.
  • Contour visualization offers a cleaner explanation: This novel visualization method provides a less intrusive way to highlight important facial regions, preserving image details better than traditional heatmaps.
  • Deep learning models generally focus on relevant facial areas: The experiments suggest that modern deep learning-based face verifiers tend to concentrate on the periocular region and nose, indicating a more robust understanding of facial features.
  • Explanations are largely consistent across demographic groups: While different face verification methods produce different saliency maps, the patterns of importance within those maps are surprisingly similar across various demographic groups, suggesting that the core facial features used for recognition are consistent.

Key Results

  • The proposed AVG method combined with contour visualization (AVGC) is highlighted as achieving stable and interpretable saliency maps, focusing on relevant facial areas.
  • Experiments on synthetic and real-world images with varying quality, poses, and occlusions showed that the proposed methods effectively identify important facial regions, even when parts of the face are obscured.
  • When comparing five different face verification algorithms (ArcFace, Dlib, FaceNet, LBP), the deep learning methods (ArcFace, FaceNet, Dlib) produced more interpretable saliency maps than LBP, which often focused on irrelevant areas.
  • Analysis across demographic groups revealed that while different algorithms have distinct explanation patterns, the core areas of focus for recognition are consistent across these groups.

Limitations and Future Work

  • Computational Cost: While the paper provides approximate computational times, some methods (like SEQ) can be more time-consuming. Future work could focus on optimizing these methods for speed.
  • Parameter Sensitivity: The performance of the saliency maps can depend on the chosen parameters (e.g., Gaussian kernel width, step size, number of iterations). The paper mentions parameters were set manually, suggesting potential for automated tuning or sensitivity analysis.
  • Generalizability of "Black-Box": While the methods are designed for black-box systems, their effectiveness might still depend on the specific output provided by the model (e.g., a raw score vs. an embedding).
  • Further Evaluation: The authors suggest that their approach can be used to evaluate commercial off-the-shelf face recognition systems, implying a need for broader testing on such systems.
  • Quantitative Evaluation of Explanations: While qualitative assessments are provided, a more rigorous quantitative evaluation of the "goodness" or accuracy of the saliency maps themselves could be a future direction.

Summary updated

BibTeX
@inproceedings{mery2022black,
  author = {Domingo Mery and Bernardita Morris},
  title = {{On Black-Box Explanation for Face Verification}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure