Beyond Model Interpretability: On the Faithfulness and Adversarial Robustness of Contrastive Textual Explanations

Julia El Zini, Mariette AwadEMNLP 2022 · DOI

Introduction

As machine learning models are increasingly deployed in high-stakes domains, the demand for explainability has shifted from simple transparency to actionable recourse. Traditional Explainable AI (ExAI) methods for text typically rely on feature attribution—heatmaps that highlight specific words (e.g., "slow" or "bad") to explain a sentiment classifier's decision. While useful, these methods fail to provide users with a path to change the outcome.

Contrastive explanations, or counterfactuals, address this by generating alternative inputs (e.g., changing "The service was slow" to "The service was prompt") that flip the model's prediction. However, the utility of these explanations depends entirely on their faithfulness: are they realistic, logically connected to the original input, and robust to minor perturbations? This paper introduces a novel framework to evaluate these properties, revealing that generative methods operating in latent spaces offer significant advantages over traditional text-level search methods.

Evaluating Faithfulness: A Three-Pillar Framework

The authors propose three novel metrics to quantify the quality of textual counterfactuals beyond simple edit distance:

  1. Proximity: This measures how closely a generated counterfactual resembles "real" data from the target class. Using Local Reachability Density (LRD), the framework determines if the counterfactual is an outlier or a plausible data point within the distribution.
  2. Connectedness: Inspired by density-based clustering (DBSCAN), this metric evaluates whether a continuous path of high-probability data points exists between the original sentence and its counterfactual. This ensures the explanation represents a logical transition rather than a disjointed "jump" in the data manifold.
  3. Stability: This assesses algorithmic consistency. If minor, semantically insignificant changes to the input sentence lead to radically different counterfactuals, the explanation generator is considered unstable and potentially untrustworthy.

Benchmarking Recourse Methods

The study benchmarks two prominent approaches using the IMDB Sentiment Analysis and Kaggle Restaurant Reviews datasets:

  • MiCE: A search-based method that masks text segments and optimizes word combinations to achieve a label flip.
  • POLYJUICE: A generative approach that leverages a pre-trained language model (GPT-2) and control codes to produce counterfactuals.

Proximity and Realism

One of the most critical findings is the "Latent Advantage." POLYJUICE, by operating within the latent space of a language model, produces counterfactuals that are significantly more "in-distribution" than those generated by MiCE.

Proximity distributions for POLYJUICE (a) and MiCE (b). Lower scores indicate counterfactuals closer to the real data distribution.

As shown in the proximity distributions, POLYJUICE exhibits a strong concentration of low scores, indicating that its outputs are statistically similar to ground-truth data. In contrast, MiCE's outputs are more widely distributed, often resulting in counterfactuals that are grammatically correct but statistically improbable.

The Connectedness Challenge

The evaluation of connectedness revealed a pervasive "Island Problem." Many counterfactuals, while successful in flipping a model's label, are essentially isolated points in the data distribution.

Percentage of connected counterfactuals as a function of neighborhood size k.

When the neighborhood requirement ($k$) is strict, only about 50% of counterfactuals from either model are considered "connected" to their original counterparts. This suggests that current methods often find shortcuts to flip labels rather than following a smooth, actionable path through the data space.

Adversarial Robustness and Stability

The paper also presents the first semantic adversarial attack on recourse methods. By slightly perturbing the input sentence, the authors tested whether the explanation generators would break. MiCE proved highly vulnerable; under attack, its counterfactuals became significantly less realistic. POLYJUICE, however, remained remarkably resilient. Its stability is further evidenced by a tight linear correlation between input similarity and counterfactual similarity, ensuring that similar inputs receive similar explanations.

Conclusion and Future Directions

Faithfulness in textual explanations is multi-faceted. While minimal edit distance has been the standard metric, it is insufficient for ensuring that recourse is realistic or actionable. The results suggest that future counterfactual generators should move away from direct word-swapping and instead incorporate "connectedness constraints" directly into their search strategies. By ensuring that explanations are not just interpretable but also faithful to the underlying data manifold, we can provide users with truly reliable paths for algorithmic recourse.

Summary updated

BibTeX
@inproceedings{zini2022beyond,
  author = {Julia El Zini and Mariette Awad},
  title = {{Beyond Model Interpretability: On the Faithfulness and Adversarial Robustness of Contrastive Textual Explanations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.100},
}
Figure