The Good, the Bad, and the Explainer: A Tool for Contrastive Explanations of Text Classifiers

Lorenzo Malandri, Fabio Mercorio, Mario Mezzanzanica, Navid Nobani, Andrea SevesoIJCAI 2022 · DOI

TL;DR

This paper introduces ContrXT, a novel tool that explains why a text classifier's predictions change after retraining. It achieves this by comparing the logic of two models using symbolic reasoning (Binary Decision Diagrams) and generating human-readable explanations, making model evolution more transparent.

What Problem Does This Paper Address?

As machine learning models, especially text classifiers, are increasingly deployed, understanding their decision-making process becomes crucial. When a model is retrained with new data, its underlying logic can shift, potentially leading to unexpected or contradictory outcomes for users. This raises a "T-contrast" question: "Why did the model classify this way before, but now it classifies differently?" Existing explainability tools often focus on explaining a single model's predictions, but there's a gap in tools that specifically address and quantify the differences in logic between two versions of a model over time. This lack of transparency hinders trust and collaborative decision-making between humans and AI systems.

Methods

ContrXT tackles this problem through a five-step process:

  1. Text Classifiers: It takes two black-box text classifiers, $\psi_1$ and $\psi_2$, trained on the same task and class set, along with their respective training data.
  2. Post-hoc Interpretation (Surrogates): To make the black-box models interpretable, ContrXT generates global, rule-based surrogate models (e.g., decision trees or logistic regressions). These surrogates approximate the behavior of the original classifiers over the entire training dataset. A key aspect here is balancing the surrogate's fidelity (how well it mimics the black-box model) with its simplicity (to ensure understandability).
  3. Trace: The logic of the surrogate models is then encoded into Binary Decision Diagrams (BDDs). BDDs are a compact and efficient way to represent Boolean functions, allowing for fast logical operations.
  4. Explain: This is the core of ContrXT. It uses BDD operations to compute the differences between the two models' logic. Specifically, it generates BDDs that represent classification paths that are satisfied by one model but falsified by the other. These are termed "added" and "deleted" paths.
  5. Generation of Final Explanations: The BDDs are translated into two forms of explanations:
  6. Indicators: Quantitative metrics (Add and Del values) that summarize the extent of changes in classification logic.
  7. Natural Language Explanations (NLE): Human-readable text that describes the specific features and rules that have been added or removed in the new model's logic, making the changes understandable to end-users.

Datasets

The primary dataset used for evaluation is 20newsgroup. This is a well-established benchmark for text classification tasks and has been used in prior work for evaluating explainability methods. The paper notes that they used data from [Jin et al., 2016] for training the models.

Key Insights and Ideas

  • Focus on Change: The fundamental insight is that explaining how a model has changed is often more valuable than just explaining its current state, especially in dynamic environments where models are frequently updated.
  • Symbolic Reasoning for Comparison: Using BDDs to represent classifier logic allows for precise, formal comparison of decision boundaries, going beyond simple feature importance.
  • Bridging the Gap: ContrXT aims to bridge the gap between complex model logic and human understanding by generating both quantitative indicators and qualitative natural language explanations.
  • Model-Agnostic Approach: The method is designed to work with any black-box text classifier, making it broadly applicable.

Key Results

  • Surrogate Fidelity: ContrXT achieved high surrogate fidelity, with F1-weighted scores up to 0.87 and never below 0.73 when evaluated on the 20newsgroup dataset across various machine learning algorithms (LR, RF, SVM, NB, bi-GRU, BERT).
  • Human Evaluation: A user study with 15 participants showed that they understood the NLE explanations with an average accuracy of 89% and an F1-score of 87%. Krippendorff's alpha coefficient of 0.7 indicated good inter-user agreement.
  • API Performance: The ContrXT API demonstrated a throughput of 2.55 users per second, with additional requests being queued, indicating reasonable scalability.
  • Qualitative Examples: The paper provides concrete examples (e.g., Figure 3 for the "atheism" class with BERT) showing how NLEs highlight specific feature changes (like "bill" vs. "keith") that explain shifts in classification.

Limitations and Future Work

  • Current Scope: ContrXT is currently limited to explaining text classifiers.
  • Future Work: The authors are actively working on extending ContrXT to handle tabular classifiers, which would significantly broaden its applicability.

Summary updated

BibTeX
@inproceedings{malandri2022good,
  author = {Lorenzo Malandri and Fabio Mercorio and Mario Mezzanzanica and Navid Nobani and Andrea Seveso},
  title = {{The Good, the Bad, and the Explainer: A Tool for Contrastive Explanations of Text Classifiers}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/858},
}
Figure