Contrastive Explanations of Text Classifiers as a Service

Lorenzo Malandri, Fabio Mercorio, Mario Mezzanzanica, Navid Nobani, Andrea SevesoNAACL 2022

TL;DR

This paper introduces ContrXT, a novel service that provides "time-contrastive" explanations for text classifiers. It helps users understand how a model's classification logic changes over time by comparing two versions of the same classifier, offering insights into the evolution of its decision-making process. This is crucial for building trust and ensuring fairness in AI systems that adapt to new data.

What Problem Does This Paper Address?

As machine learning models, particularly "black-box" ones, are retrained with new data, their decision-making logic can subtly shift. This can lead to outcomes that seem contradictory to users, who are left wondering why a previously classified item is now treated differently. The core problem is the lack of transparency into why these changes occur, making it difficult to trust or debug evolving AI systems. This is especially critical in sensitive domains where fairness and consistency are paramount. The paper frames this as a "T-contrast" question: "Why does object A have property P at time t₁, but property Q at time t₂?"

Methods

ContrXT tackles this problem through a multi-step process:

  1. Post-hoc Interpretation: It first approximates the black-box classifiers (ψ₁ and ψ₂) with interpretable, rule-based "surrogate" models (pg₁ and pg₂). This is done globally, meaning the surrogate aims to mimic the black-box model's behavior across the entire dataset. A key innovation here is the addition of a complexity constraint (Ω(pg) ≤ Γ) to keep the surrogate models understandable.
  2. Trace: The logic of these surrogate models is then encoded into Binary Decision Diagrams (BDDs). BDDs are a compact and efficient way to represent Boolean functions, allowing for precise logical operations.
  3. eXplain: This is the core of the contrastive explanation. ContrXT manipulates the BDDs of the two classifiers to identify and quantify the differences in their classification logic. It generates BDDs that represent rules added or deleted between the two time points.
  4. Generation of Final Explanations: The differences identified in the BDDs are translated into human-readable formats:
  5. Indicators: Quantitative metrics (Add and Del values) that estimate the extent of changes in classification paths.
  6. Natural Language Explanations (NLE): Textual descriptions of the added and deleted classification rules, making the changes understandable to end-users. The system leverages NLG techniques for this.
  7. Rule Examples: The system can also provide concrete examples of data instances that are affected by specific added or deleted rules, further illustrating the changes.

ContrXT is offered as a service via a REST API, making it accessible to users without requiring them to install complex software.

Datasets

The primary dataset used for evaluation is 20newsgroups, a well-known benchmark for text classification. The paper mentions using this dataset in the context of evaluating classifiers trained on it, as well as for demonstrating ContrXT's capabilities. The authors also refer to prior work that used this dataset for building reproducible text classifiers and evaluating explanation methods.

Key Insights and Ideas

  • Bridging the Gap in Evolving Models: The most significant insight is the development of a method to explicitly track and explain how a classifier's logic evolves over time. This moves beyond static explanations to dynamic ones, crucial for systems that learn and adapt.
  • BDDs as a Powerful Tool for Contrastive Analysis: The paper highlights the effectiveness of BDDs in precisely encoding and comparing complex logical rules between two models. Their canonical form ensures that equivalent functions are represented identically, facilitating robust comparisons.
  • Beyond Accuracy: Understanding Why Performance Changes: ContrXT focuses on explaining changes in classification logic, not just overall performance metrics like F1-score. This is vital for understanding the root causes of performance shifts, whether positive or negative.
  • Making Explanations Actionable: By providing both quantitative indicators and qualitative natural language explanations, along with example instances, ContrXT aims to make the insights actionable for users, enabling them to trust, debug, or improve their models.

Key Results

  • Quantitative Evaluation: The paper presents a load test showing the ContrXT API can handle a throughput of 2.55 users per second before requests are queued, demonstrating its scalability as a service.
  • Surrogate Model Fidelity: Table 1 shows that ContrXT's surrogate models (decision trees) achieve high fidelity (F1-w scores around 0.76-0.78) when approximating various black-box classifiers (LR, RF, SVM, NB, bi-GRU, BERT) on the 20newsgroups dataset.
  • Qualitative Evaluation (Human Subjects): A user study with 15 participants showed that they understood the NLE format with 89% accuracy and achieved an F1-score of 87%. The inter-rater agreement, measured by Krippendorff's alpha, was 0.7, considered acceptable.
  • Demonstration of Contrastive Explanations: Figures 4, 5, 6, and 7 provide concrete examples of how ContrXT visualizes and explains changes in classification rules for specific classes (e.g., alt.atheism, Systems Analysts) across different models (BERT, RF). These examples clearly illustrate how new terms become important (e.g., "Bill" for alt.atheism) or how occupations are re-categorized (e.g., "business analysts" as "Systems Analysts").

Limitations and Future Work

  • Current Scope: ContrXT is currently limited to explaining text classifiers.
  • Future Work: The authors are actively working to extend ContrXT to tabular classifiers. They also mention ongoing efforts to improve the system and its explanations.

Summary updated

BibTeX
@inproceedings{malandri2022contrastive,
  author = {Lorenzo Malandri and Fabio Mercorio and Mario Mezzanzanica and Navid Nobani and Andrea Seveso},
  title = {{Contrastive Explanations of Text Classifiers as a Service}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure