Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales

Lucas Resck, Marcos M. Raimundo, Jorge PocoNAACL 2024 · DOI

TL;DR

This paper introduces a novel method to improve the plausibility of text classifier explanations by incorporating human-provided rationales during training. By using a multi-objective optimization approach, they demonstrate that it's possible to achieve more human-understandable explanations with minimal impact on model performance.

What Problem Does This Paper Address?

As text classification models become more complex, understanding why they make certain predictions becomes increasingly difficult. While "post-hoc" explanation methods (like saliency maps) can highlight important words, these explanations often don't align with human intuition, making them "implausible." This lack of plausibility hinders trust and adoption. The authors aim to bridge this gap by making these explanations more intuitive and convincing to humans, without sacrificing the model's accuracy.

Methods

The core of the paper is a new contrastive rationale loss function. This loss function is integrated into the training process alongside the standard cross-entropy loss for classification. The key idea is to train the model not just to classify text, but also to pay attention to human-provided "rationales" (text snippets that explain human decisions).

Here's how it works:

  • Rationales as Ground Truth: Human-annotated rationales are treated as the "correct" explanations.
  • Contrastive Learning: The model is trained to maximize its prediction confidence for the actual rationale while minimizing it for randomly selected "negative" rationales. This encourages the model to learn to focus on the human-defined important parts of the text.
  • Multi-Objective Optimization (MOO): To balance the trade-off between classification performance (cross-entropy loss) and explanation plausibility (contrastive rationale loss), the authors employ a MOO solver. This allows them to explore a "Pareto-frontier" of models, offering various options that balance these two competing objectives.
  • Model and Explainer Agnostic: Crucially, their approach doesn't require modifying the model architecture or assuming a specific type of explanation function, making it broadly applicable.

Datasets

The authors evaluated their method on several publicly available datasets:

  • HateXplain: Annotated hate speech detection data with human rationales.
  • Twitter Sentiment Extraction (TSE): Sentiment analysis data with human rationales.
  • Movie Reviews: Sentiment analysis data with human rationales.

They also mention using a related dataset, HatEval, for out-of-distribution testing.

Key Insights and Ideas

  • Plausibility is Achievable Without Major Performance Hits: The most significant finding is that it's possible to significantly improve explanation plausibility (measured by AUPRC) with only a minor decrease in model accuracy. This challenges the notion that there's an unavoidable, large trade-off.
  • Rationales as "Teaching Signals": Human rationales can effectively guide a model's internal decision-making process to produce more intuitive explanations.
  • MOO is Key to Navigating Trade-offs: The multi-objective optimization framework is essential for systematically exploring and identifying models that strike a good balance between performance and explanation quality.
  • "Bad" Explanations Can Be Fixed: The method shows particular promise in improving explanations that were initially poor, turning them into more plausible ones.

Key Results

  • Improved Plausibility: The proposed method consistently improved explanation plausibility (measured by AUPRC) across various models and datasets. For instance, in one experiment, AUPRC increased by 1.1% (absolute) with only a 0.8% decrease in accuracy.
  • Minimal Performance Degradation: While there was a slight decrease in accuracy in some cases, it was generally negligible (often less than 1%).
  • Enhanced Faithfulness: The method also showed improvements in explanation faithfulness (sufficiency), meaning the explanations more accurately reflected the model's reasoning.
  • Robustness: The approach demonstrated robustness across different models (DistilBERT, BERT-Mini, TF-IDF with Logistic Regression, BERT-Large), explainability methods (LIME, SHAP), and datasets.

Limitations and Future Work

  • Computational Cost: The MOO approach can be computationally intensive, requiring thousands of inferences per sample.
  • Model Scaling for Long Texts: Transformer-based models used in the study have input length limitations, which required truncating longer texts.
  • Dataset Availability: The availability of large, diverse datasets with human annotations remains a bottleneck for applying such methods broadly.
  • Future Work: The authors suggest exploring:
  • Training the entire model (not just the classifier layer) for potentially greater benefits.
  • Investigating how to reduce network "shortcuts" that lead to implausible reasoning.
  • Conducting large-scale user studies to evaluate trust and understandability.
  • Extending the approach to other explainers, datasets, and model types.

Summary updated

BibTeX
@inproceedings{resck2024exploring,
  author = {Lucas Resck and Marcos M. Raimundo and Jorge Poco},
  title = {{Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-naacl.262},
}
Figure