Learning by Interpreting

Xuting Tang, Abdul Rafae Khan, Shusen Wang, Jia XuIJCAI 2022 · DOI

TL;DR

This paper proposes a novel approach to enhance NLP model accuracy by incorporating interpretability insights as additional input features, rather than treating accuracy and interpretability as a trade-off. By feeding learned explanations from methods like attention, LIME, and SHAP back into the model, they demonstrate significant improvements in NMT and GLUE tasks, suggesting that interpretability can actively aid learning.

What Problem Does This Paper Address?

Deep learning models have achieved state-of-the-art performance in Natural Language Processing (NLP), but their "black-box" nature makes it difficult to understand why they make certain predictions. This lack of transparency hinders model improvement and debugging. Traditionally, efforts to make models more interpretable often involve sacrificing predictive accuracy, creating a dilemma between performance and explainability. This paper challenges this dichotomy, asking: can interpretability actually improve model accuracy? They hypothesize that by providing models with insights into their own decision-making process, learning can become more efficient and effective.

Methods

The core of this work is the "interpretability enhanced learning" paradigm. The authors propose a three-step process:

  1. Train a Baseline Model: First, a standard NLP model is trained on the given task.
  2. Extract Interpretability Features: Using established interpretability methods, the model's "reasoning" is analyzed. The paper explores three main categories:
  3. Attention Mechanisms: For models like Transformer and ConvS2S, attention weights are used to identify which input words are most influential for generating each output word. These influential words (and their positions) are extracted.
  4. LIME (Local Interpretable Model-agnostic Explanations): This post-hoc method generates local explanations for individual predictions. For text, it assigns importance scores to words, and the word with the highest score is selected as the "important word."
    • SHAP (SHapley Additive exPlanations): Another post-hoc method that uses Shapley values from game theory to assign importance to each feature. Similar to LIME, the word with the highest Shapley value is identified.
  5. Incorporate Features and Retrain: The extracted interpretability features (e.g., attention words, important words) are concatenated with the original input sequences. This augmented input is then used to fine-tune the baseline NLP model.

This process is applied to both training and testing phases, with a specific strategy for extracting features during the test mode to avoid influencing the model's original behavior. They also introduce a "bootstrapping" approach where the fine-tuned model is further improved by extracting interpretability features from it and retraining again.

Datasets

The paper evaluates its approach on two main types of NLP tasks:

  • Machine Translation (NMT):
    • WMT 2014 News (German-English): A large-scale dataset.
  • IWSLT 2017 TEDTalk (French-English): A medium-scale dataset. Both datasets are pre-processed using standard tools like Moses tokenizer and Byte-Pair Encoding (BPE).

  • GLUE Benchmark: A collection of diverse NLP tasks used for evaluating general language understanding. The paper specifically mentions using QQP, COLA, STS-B, and RTE from this benchmark. For GLUE tasks, they fine-tune a BERT-base model.

Key Insights and Ideas

  • Interpretability as a Learning Aid: The central idea is that interpretability is not just for human understanding but can actively improve model performance. By showing the model what it should focus on, learning becomes more efficient.
  • Analogy to Human Learning: The paper draws parallels to human learning, where highlighting important parts of a text aids comprehension and retention.
  • XOR Function Example: A simple XOR function, which is non-linearly separable, becomes linearly separable and easier to learn when an interaction term (x1 * x2) is added as an input feature. This illustrates how adding relevant derived information can simplify learning.
  • Unified Goals: The research suggests that accuracy and interpretability are not mutually exclusive but can be unified and mutually beneficial.
  • Quality of Explanation Matters: An ablation study shows that adding random noise to the extracted explainable features degrades performance, highlighting that the quality of the interpretation is crucial for the accuracy gains.

Key Results

The proposed "interpretability enhanced learning" paradigm led to significant improvements across various NLP tasks:

  • Neural Machine Translation (NMT):
    • WMT 2014 DE-EN: Up to +1.4 BLEU points improvement on both ConvS2S and Transformer models.
  • IWSLT 2017 FR-EN: Up to +2.8 BLEU points for ConvS2S and +3.4 BLEU points for Transformer.
  • Bootstrapping further yielded an additional +0.4 BLEU points on IWSLT'17 FR-EN.
  • GLUE Benchmark:
    • Incorporating attention words achieved up to +2.3 points improvement.
  • Incorporating LIME/SHAP important words achieved up to +4.8 points improvement over the baseline BERT-base model.

Limitations and Future Work

  • Computational Cost: While the training time itself is comparable, the extraction of interpretability features (especially for LIME/SHAP) adds computational overhead.
  • Generalizability of Interpretability Methods: The paper uses established methods (attention, LIME, SHAP), but the effectiveness might depend on the specific task and model architecture.
  • Future Work: The authors suggest that this paradigm can be applied to various NLP tasks and models. The bootstrapping approach shows promise for iterative improvement. Further research could explore more sophisticated ways to integrate interpretability insights and investigate their impact on other aspects like model robustness and fairness.

Summary updated

BibTeX
@inproceedings{tang2022learning,
  author = {Xuting Tang and Abdul Rafae Khan and Shusen Wang and Jia Xu},
  title = {{Learning by Interpreting}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/609},
}
Figure